vary vast
With no flags, this runs 100 programs using the core profile and a clock-based seed.
| Flag | Short | Default | Description |
|---|---|---|---|
--profile | -p | core | Generation profile (core, control, complete, etc.) |
--count | -n | 100 | Number of programs to generate and test |
--seed | -s | clock | Base random seed (deterministic when set) |
--verbose | -v | off | Print per-seed results |
--timeout | 5000 | Per-program JVM execution timeout in milliseconds | |
--stop-on-first-mismatch | off | Stop after the first mismatch | |
--size | none | Program size tier: small, medium, or large |
| Flag | Default | Description |
|---|---|---|
--mode | explore | CI mode: explore, fast, deep, continuous, or rc-blocking |
--rotate-seed | off | Combine seed with SHA-256(date + commit) for daily variation |
--commit-hash | auto-detect | Git commit hash for seed rotation |
--metrics-file | .vast-logs/vast-metrics.jsonl | Path for JSONL metrics output |
--regression-dir | none | Directory for regression artifacts (deep mode) |
--duration | 60 | Time budget in seconds (continuous mode only) |
| Flag | Default | Description |
|---|---|---|
--reduce | off | Shrink mismatches to minimal reproducers |
--max-shrink | 1000 | Max oracle calls during reduction |
--max-reduce-time | 30000 | Max reduction time in milliseconds |
--corpus-dir | none | Directory for reproducer corpus |
--replay-corpus | off | Replay saved corpus before exploration |
| Flag | Default | Description |
|---|---|---|
--metamorphic | off | Run metamorphic testing with semantics-preserving transforms |
--mutate | off | Run mutation expansion testing |
--mutants-per-seed | 5 | Max mutants to test per seed |
--round-trip | off | Run parser round-trip validation |
--opt-check | off | Enable 4-path optimizer differential validation |
--stress | off | Enable stress testing with boundary values and edge cases |
--calibrate | off | Run path calibration before testing |
These flags enable specialized program generators that target specific compiler subsystems. They require --mode explore (not deep) because deep mode silently ignores specialized flags.
| Flag | Default | Description |
|---|---|---|
--stateful | off | Generate stateful programs with persistent mutable state |
--state-steps | 100 | Number of state transition steps per program |
--state-structs | 1 | Number of state struct fields to generate |
--aliasing | off | Generate heap aliasing verification programs |
--concurrency | off | Generate concurrency semantics test programs |
--threads | 4 | Number of simulated concurrent threads |
--exception-propagation | off | Generate exception propagation verification programs |
--symbolic | off | Symbolic input guidance for deeper state coverage |
--ir-check | off | Verify IR translation equivalence (AST to IR to JVM) |
--verify-all-passes | off | Verify all optimizer passes independently |
--verify-pass | none | Verify a specific optimizer pass |
--large-programs | off | Generate large programs (500-10000 AST nodes) |
--large-program-count | 10 | Number of large programs to generate |
| Flag | Default | Description |
|---|---|---|
--check-acceptance | off | Check acceptance-set coverage against the corpus |
--write-acceptance-corpus | off | Write acceptance-set reproducers to the corpus directory |
--closeout-report | off | Generate the VAST closeout report with evidence tiers, trigger quality table, and verdict |
--show-lanes | off | Print the nightly lane registry and exit |
--verify-config | off | Verify CI mode configuration invariants (profile lists, seeds, counts, lane registry) |
| Flag | Default | Description |
|---|---|---|
--lane | none | Lane name for nightly metrics tagging (e.g., vast-deep-differential) |
--write-summary | off | Write a nightly summary JSON to the verification-history ledger |
--run-class | nightly | Run class for summary: nightly, rc, or release |
--prune-history | off | Remove nightly summaries beyond retention limit |
--check-history | off | Check ledger for staleness |
| Flag | Default | Description |
|---|---|---|
--validate-corpus | off | Validate corpus entries against schema |
--corpus-index | off | Generate INDEX.md listing all corpus entries |
--history-summary | off | Generate SUMMARY.md from verification history |
| Flag | Default | Description |
|---|---|---|
--show-coverage | off | Print feature coverage table |
--semantic-coverage | off | Print semantic coverage table (27 properties) |
--interaction-coverage | off | Print feature interaction coverage |
--confidence | off | Print confidence report (LOW/MODERATE/HIGH/VERY_HIGH) |
--path-health | off | Print path agreement matrix and reliability |
Straight-line programs only. No loops, no helper functions.
Constructs: integer and boolean literals, variables (let/mut), arithmetic (+, -, *, //, %), comparisons (==, !=, <, >, <=, >=), boolean logic (and, or, not), unary negation, if/else statements, if-expressions, return.
Fast execution. Use this for high-volume CI runs.
vary vast --profile core --count 1000 --seed 1
Adds helper functions and bounded while loops to everything in core.
Slower due to JVM compilation of larger programs. Use this for deeper coverage.
vary vast --profile control --count 100 --seed 1
| Profile | Adds |
|---|---|
text | String literals, concatenation, comparison |
enums | Enum declarations, variant construction, match |
records | Data declarations, field access, construction |
collections | List literals, indexing, len(), for..in |
optional | T?, None, null narrowing |
float | Float literals, arithmetic, int() conversion |
patterns | Pattern matching with destructuring |
errors | try/except, raise |
generics | Generic functions and data types |
| Profile | Includes |
|---|---|
types | Core + all feature profiles (without patterns/errors/generics) |
complete | Everything: all features enabled |
| Mode | Programs | Profiles | Use case |
|---|---|---|---|
explore | Configurable | 1 | Manual investigation, single profile. Required for specialized flags. |
fast | ~100 | 4 | Quick smoke test, under 2 minutes |
deep | ~9,000 | 12 | Nightly CI, broad coverage |
continuous | Time-bounded | 12 (adaptive) | Long-running exploration, overnight fuzzing |
rc-blocking | ~100 | 3 | Release candidate gate with fixed seed |
vary vast --mode fast # Quick smoke test
vary vast --mode deep --verbose # Nightly
vary vast --mode continuous --duration 300 # 5-minute exploration
See CI integration for how these are used in practice.
vary vast --profile core --count 200 --seed 12345
Deterministic and fast. Runs the same programs every time. Exit 0 on success, exit 1 on any mismatch.
vary vast --profile types --count 100 --seed 42 --opt-check
Runs four paths (AST, IR, JVM-optimized, JVM-unoptimized) to catch optimizer bugs.
vary vast --profile core --count 100 --seed 42 --stress
Generates programs with boundary values (Long.MAX_VALUE, 0, -1), overflow chains, identity operations, and float precision stress patterns.
vary vast --mode continuous --duration 600 --stress
Runs for 10 minutes, adaptively selecting profiles based on coverage gaps. Produces a report with per-profile summary, confidence score, and gap analysis.
vary vast --profile complete --count 50 --seed 42 \
--show-coverage --semantic-coverage --interaction-coverage --confidence
vary vast --seed 842193 --count 1
Generates and tests exactly one program. The same seed always produces the same program and the same result.
vary vast --count 10000 --seed 1 --stop-on-first-mismatch
Stops as soon as a mismatch is found and prints the full mismatch report with replay command.
| Code | Meaning |
|---|---|
| 0 | All programs agreed (or were invalid) |
| 1 | At least one mismatch, path failure, or equivalence failure |
The summary line always appears:
VAST: 100/100 passed (0.3s)
If mismatches occur, each one is printed before the summary with the seed, verdict, path outcomes, generated source, and replay command.