CLI reference

Run vary vast to generate random programs and test the compiler. Flags control program count, language features, seed, coverage reporting, and CI modes.

Basic usage

vary vast

With no flags, this runs 100 programs using the core profile and a clock-based seed.

Options

Core options

Flag	Short	Default	Description
--profile	-p	core	Generation profile (core, control, complete, etc.)
--count	-n	100	Number of programs to generate and test
--seed	-s	clock	Base random seed (deterministic when set)
--verbose	-v	off	Print per-seed results
--timeout		5000	Per-program JVM execution timeout in milliseconds
--stop-on-first-mismatch		off	Stop after the first mismatch
--size		none	Program size tier: small, medium, or large

Mode and CI options

Flag	Default	Description
--mode	explore	CI mode: explore, fast, deep, continuous, or rc-blocking
--rotate-seed	off	Combine seed with SHA-256(date + commit) for daily variation
--commit-hash	auto-detect	Git commit hash for seed rotation
--metrics-file	.vast-logs/vast-metrics.jsonl	Path for JSONL metrics output
--regression-dir	none	Directory for regression artifacts (deep mode)
--duration	60	Time budget in seconds (continuous mode only)

Reduction and corpus options

Flag	Default	Description
--reduce	off	Shrink mismatches to minimal reproducers
--max-shrink	1000	Max oracle calls during reduction
--max-reduce-time	30000	Max reduction time in milliseconds
--corpus-dir	none	Directory for reproducer corpus
--replay-corpus	off	Replay saved corpus before exploration

Testing mode options

Flag	Default	Description
--metamorphic	off	Run metamorphic testing with semantics-preserving transforms
--mutate	off	Run mutation expansion testing
--mutants-per-seed	5	Max mutants to test per seed
--round-trip	off	Run parser round-trip validation
--opt-check	off	Enable 4-path optimizer differential validation
--stress	off	Enable stress testing with boundary values and edge cases
--calibrate	off	Run path calibration before testing

Specialized generator options

These flags enable specialized program generators that target specific compiler subsystems. They require --mode explore (not deep) because deep mode silently ignores specialized flags.

Flag	Default	Description
--stateful	off	Generate stateful programs with persistent mutable state
--state-steps	100	Number of state transition steps per program
--state-structs	1	Number of state struct fields to generate
--aliasing	off	Generate heap aliasing verification programs
--concurrency	off	Generate concurrency semantics test programs
--threads	4	Number of simulated concurrent threads
--exception-propagation	off	Generate exception propagation verification programs
--symbolic	off	Symbolic input guidance for deeper state coverage
--ir-check	off	Verify IR translation equivalence (AST to IR to JVM)
--verify-all-passes	off	Verify all optimizer passes independently
--verify-pass	none	Verify a specific optimizer pass
--large-programs	off	Generate large programs (500-10000 AST nodes)
--large-program-count	10	Number of large programs to generate

Acceptance and closeout options

Flag	Default	Description
--check-acceptance	off	Check acceptance-set coverage against the corpus
--write-acceptance-corpus	off	Write acceptance-set reproducers to the corpus directory
--closeout-report	off	Generate the VAST closeout report with evidence tiers, trigger quality table, and verdict
--show-lanes	off	Print the nightly lane registry and exit
--verify-config	off	Verify CI mode configuration invariants (profile lists, seeds, counts, lane registry)

Lane and metrics options

Flag	Default	Description
--lane	none	Lane name for nightly metrics tagging (e.g., vast-deep-differential)
--write-summary	off	Write a nightly summary JSON to the verification-history ledger
--run-class	nightly	Run class for summary: nightly, rc, or release
--prune-history	off	Remove nightly summaries beyond retention limit
--check-history	off	Check ledger for staleness

Corpus management options

Flag	Default	Description
--validate-corpus	off	Validate corpus entries against schema
--corpus-index	off	Generate INDEX.md listing all corpus entries
--history-summary	off	Generate SUMMARY.md from verification history

Coverage and reporting options

Flag	Default	Description
--show-coverage	off	Print feature coverage table
--semantic-coverage	off	Print semantic coverage table (27 properties)
--interaction-coverage	off	Print feature interaction coverage
--confidence	off	Print confidence report (LOW/MODERATE/HIGH/VERY_HIGH)
--path-health	off	Print path agreement matrix and reliability

Profiles

core

Straight-line programs only. No loops, no helper functions.

Constructs: integer and boolean literals, variables (let/mut), arithmetic (+, -, *, //, %), comparisons (==, !=, <, >, <=, >=), boolean logic (and, or, not), unary negation, if/else statements, if-expressions, return.

Fast execution. Use this for high-volume CI runs.

vary vast --profile core --count 1000 --seed 1

control

Adds helper functions and bounded while loops to everything in core.

Slower due to JVM compilation of larger programs. Use this for deeper coverage.

vary vast --profile control --count 100 --seed 1

Feature profiles

Profile	Adds
text	String literals, concatenation, comparison
enums	Enum declarations, variant construction, match
records	Data declarations, field access, construction
collections	List literals, indexing, len(), for..in
optional	T?, None, null narrowing
float	Float literals, arithmetic, int() conversion
patterns	Pattern matching with destructuring
errors	try/except, raise
generics	Generic functions and data types

Composite profiles

Profile	Includes
types	Core + all feature profiles (without patterns/errors/generics)
complete	Everything: all features enabled

CI modes

Mode	Programs	Profiles	Use case
explore	Configurable	1	Manual investigation, single profile. Required for specialized flags.
fast	~100	4	Quick smoke test, under 2 minutes
deep	~9,000	12	Nightly CI, broad coverage
continuous	Time-bounded	12 (adaptive)	Long-running exploration, overnight fuzzing
rc-blocking	~100	3	Release candidate gate with fixed seed

vary vast --mode fast                    # Quick smoke test
vary vast --mode deep --verbose          # Nightly
vary vast --mode continuous --duration 300  # 5-minute exploration

See CI integration for how these are used in practice.

Examples

CI smoke test

vary vast --profile core --count 200 --seed 12345

Deterministic and fast. Runs the same programs every time. Exit 0 on success, exit 1 on any mismatch.

Optimizer validation

vary vast --profile types --count 100 --seed 42 --opt-check

Runs four paths (AST, IR, JVM-optimized, JVM-unoptimized) to catch optimizer bugs.

Stress testing

vary vast --profile core --count 100 --seed 42 --stress

Generates programs with boundary values (Long.MAX_VALUE, 0, -1), overflow chains, identity operations, and float precision stress patterns.

Continuous exploration

vary vast --mode continuous --duration 600 --stress

Runs for 10 minutes, adaptively selecting profiles based on coverage gaps. Produces a report with per-profile summary, confidence score, and gap analysis.

Full coverage report

vary vast --profile complete --count 50 --seed 42 \
  --show-coverage --semantic-coverage --interaction-coverage --confidence

Replay a specific seed

vary vast --seed 842193 --count 1

Generates and tests exactly one program. The same seed always produces the same program and the same result.

Stop on first mismatch

vary vast --count 10000 --seed 1 --stop-on-first-mismatch

Stops as soon as a mismatch is found and prints the full mismatch report with replay command.

Exit codes

Code	Meaning
0	All programs agreed (or were invalid)
1	At least one mismatch, path failure, or equivalence failure

Output format

The summary line always appears:

VAST: 100/100 passed (0.3s)

If mismatches occur, each one is printed before the summary with the seed, verdict, path outcomes, generated source, and replay command.