Alpha. Vary is under active development and not ready for production use. Syntax, APIs, performance, and behaviour may change between releases.
Quick reference
Daily command sequences for Vary development.
The inner loop
The local development cycle:
check → test → fix → mutate (optional)
| Step | Command | What to look for |
|---|---|---|
| Check | vary check src/ | Type errors, lint warnings, effect violations |
| Test | vary test tests/ | Failing observe assertions |
| Fix | vary check src/ --fix | Auto-applied fixes (review the diff) |
| Mutate | vary mutate src/ --tests tests/ | Surviving mutants, gaps in test coverage |
Run check and test after every edit. Run mutate before opening a PR or when strengthening tests.
Flow: I edited one file
You changed src/store.vary and want to know if it's correct.
# 1. Type-check and lint
vary check src/store.vary
# 2. Run related tests
vary test tests/store_test.vary
# 3. If check offered auto-fixes
vary check src/store.vary --fix
Zero exit code from check and test means you're good. Check warnings (VCS/VCA codes) are advisory: read them, fix what matters. Test failures print the failing observe expression and expected vs actual values.
Flow: I'm about to PR
Everything should be clean before you push.
# 1. Run the local profile (analyze + static + tests)
vary validate src/ --profile local
# 2. Run mutation testing to find weak spots
vary validate src/ --profile ci
| Profile | Stages | When to use |
|---|---|---|
local | analyze, static, tests | Every PR, fast |
ci | analyze, static, tests, mutation | PR gate, thorough |
validate exits non-zero if any stage fails. Mutation score below threshold means tests don't catch enough: add assertions for surviving mutants. Use vary mutate src/ --why to understand why specific mutants survived.
Flow: nightly failed
The nightly CI runs the full verification suite including VAST oracle checks.
# 1. Reproduce locally with the nightly profile
vary validate src/ --profile nightly
# 2. If mutation stage failed, inspect survivors
vary mutate src/ --top 10
vary mutate src/ --why
# 3. If oracle stage failed, check VAST output
vary validate src/ --profile oracle
| Profile | Stages | When to use |
|---|---|---|
nightly | analyze, static, tests, mutation, oracle | Scheduled deep run |
oracle | analyze, static, oracle | Debugging oracle failures |
Nightly failures are usually surviving mutants or VAST oracle mismatches. --top N shows the N most impactful surviving mutants. --group clusters survivors by module and category for systematic fixing. Oracle failures mean generated test scenarios found unexpected behaviour: read the diff.
Verification profiles
Profiles bundle stages so you don't have to remember flags:
vary validate src/ --profile <name>
| Profile | Stages | Typical context |
|---|---|---|
fast | analyze, static, tests | Quick local check |
local | analyze, static, tests | Alias for fast |
ci | analyze, static, tests, mutation | PR gate |
nightly | analyze, static, tests, mutation, oracle | Scheduled deep run |
mutation | analyze, static, mutation | Targeted mutation run |
mutation-strict | analyze, static, mutation (strict) | Hardening tests |
Custom profiles go in vary.toml:
[profiles.myteam]
checks = ["check", "test", "mutate"]
[profiles.myteam.thresholds]
mutation_score = 85
Command cheat sheet
| Task | Command |
|---|---|
| Type-check one file | vary check src/foo.vary |
| Type-check everything | vary check src/ |
| Auto-fix lint issues | vary check src/ --fix |
| Run all tests | vary test tests/ |
| Run one test file | vary test tests/foo_test.vary |
| Mutation test | vary mutate src/ --tests tests/ |
| Show why mutants survived | vary mutate src/ --why |
| Top surviving mutants | vary mutate src/ --top 10 |
| Full local validation | vary validate src/ --profile local |
| PR validation | vary validate src/ --profile ci |
| Nightly validation | vary validate src/ --profile nightly |
| Generate service client stub | vary client src/server.vary |
| Scaffold serious project | vary new myapp --template serious |
| Format source | vary format src/ |