Daily command sequences for Vary development.
The local development cycle:
check → test → fix → mutate (optional)
| Step | Command | What to look for |
|---|---|---|
| Check | vary check src/ | Type errors, lint warnings, effect violations |
| Test | vary test tests/ | Failing observe assertions |
| Fix | vary check src/ --fix | Auto-applied fixes (review the diff) |
| Mutate | vary mutate src/ --tests tests/ | Surviving mutants, gaps in test coverage |
Run check and test after every edit. Run mutate before opening a PR or when strengthening tests.
You changed src/store.vary and want to know if it's correct.
# 1. Type-check and lint
vary check src/store.vary
# 2. Run related tests
vary test tests/store_test.vary
# 3. If check offered auto-fixes
vary check src/store.vary --fix
Zero exit code from check and test means you're good. Check warnings (VCS/VCA codes) are advisory: read them, fix what matters. Test failures print the failing observe expression and expected vs actual values.
Everything should be clean before you push.
# 1. Run the local profile (analyze + static + tests)
vary validate src/ --profile local
# 2. Run mutation testing to find weak spots
vary validate src/ --profile ci
| Profile | Stages | When to use |
|---|---|---|
local | analyze, static, tests | Every PR, fast |
ci | analyze, static, tests, mutation | PR gate, thorough |
validate exits non-zero if any stage fails. Mutation score below threshold means tests don't catch enough: add assertions for surviving mutants. Use vary mutate src/ --why to understand why specific mutants survived.
The nightly CI runs the full verification suite including VAST oracle checks.
# 1. Reproduce locally with the nightly profile
vary validate src/ --profile nightly
# 2. If mutation stage failed, inspect survivors
vary mutate src/ --top 10
vary mutate src/ --why
# 3. If oracle stage failed, check VAST output
vary validate src/ --profile oracle
| Profile | Stages | When to use |
|---|---|---|
nightly | analyze, static, tests, mutation, oracle | Scheduled deep run |
oracle | analyze, static, oracle | Debugging oracle failures |
Nightly failures are usually surviving mutants or VAST oracle mismatches. --top N shows the N most impactful surviving mutants. --group clusters survivors by module and category for systematic fixing. Oracle failures mean generated test scenarios found unexpected behaviour: read the diff.
Profiles bundle stages so you don't have to remember flags:
vary validate src/ --profile <name>
| Profile | Stages | Typical context |
|---|---|---|
fast | analyze, static, tests | Quick local check |
local | analyze, static, tests | Alias for fast |
ci | analyze, static, tests, mutation | PR gate |
nightly | analyze, static, tests, mutation, oracle | Scheduled deep run |
mutation | analyze, static, mutation | Targeted mutation run |
mutation-strict | analyze, static, mutation (strict) | Hardening tests |
Custom profiles go in vary.toml:
[profiles.myteam]
checks = ["check", "test", "mutate"]
[profiles.myteam.thresholds]
mutation_score = 85
| Task | Command |
|---|---|
| Type-check one file | vary check src/foo.vary |
| Type-check everything | vary check src/ |
| Auto-fix lint issues | vary check src/ --fix |
| Run all tests | vary test tests/ |
| Run one test file | vary test tests/foo_test.vary |
| Mutation test | vary mutate src/ --tests tests/ |
| Show why mutants survived | vary mutate src/ --why |
| Top surviving mutants | vary mutate src/ --top 10 |
| Full local validation | vary validate src/ --profile local |
| PR validation | vary validate src/ --profile ci |
| Nightly validation | vary validate src/ --profile nightly |
| Generate service client stub | vary client src/server.vary |
| Scaffold serious project | vary new myapp --template serious |
| Format source | vary format src/ |