Alpha. Vary is under active development and not ready for production use. Syntax, APIs, performance, and behaviour may change between releases.
Observability
The mutation engine records per-test reads, writes, assertions, and branch entries during test execution. This data powers the --why diagnostics and survivor classification, turning "mutant survived" into an actionable explanation.
For an overview of all advanced features, see Advanced overview.
What it does
When you enable --observe, the ObservabilityAnalyzer classifies each surviving mutant into one of five causes:
| Cause | What it means |
|---|---|
| Unobserved side effect | Code writes a value but no test reads it |
| Not asserted value | Tests read the value but use a loose check (observe x > 0) instead of an exact one (observe x == 5) |
| Partial object assert | Tests check order.id but not order.status |
| Executed but unobserved | A branch runs but nothing checks its outcome |
| Well tested | Survived despite good observability (possible equivalent mutant) |
The runtime infrastructure uses thread-local IntArrayBuffer trace recording and trace hashing via VaryObserver.
What it outputs
Why explanations
The --why flag produces per-mutant explanations with zero internal vocabulary:
Mutant: ARITH_ADD_TO_SUB
Location: math.vary line 15, in calculate_total
Change: replaced + with -
Why it survived: Tests read the return value but only check > 0, not the exact amount
Fix: Add observe calculate_total(10, 5) == 15
Replay: vary mutate math.vary --replay m-0042
Differential detection
--differential compares trace hashes between baseline and mutant runs. When a test passes on both but the trace hash differs, the mutant changed observable behaviour that no assertion caught:
DIFFERENTIAL DETECTION
4 mutants changed behaviour but tests still passed
Affected tests: 3
These are high-value targets: the test noticed the change internally but no assertion checked it.
Structured assertion groups
VaryExpectGroup tracks field-level assertions for structured objects:
let group = ExpectGroup("order")
group.field("id", order.id, 7)
group.field("status", order.status, "FILLED")
group.check()
The observability analyzer uses group snapshots to detect assertion gaps: objects where some fields were asserted but others were missed.
How to use it
Enable observability with the --observe flag:
vary mutate source.vary --tests test.vary --observe
Combine with --why to get explanations for specific survivors:
vary mutate source.vary --tests test.vary --observe --why m-0042
Combine with --differential to find mutants that changed trace behaviour:
vary mutate source.vary --tests test.vary --observe --differential
How to interpret the results
| You see... | What to do |
|---|---|
| "Unobserved side effect" | Add an assertion on the side-effected value |
| "Not asserted value" | Tighten the assertion from a range check to an exact check |
| "Partial object assert" | Assert on the missing fields using ExpectGroup or individual observe |
| "Executed but unobserved" | Add an assertion that checks the outcome of the branch |
| "Well tested" | Likely an equivalent mutant; consider quarantining it |
| Differential detection hits | The test internals noticed the change, so add an observe for that value |
What to do next
| Page | Topic |
|---|---|
| Oracle analysis | Check oracle quality per test |
| Contracts | Use contracts as automatic oracles |
| Effects | Understand how effect classification filters flaky mutants |