Every function is tagged with a set of effects based on the built-in functions it calls. The mutation engine uses this classification to skip flaky mutants, seal nondeterminism at test time, and compute an effect stability score that feeds into the integrity grade.
For an overview of all advanced features, see Advanced overview.
The engine uses fixpoint analysis to propagate effects through the call graph. Each function receives one or more effect tags:
| Effect | Triggered by | Example built-ins |
|---|---|---|
PURE | No side effects | abs, len, str, sorted, range |
IO | Console/output | print, println, input |
TIME | Clock access | time, sleep |
RANDOM | Nondeterminism | rand_int, rand_seed |
NETWORK | HTTP calls | _http_request |
FS | File system | read_text, write_text, list_dir, mkdir |
PROCESS | OS interaction | exit, getenv, argv, spawn |
Effects propagate transitively: a function that calls a RANDOM function is also tagged RANDOM.
Mutants inside functions tagged TIME, RANDOM, or NETWORK are skipped by default because they produce flaky results:
Skipped 3 nondeterministic mutants (use --unstable to include)
For RANDOM-effect functions, the engine sets a deterministic seed before running each mutant. This makes rand_int() repeatable and rand_seed() a no-op during mutation testing, so RANDOM mutants produce stable results even without --unstable.
The effect stability metric measures the fraction of functions with stable (non-flaky) effects. It contributes 20% to the integrity score.
The integrity score is a weighted composite of four metrics:
| Component | Weight | Source |
|---|---|---|
| Mutation score | 40% | Fraction of mutants killed |
| Contract adequacy | 20% | Fraction of contract obligations defended |
| Oracle coverage | 20% | Fraction of tests with strong oracles |
| Effect stability | 20% | Fraction of functions with stable effects |
The score produces a letter grade:
| Grade | Score range |
|---|---|
| A | 90 to 100 |
| B | 75 to 89 |
| C | 60 to 74 |
| D | 40 to 59 |
| F | 0 to 39 |
The integrity score is included in mutation certificates and the manifest. Set a minimum:
[mutation]
min_integrity = 75.0 # Fail if integrity score < 75 (grade B)
When the threshold is set, vary mutate exits with code 1 if the integrity score falls below it. The --why output includes a component breakdown showing which area is weakest.
Effect classification runs automatically. To include nondeterministic mutants:
vary mutate source.vary --tests test.vary --unstable
To see which functions have which effects, use --observe:
vary mutate source.vary --tests test.vary --observe
| You see... | What to do |
|---|---|
| Mutants skipped as nondeterministic | These functions use TIME/RANDOM/NETWORK; use --unstable to include them or accept the filtering |
| Low effect stability | Many functions use nondeterministic built-ins; consider isolating side effects behind interfaces |
RANDOM mutants still flaky with sealing | The function may use external randomness not captured by rand_seed; quarantine or isolate |
| Integrity grade below threshold | Check the component breakdown: the weakest component tells you where to focus |
| Page | Topic |
|---|---|
| Signatures | How mutant identities survive refactors |
| Contracts | Contract adequacy contributes 20% to the integrity score |
| Oracle analysis | Oracle coverage contributes 20% to the integrity score |