Testing

Check rules

vary check is a rule-based diagnostic engine that catches non-idiomatic patterns, missing contracts, weak tests, and mutation-unfriendly code before you run vary mutate.

vary check src/main.vary
vary check src/
vary check src/ --fix
vary check src/ --category idioms
vary check src/ --rule VCI001
vary check src/ --json

Rule categories

Rules are grouped by category and identified by stable IDs. The prefix indicates the category.

PrefixCategoryFocus
VCIIdiomsIdiomatic Vary style
VCCContractsContract usage and quality
VCTTestingTest coverage and assertion strength
VCMMutationMutation-testing readiness
VCSSafetySide effects in logic
VCAAPIEffect boundaries and capability enforcement
VCLStructureProject-level architectural violations
VCGGreat codeDesign smells surfaced by vary check --great-code

Rules reference

Idioms

VCI001 none-comparison

Use is None / is not None instead of == None / != None.

Severity: suggestion | Auto-fix: yes

let x: Str? = None

# Before
if x == None {
    print("missing")
}

# After (fixed)
if x is None {
    print("missing")
}

is None expresses nil identity explicitly and avoids generic equality phrasing.

VCI002 bool-return

Avoid bare Bool return types on non-trivial functions.

Severity: suggestion | Auto-fix: no

# Flagged
def check_site(pages: List[Str]) -> Bool {
    for p in pages {
        if not fs.exists(fs.path(p)) {
            return False
        }
    }
    return True
}

A Bool hides what was checked, what failed, and what was skipped. Return a structured data type instead:

data CheckReport {
    success: Bool
    issues: List[Str]
    checked_files: List[Str]
}

def check_site(pages: List[Str]) -> CheckReport {
    mut issues: List[Str] = []
    for p in pages {
        if not fs.exists(fs.path(p)) {
            issues = issues + ["missing: " + p]
        }
    }
    return CheckReport(len(issues) == 0, issues, pages)
}

VCI003 exit-in-logic

Avoid calling exit() inside non-main functions.

Severity: warning | Auto-fix: no

import fs
import process

# Flagged
def validate(path: Str) {
    if not fs.exists(fs.path(path)) {
        process.exit(1)
    }
}

Move process termination to the CLI layer and return structured errors from inner logic:

import fs

def validate(path: Str) -> Result[Str, Str] {
    if not fs.exists(fs.path(path)) {
        return Err("file not found: " + path)
    }
    return Ok(path)
}

VCI004 shallow-observe

Avoid shallow observe statements that are too weak to kill mutants.

Severity: suggestion | Auto-fix: no

Detected patterns:

observe result is not None   # only checks existence
observe result == True        # only checks a boolean
observe x                     # bare identifier

Replace with field-level assertions:

observe report.success == True
observe report.issues.len() == 0
observe report.checked_files.len() == 5

VCI005 string-enum-smell

Data class Str fields named kind, mode, type, status, strategy, category, level, state, severity, stage, phase, role, action, or variant suggest a finite set of values that should be an enum.

Severity: suggestion | Auto-fix: no

# Flagged
data Issue {
    kind: Str
    message: Str
}

# Better
enum IssueKind {
    BrokenLink
    MissingFile
    InvalidConfig
}

data Issue2 {
    kind: IssueKind
    message: Str
}

VCI017 stale-workaround-comment

Comments containing workaround language (workaround, hack, temporary, fixme) may reference code that has already been fixed. The rule warns on every such comment so you can verify the workaround is still needed.

Severity: warning | Auto-fix: no

# Flagged
# workaround: retry once because the upstream service 500s intermittently
for _ in 0..2 {
    let result = fetch()
    if result.is_ok() { return result }
}

Either delete the comment (and simplify the code) if the upstream bug is fixed, or rewrite the comment to explain the lasting reason.

Contracts

VCC001 missing-contract

Non-trivial functions with a return type but no in {} or out {} blocks.

Severity: suggestion | Auto-fix: no

# Flagged
def parse_doc(md: Str) -> DocPage {
    let slug = extract_slug(md)
    let title = extract_title(md)
    let body = extract_body(md)
    return DocPage(slug, title, body)
}

# Better
def parse_doc(md: Str) -> DocPage {
    in {
        len(md) > 0
    }
    out (result) {
        result.slug != ""
    }
    let slug = extract_slug(md)
    let title = extract_title(md)
    let body = extract_body(md)
    return DocPage(slug, title, body)
}

Contracts provide validation, documentation, and automatic mutant-kill conditions.

VCC002 trivial-contract

Contracts that contain only True or other trivially satisfied expressions.

Severity: warning | Auto-fix: no

# Flagged
def process(x: Int) -> Int {
    in {
        True
    }
    return x + 1
}

Replace with a condition that expresses a real semantic obligation:

in {
    x > 0
}

Testing

VCT001 single-observe

Test blocks with only one observe statement are likely under-asserting.

Severity: suggestion | Auto-fix: no

# Flagged
test "plan has correct fields" {
    let plan = plan_deploy("prod", "/var/www")
    observe plan is not None
}

# Better
test "plan has correct fields" {
    let plan = plan_deploy("prod", "/var/www")
    observe plan.strategy == DeployStrategy.Rsync
    observe plan.source_dir == "build/"
    observe plan.destination == "/var/www"
}

VCT002 no-negative-test

Module has test blocks but none exercise error or failure paths (no observe throws blocks).

Severity: suggestion | Auto-fix: no

Add tests for invalid input, missing data, empty collections, or error cases:

test "rejects empty input" {
    observe throws { parse_doc("") }
}

test "handles missing file" {
    let result = load_doc("/does/not/exist")
    observe result.is_err()
}

Mutation

VCM001 pure-candidate

Function has no apparent side effects but is not marked pure def.

Severity: suggestion | Auto-fix: no

# Before
def classify_change(path: Str) -> ChangeKind {
    if path.ends_with(".css") {
        return ChangeKind.Css
    }
    return ChangeKind.Unknown
}

# After (fixed)
pure def classify_change(path: Str) -> ChangeKind {
    if path.ends_with(".css") {
        return ChangeKind.Css
    }
    return ChangeKind.Unknown
}

Pure functions are easier to test, contract, and mutate. The rule checks for known effectful calls (print, exit, write_text, read_text, run, etc.) and only flags functions that have none.

Safety

VCS001 print-in-logic

print() or println() used inside non-main functions hides observable behaviour.

Severity: suggestion | Auto-fix: no

import fs

# Flagged
def validate(paths: List[Str]) {
    for p in paths {
        if not fs.exists(fs.path(p)) {
            print("missing: " + p)
        }
    }
}

Return structured data instead so tests and mutation testing can verify it:

import fs

def validate(paths: List[Str]) -> List[Str] {
    mut missing: List[Str] = []
    for p in paths {
        if not fs.exists(fs.path(p)) {
            missing = missing + [p]
        }
    }
    return missing
}

API

VCA001 broad-effects

Function produces too many distinct side effect classes. Configure the threshold in vary.toml:

Severity: warning | Auto-fix: no

[check.analysis]
broad_effects_threshold = 3

A function that does IO, network, process execution, and state mutation likely has too many responsibilities. Split it into smaller functions with focused effects.

VCA002 denied-effect

Function produces an effect that is denied by project policy. Configure denied effects in vary.toml:

Severity: error | Auto-fix: no

[check.effects]
deny = ["PROCESS", "NETWORK"]

This enables "pure core" architectural patterns where domain modules are guaranteed effect-free.

VCA005 service-server-drift

Detect drift between service declarations (client) and expose statements (server). The rule compares endpoint names, HTTP methods, URL paths, parameter types, and return types.

Severity: error | Auto-fix: no

# Server defines:
# expose ItemService via http
#   → GET /item-service/item/{id}

# Client declares a mismatched path:
service ItemApi at "https://api.example.com" {
    endpoint get_item(id: Int) -> Str via GET "/items/{id}"
    # Error: path mismatch — server has /item-service/item/{id}
}

Warns if server methods are not declared as client endpoints.

VCS008 service-host-denied

Flag service declarations targeting hosts not in the project's allowed list or explicitly denied.

Severity: error | Auto-fix: no

Configure in vary.toml:

[services]
allowed_hosts = ["api.example.com", "*.internal.example.com"]
denied_hosts = ["untrusted.com"]

Wildcard patterns (*.example.com) match any subdomain. When allowed_hosts is non-empty, unlisted hosts are blocked.

Structure

Structure rules detect project-level architectural violations that cross file boundaries. They run when vary check is given a directory and analyze the whole project.

VCL001 effects-in-wrong-module

Effectful calls (I/O, network, filesystem) in modules under src/core/ or src/logic/, which should contain pure logic.

Severity: warning | Auto-fix: no

# File: src/core/validator.vary — Flagged
import http

def validate_url(url: Str) -> Bool {
    let resp = http_get(url)   # effectful call in core module
    return resp.status == 200
}

Move effectful calls to a handler or adapter module and pass data into the logic module instead.

VCL002 fixture-in-production

Test-fixture patterns (functions named make_*, fake_*, mock_*, stub_*, setup_*; variables named FIXTURE_*, TEST_*) in production code under src/.

Severity: warning | Auto-fix: no

# File: src/services/user_service.vary — Flagged
def make_test_user() -> User {
    return User("test", "test@example.com")
}

Move fixture helpers to a test/ or tests/ directory.

VCL003 generated-code-location

Files with generated-code markers (# generated or // generated in the first 5 lines) that are not inside a generated/ or gen/ directory.

Severity: warning | Auto-fix: no

# File: src/models/schema.vary — Flagged
# generated
data Schema {
    name: Str
    version: Int
}

Move generated files to a generated/ or gen/ directory to separate them from hand-written code.

VCL004 boundary-bypass

Direct HTTP calls (http_get, http_post, etc.) that bypass a declared service client boundary. Only fires when the project contains at least one service declaration.

Severity: warning | Auto-fix: no

# File: src/handlers/order_handler.vary — Flagged
# (project has a service ItemApi declaration elsewhere)
def fetch_item(id: Int) -> Str {
    return http_get("https://api.example.com/items/" + str(id))
}

Use the corresponding service endpoint method instead of calling HTTP functions directly. This ensures consistent base URLs, error handling, and type-safe request/response mapping.

Great code

Great-code rules catch design smells that compile and type-check cleanly but suggest a better shape. They are enabled under the --great-code profile.

VCG008 parallel-collection

Two or more List variables indexed by the same loop variable in lockstep inside a single scope. This usually means the lists are really one list of a data class.

Severity: warning | Auto-fix: no

# Flagged
let names: List[Str] = [...]
let ages: List[Int]  = [...]
for i in 0..names.len() {
    print(names[i], ages[i])
}

# Better
data Person { name: Str, age: Int }
let people: List[Person] = [...]
for p in people {
    print(p.name, p.age)
}

VCG009 tagged-union-smell

A class with a discriminator field named kind, type, or tag (Str or Int) combined with if/elif chains that dispatch on that field. This is a hand-rolled tagged union that should be a Vary enum with payloads.

Severity: warning | Auto-fix: no

The rule only fires when both the discriminator field exists and at least one method branches on its value, so legitimate flag fields are not flagged.

VCG010 mixed-concern-module

A module whose public exports cluster into two or more unrelated concern prefixes (for example, parse_* and format_*), each with at least two exports. Suggests splitting the module along concern boundaries.

Severity: warning | Auto-fix: no

CLI flags

FlagDescription
-v, --verboseVerbose output
-s, --summaryOne line per file
-W, --warningsWarning mode: all (default), none, error
--fix, --fix-safeApply safe auto-fixes in place
--categoryFilter rules by category (repeatable)
--ruleFilter rules by ID (repeatable)
--jsonMachine-readable JSON output
--explain [RULE]Explain a rule by ID or name; omit value to list all rules
--profileCheck profile: ci (JSON output, warnings as errors) or local (defaults)
--planOutput an ordered repair plan
--groupGroup output by priority (default) or file

Explain a rule

Show why a rule exists, what it detects, and how to fix findings:

vary check --explain VCI001
vary check --explain            # list all rules

This prints the same rationale as vary explain VCI001 but scoped to check rules.

Repair plans

Generate an ordered plan showing which findings have safe fixes and in what order to apply them:

vary check src/ --plan

The plan groups fixes by file and respects dependency order (e.g., import additions before the code that uses them).

CI profile

The ci profile switches to JSON output and treats warnings as errors, suitable for CI pipelines:

vary check src/ --profile ci

You can also set this in vary.toml:

[check]
profile = "ci"

See CI verification for GitHub Actions examples and verification profiles.

Filtering rules

Run only idiom rules:

vary check src/ --category idioms

Run a single rule:

vary check src/ --rule VCI001

Combine multiple categories:

vary check src/ --category idioms --category contracts

Auto-fix

Rules marked with auto-fix support can be applied automatically:

vary check src/ --fix

Only fixes marked SAFE are applied. The command reports how many fixes were applied and how many were skipped due to conflicts.

Testing →