Testing course

Vary Testing

Seven runnable lessons covering test, observe, edge cases, error paths, output, and naming.

For each lesson, copy the code into the filename shown in the terminal, run the command next to the lesson, then paste the output of the program into the answer box and validate it. Your score is saved in this browser, so you can leave and come back later on the same device.

Section 1

First tests

One observation, several observations, several tests.

pure def is_even(n: Int) -> Bool {
    return n % 2 == 0
}

test "4 is even" {
    observe is_even(4) == True
}

Lesson 1 / tests/even_test.vary

The smallest test

Verify one fact about one function.

A test block holds one or more observe assertions. Each observe is a boolean expression that must be true at runtime; if it is not, the test fails and the runner exits non-zero.

Test DSL CLI Reference Quick Reference

Run vary test tests/even_test.vary

Expected output

Results: 1 passed, 0 failed

Paste the test output

pure def normalize_path(path: Str) -> Str {
    if path.startswith("/") {
        return path
    }
    return "/" + path
}

test "normalize_path enforces a single leading slash" {
    observe normalize_path("api") == "/api"
    observe normalize_path("/api") == "/api"
    observe normalize_path("") == "/"
}

Lesson 2 / tests/path_test.vary

Multiple observations per test

Group several related facts under one named behaviour.

observe lines stack inside one test block. Group facts that describe the same behaviour: if any one fails, the whole test fails, so callers see one named failure instead of three nameless ones.

Test DSL Functions Tour CLI Reference

Run vary test tests/path_test.vary

Expected output

Results: 1 passed, 0 failed

Paste the test output

pure def class_of(status: Int) -> Str {
    if status >= 500 {
        return "5xx"
    }
    if status >= 400 {
        return "4xx"
    }
    if status >= 200 {
        return "2xx"
    }
    return "other"
}

test "2xx range" {
    observe class_of(200) == "2xx"
}

test "4xx range" {
    observe class_of(404) == "4xx"
}

test "5xx range" {
    observe class_of(503) == "5xx"
}

test "below 200 is other" {
    observe class_of(100) == "other"
}

Lesson 3 / tests/status_test.vary

Multiple tests per file

Cover distinct behaviours with separate named tests.

Distinct behaviours belong in their own test blocks. The failure output names which behaviours broke, which is much more useful than a single test that lumps unrelated facts together.

Test DSL CLI Reference Errors

Run vary test tests/status_test.vary

Expected output

Results: 4 passed, 0 failed

Paste the test output

Section 2

Edge cases and errors

Test the boundaries and the failure paths.

pure def is_2xx(status: Int) -> Bool {
    return status >= 200 and status < 300
}

test "200 is the lower edge" {
    observe is_2xx(200) == True
    observe is_2xx(199) == False
}

test "299 is the upper edge" {
    observe is_2xx(299) == True
    observe is_2xx(300) == False
}

Lesson 4 / tests/range_test.vary

Test the boundaries

Assert both sides of every edge where behaviour changes.

Off-by-one bugs hide on the edges of >=, <, and ==. Assert the boundary value and the value one step off it; mutation testing rewards this because boundary mutants only break at the edge.

Test DSL Mutation Workflow Mutation Testing

Run vary test tests/range_test.vary

Expected output

Results: 2 passed, 0 failed

Paste the test output

def parse_port(s: Str) -> Int {
    let n = int(s)
    if n < 1 or n > 65535 {
        raise "port out of range: " + s
    }
    return n
}

test "valid port parses" {
    observe parse_port("8080") == 8080
}

test "out of range raises" {
    observe throws { parse_port("99999") }
}

test "non-numeric raises" {
    observe throws { parse_port("nope") }
}

Lesson 5 / tests/port_test.vary

Test error paths

Use observe throws to assert a block raises.

observe throws { expr } passes only if expr raises. Use it for input validation, parser errors, and anywhere else the code reports failure by throwing.

Test DSL Errors Mutation Workflow

Run vary test tests/port_test.vary

Expected output

Results: 3 passed, 0 failed

Paste the test output

Section 3

Running and reading

Test output, exit codes, and writing names that read like a spec.

pure def retry_delay_ms(attempt: Int) -> Int {
    if attempt <= 0 {
        return 100
    }
    if attempt >= 5 {
        return 16000
    }
    mut delay: Int = 100
    mut i: Int = 0
    while i < attempt {
        delay = delay * 2
        i = i + 1
    }
    return delay
}

test "first attempt is 100ms" {
    observe retry_delay_ms(0) == 100
}

test "doubles each attempt" {
    observe retry_delay_ms(1) == 200
    observe retry_delay_ms(2) == 400
    observe retry_delay_ms(3) == 800
}

test "saturates at 16000ms" {
    observe retry_delay_ms(5) == 16000
    observe retry_delay_ms(10) == 16000
}

Lesson 6 / tests/backoff_test.vary

Read the test output

Locate the pass/fail summary line and understand the exit code.

The runner prints each PASS: or FAIL: line, then a Results: N passed, M failed summary. The exit code is non-zero whenever M is non-zero, so CI scripts can treat the summary line as a sanity check next to $?.

Test DSL CLI Reference Errors

Run vary test tests/backoff_test.vary

Expected output

Results: 3 passed, 0 failed

Paste the test output

pure def truncate(s: Str, limit: Int) -> Str {
    if len(s) <= limit {
        return s
    }
    return s[:limit - 1] + "…"
}

test "truncate returns the input when it fits inside the limit" {
    observe truncate("hello", 10) == "hello"
}

test "truncate replaces the last character with an ellipsis when too long" {
    observe truncate("hello world", 5) == "hell…"
}

test "truncate handles the exact-fit boundary without modification" {
    observe truncate("hello", 5) == "hello"
}

Lesson 7 / tests/truncate_test.vary

Names are documentation

Write test names that read like the spec.

Test names appear verbatim in pass/fail output. A name that says what the function should do (truncate returns the input when it fits inside the limit) doubles as documented behaviour: whoever reads the failure does not have to read the code first.

Test DSL Mutation Workflow Functions Tour

Run vary test tests/truncate_test.vary

Expected output

Results: 3 passed, 0 failed

Paste the test output

Course score

0/7 lessons complete

Validate each lesson output to finish this course.

Next course

Mutation Testing Five runnable lessons covering `vary mutate`, scores, survivors, boundary mutations, and the inner-loop workflow. Continue