Skip to content

mathema verify

The test-runner face and the CI gate: sweeps the spec store, re-adjudicates every function whose form hash no longer matches its record (new or changed code), and refreshes the machine records so the next run has a baseline.

mathema verify [<key> ...] [--root .]

Given one or more store keys (canonical dotted module.qualname), the sweep is confined to exactly those records, re-adjudicating and re-stamping each in place. Freshness never skips a key you named: the form-hash short-circuit is an optimisation for a whole-store sweep, so asking about one key always re-checks it (this is the fix the integrity-mismatch warning points at): the single-key form the reconcile workflow points you at (mathema verify mypkg.mod.fn after a merge touched one record). With no key it sweeps the whole store as before. A key that names no record fails as a clear per-key problem line, exit 2.

Arguments

Flag Meaning
<key> ... zero or more store keys to confine the sweep to; omit for the whole store
--root project root holding .mathema/verified and claimspec.yaml (default .)
--all re-adjudicate everything, ignoring form-hash freshness
--strict / --lenient one strictness pair shared with check; strict is the default here (CI gates a settled store), --lenient reports unverifiable claims and accepted risk instead of failing on them
--status [TARGET] report fresh/stale per @track_claims-tagged function, adjudicate nothing, exit 0; an optional TARGET (dotted name or file path) is imported first so its tagged functions register
--format text (default) or json: the whole sweep as data; per-key entries with why, gate counts, and claim rows in the same vocabulary check --format compact and the MCP tools speak (stance/verdict/route/source/gates, counterexample iff refuted). Exit codes are identical either way
--output FILE write the report to a file instead of stdout

Store keys are canonical dotted module.qualname names, resolved by importing from --root; the target must be importable there, and a bare-name key fails as a clear per-key problem line.

What fails the run

Finding Default (strict) --lenient
falsified claim fails fails
invalidated claim (held before, fails now) fails fails
unknown claim, not accepted fails fails
unknown claim, accepted as risk fails (N accepted-risk claim(s)) passes, named in the row
skipped (unverifiable) claim fails passes, informational
unresolved global name fails fails
silently-unenforced declared domain fails passes, informational
record whose function no longer resolves fails (names the rename when an unrecorded function has its form hash) fails
locked function whose body changed fails, record untouched fails, record untouched
lock removed outside mathema unlock fails fails
acceptance the policy rejects fails fails

A falsified claim fails in every mode: the record keeps the counterexample permanently (refutation is knowledge), but knowledge of a broken claim is exactly what a gate is for. An unknown claim also fails in every mode; in an agentic loop a claim nobody could adjudicate is indistinguishable from one that would have failed, until a human explicitly owns the gap with mathema accept --as risk. Accepted risk is visible relaxation, not laundering: --lenient proceeds past it (the row still names it), strict mode still refuses it.

Freshness never bypasses the gate: a function whose form hash is unchanged skips re-adjudication, but its stored verdicts are still checked, a record carrying a falsified or open-unknown claim fails the run even when nothing was re-run.

Exit codes: 0 clean; 1 gate failure; 2 usage/target/store error; 130 interrupted.

Worked example

Continuing from mathema check's softmax example:

import math
from typing import Annotated
from mathema.types import Shape

def softmax(scores: Annotated[list, Shape("n")]) -> Annotated[list, Shape("n")]:
    """Turn a vector of real-valued scores into a probability distribution.

    Claims:
        sums_to_one: sum(f(scores)) == 1
    """
    m = max(scores)
    exps = [math.exp(s - m) for s in scores]
    total = sum(exps)
    return [e / total for e in exps]

with its record written by write_spec():

import mathema
from functions import softmax

mathema.write_spec(softmax)
$ mathema verify --root .
ok   functions.softmax: fresh
1 fresh (form unchanged, skipped), 0 adjudicated, 0 problem(s)
grammars detected: mathema; verified by this run: mathema

Drop the normalization on purpose (return exps instead of dividing by the total):

import math
from typing import Annotated
from mathema.types import Shape

def softmax(scores: Annotated[list, Shape("n")]) -> Annotated[list, Shape("n")]:
    """Turn a vector of real-valued scores into a probability distribution.

    Claims:
        sums_to_one: sum(f(scores)) == 1
    """
    m = max(scores)
    exps = [math.exp(s - m) for s in scores]
    total = sum(exps)
    return exps

and re-run:

$ mathema verify --root .
FAIL functions.softmax: form changed; 1 proven, 1 holds, 0 falsified, 1 invalidated  <- 1 invalidated claim(s)
0 fresh (form unchanged, skipped), 1 adjudicated, 1 problem(s)
grammars detected: mathema; verified by this run: mathema

sums_to_one now fails, and because it held before it is reported as invalidated rather than falsified; the counterexample is kept in the record, and the exit code is 1. Restore the fix (the first version of functions.py above) and re-verify:

import math
from typing import Annotated
from mathema.types import Shape

def softmax(scores: Annotated[list, Shape("n")]) -> Annotated[list, Shape("n")]:
    """Turn a vector of real-valued scores into a probability distribution.

    Claims:
        sums_to_one: sum(f(scores)) == 1
    """
    m = max(scores)
    exps = [math.exp(s - m) for s in scores]
    total = sum(exps)
    return [e / total for e in exps]
$ mathema verify --root .
ok   functions.softmax: form changed; 1 proven, 2 holds, 0 falsified
0 fresh (form unchanged, skipped), 1 adjudicated, 0 problem(s)
grammars detected: mathema; verified by this run: mathema

Back to clean, and the baseline is refreshed; the next verify will be fresh again until the code or the claim set actually changes.

Unknown claims and accepted risk

A claim neither route could decide stays unknown and fails the run. Add a running balance in balances.py, with three claims in claims/running_total.claims.yaml:

def running_total(xs: list, y0: float) -> float:
    """Add every value in xs to a starting balance y0."""
    total = y0
    for v in xs:
        total = v + total
    return total
balances.running_total:
  claims:
    - name: shifts_with_start
      statement: "f(xs, y0) == f(xs, 0) + y0"
    - name: nonneg_for_nonneg_steps
      statement: "for xs in [0, 1]^n, f(xs, 0) >= 0"
    - name: never_overshoots_much
      statement: "assuming nonneg_for_nonneg_steps is proven, for xs in [0, 1]^n, f(xs, 0) <= len(xs)"

never_overshoots_much rests on nonneg_for_nonneg_steps being proven, and that one only holds (the probe agrees; derive cannot settle the sign of the lifted sum), so the dependent claim is unknown. shifts_with_start proves on the derive route, and every proof spawns a shifts_with_start[float] companion, the same law checked in floating point (see the evidence ladder), which holds. The second proven claim in the count is dependencies_current, which verify adds to every record:

$ mathema verify --root .
FAIL balances.running_total: no baseline record; 2 proven, 2 holds, 0 falsified, 1 unknown  <- 1 unknown claim(s)
ok   functions.softmax: fresh
1 fresh (form unchanged, skipped), 1 adjudicated, 1 problem(s)
grammars detected: mathema; verified by this run: mathema

The ways out are real evidence (rewrite the claim or the code so a route can decide it) or an explicit human decision to own the gap (--yes here answers the confirmation prompt). Naming the key re-checks it, so the row counts the accepted claim:

$ mathema accept balances.running_total never_overshoots_much --as risk --note "the premise only holds empirically; monitored" --by "Charles Babbage" --yes
accepting balances.running_total :: never_overshoots_much (verdict unknown) as risk, by Charles Babbage
  - reclassify never_overshoots_much: unknown -> skipped:unknown_but_accepted (strict mode still refuses it; lenient proceeds)
written: reclassify never_overshoots_much: unknown -> skipped:unknown_but_accepted (strict mode still refuses it; lenient proceeds)
$ mathema verify balances.running_total --root . --lenient
ok   balances.running_total: targeted re-verify; 2 proven, 2 holds, 0 falsified, 1 accepted risk
0 fresh (form unchanged, skipped), 1 adjudicated, 0 problem(s)
grammars detected: mathema; verified by this run: mathema

Strict mode (the default) still refuses the accepted risk, so a pipeline can choose whether owned gaps block it:

$ mathema verify balances.running_total --root .
FAIL balances.running_total: targeted re-verify; 2 proven, 2 holds, 0 falsified, 1 accepted risk  <- 1 accepted-risk claim(s)
0 fresh (form unchanged, skipped), 1 adjudicated, 1 problem(s)
grammars detected: mathema; verified by this run: mathema

See mathema accept.

Moved functions

A record is keyed by the function's dotted name, so a function moved to another module leaves its record under a key that no longer resolves, and that key fails the run. Move running_total from balances.py to ledger.py, and rename its claims file stanza to match:

mv balances.py ledger.py
ledger.running_total:
  claims:
    - name: shifts_with_start
      statement: "f(xs, y0) == f(xs, 0) + y0"
    - name: nonneg_for_nonneg_steps
      statement: "for xs in [0, 1]^n, f(xs, 0) >= 0"
    - name: never_overshoots_much
      statement: "assuming nonneg_for_nonneg_steps is proven, for xs in [0, 1]^n, f(xs, 0) <= len(xs)"

When the orphan's form hash matches a function that has no record, the failure line says so and names the exact remedy, and the new key is held back rather than given a fresh record that would start its history over:

$ mathema verify --root .
FAIL balances.running_total: cannot resolve to a live function (declared in .mathema/verified/balances.running_total.yaml); its form hash matches ledger.running_total, which has no record. If it moved, a human keeps its history with: mathema accept ledger.running_total --as reconciled --from balances.running_total
FAIL ledger.running_total: no record yet, and its form hash matches the orphan record balances.running_total; nothing was adjudicated or written for this key. If it moved, a human keeps its history with: mathema accept ledger.running_total --as reconciled --from balances.running_total; if it is a different function, remove the orphan record instead
ok   functions.softmax: fresh
1 fresh (form unchanged, skipped), 0 adjudicated, 2 problem(s)
grammars detected: mathema; verified by this run: mathema

The rename carries the whole record (claims, acceptances and their history, lineage, PIN stamp, lock) to the new key and removes the old file; see moved functions. The project's source is parsed, never imported, to find the match, and only when an orphan exists. In --format json the orphan's entry carries moved_to and remedy, and the held-back key has why: "moved-pending" with moved_from and remedy.

A record's history

A claim row's meta keeps mathema.previous_verdict when its verdict changes: one step, the verdict immediately before this one. There is no history field, because the verified layer is committed to git and git is the history. mathema review [<ref>] reads it by claim (which verdicts flipped, which claims were added, removed or reconciled), and git log .mathema/verified lists every commit that changed a record.

--status: the fresh/stale report

mathema verify --status is the report-only face: one fresh/stale line per @mathema.track_claims-tagged function, no adjudication, always exit 0. "Fresh" means the function's current identity hash matches its last verified record; "stale" means it changed since. Pass a target (mathema verify --status pkg.mod or a file path) to import it first, so a script's own tagged functions register under their real dotted keys:

@mathema.track_claims
def my_fn(x: float) -> float:
    ...

Mixed-grammar claims files

A declared-claims file/key can legitimately mix grammars, a core mathema claim sitting next to a claim in some other grammar (tagged grammar: mathema-data, say) under the same key. verify correctly distinguishes a foreign-grammar claim from a genuine failure: it's reported on its own, non-fatal line (N not this grammar (mathema-data)), not counted toward strict-mode failure. The sweep's closing lines name every grammar seen across the whole run versus what this particular command actually verifies, verify never adjudicates a foreign-grammar claim itself, whatever module owns that grammar; that needs real data/context a static sweep can't provide.