Skip to content

The evidence ladder

Evidence isn't binary, and the most useful thing a verification tool can do is tell you honestly how strong each piece of it is. mathema ranks every supported claim by how it was established, from a proof that covers every input in the declared domain down to a sentence someone wrote in a docstring, and it never lets a weaker result be reported as a stronger one.

The evidence ladder Six rungs, strongest at the top: derive, derive extensive, informed probing, probing, documented, declared. deriveproven derive:extensiveproven probe:semi_analytical · probe:algorithmicholds (n=…) probeholds (n=…) documentedstated deliberately declaredinferred from a summary

The rungs, strongest first

Rung Verdict What established it
derive proven The function's body was lifted to a symbolic expression and the claim settled over the whole declared domain.
derive:extensive proven The same, reached only by the deeper search you opt into with extensive=True.
probe:semi_analytical, probe:algorithmic holds (n=...) The real function survived n trials whose inputs were chosen by analysis, such as the points where a denominator vanishes, or by a technique specific to the claim.
probe holds (n=...) The real function survived n seeded random trials.
documented none Stated intent that a person has accepted with mathema accept --intent.
declared none Stated intent (a docstring summary, an Intent: block, an intent: field) that no person has accepted yet, the weakest rung there is.

The two informed probing routes share a rung because they draw on different sources of information without either being stronger than the other. The ladder is defined in the engine as mathema.conjecture.EVIDENCE_LADDER, and a route mathema does not recognise, such as one from a verification technique you have plugged in yourself, ranks below everything it does.

A proof is the mathematics; [float] is the code

A proven from the derive route means the claim holds in exact real arithmetic over the declared domain, and nothing more. It does not say the float implementation gets the same answer. That is a separate claim, and mathema makes it for you: every claim the derive route proves spawns a companion named <name>[float], in the numerical stability family, adjudicated on the probe route against the real code. The companion runs the relation at every corner of the declared domain and at sampled interior points. A raise, a NaN, or an inf or a loss of precision where the relation fails on the executed values falsifies it, with that point as the witness. An unbounded direction runs to the claim's |inf| when one is declared, and otherwise out to 1e308, sampled log-uniformly so moderate magnitudes are visited too.

import mathema
from mathema.conjecture import claim


def one(x: float) -> float:
    """One, computed the long way round."""
    return (x + 1.0) - x


for law, route in [("for x in [0, 1e6], f(x) == 1", "derive"),
                   ("f(x) == 1", "derive"),
                   ("f(x) == 1", "derive:math_only")]:
    rec = mathema.check(one, claims=[claim(law, name="one", route=route)])
    for p in rec.probes:
        print(f"{route:16} {p.name:10} {p.verdict:9} "
              f"{p.counterexample or ''}".rstrip())
derive           one        proven
derive           one[float] holds
derive           one        proven
derive           one[float] falsified x=-1e+308
derive:math_only one        proven

(x + 1) - x is 1 for every real x, so all three proofs stand. In float64 the + 1 is lost once |x| passes 2^53, so the companion of the unbounded claim is falsified, and its row names the stratum: mathematics sound, implementation numerically unstable. Two claims, two verdicts, and the companion gates mathema verify like any other claim. The remedies are the ordinary ones: narrow the domain, declare the |inf| the code has to reach, fix the code, accept the companion as a discovery with mathema accept, or state the claim with route="derive:math_only" ([derive:math_only] in a docstring), which proves the mathematics alone, spawns no companion, and records the opt-out on the proof's row.

The companion is written to the verified record beside its parent, with meta.mathema.companion_of naming it. It is never part of the declared layer: it is respawned from its parent on every adjudication, so it does not enter the claims fingerprint and is not repopulated as a claim of its own. A limit or an integral is a statement about the mathematics and spawns no companion.

What sits off the ladder

The ladder ranks how a claim came to be supported. Four other outcomes are reported alongside it, and none of them is a weak form of support:

Verdict Means
falsified A counterexample was found by running the function, and it is kept permanently. A falsification is equally definitive whichever route found it.
unknown An adjudication ran and decided nothing either way, for instance a proof attempt that could not close.
skipped The claim could not be adjudicated at all, for a reason the record names.
invalidated A claim that was once proven or holds could no longer be established after the code changed, and the record says what it used to be.

A falsified verdict is knowledge rather than failure: it tells you exactly where the function and the claim part ways, and the accept workflow lets a person decide whether that is a bug in the code or a discovery about the specification.

Why the rungs are kept apart

A result that holds on a thousand random inputs and a result that holds for every input are different kinds of knowledge, and a report that blurs them is telling you what you want to hear. So every verdict carries its route, a holds always carries its trial count, a derive attempt that cannot close hands the claim to the probe route with the reason it stopped kept in the record (a sampled holds never passes itself off as a proof), and a symbolic disproof that nothing can reproduce against the real function, even compared exactly, comes back unknown and flagged instead of falsified, since it more likely points at a fault in the engine than in your code.

Claim-driven development has the full verdict vocabulary, and the derive route covers which functions can reach the top rung.