Green Tests Can Assert Less Than the Invariant Requires
Aug 11, 2026
A test suite going green tells you one thing: the assertions you wrote passed. It does not tell you that the property you actually care about holds. Those two things usually overlap. On a recent piece of infrastructure work — a system meant to coordinate independent actors under a set of hard invariants, reviewed by an independent audit pass — they didn't, seven separate times, and every single gap had the same shape.
The problem
The system in question had to enforce a handful of "this must never happen" rules: no duplicate execution under concurrency, no writes to storage that hadn't been explicitly declared, no record whose lineage could be confused with another's. The rules were written down precisely before any code existed, and the test suite grew large — hundreds of tests, full coverage by the usual measures. By every visible signal, the work was solid.
Then an independent audit went looking specifically for weak tests, not wrong code. It found seven defects. Not one was a broken algorithm. Every one was a test that had been written to check something adjacent to the invariant, and close enough to pass casual review, while leaving the actual rule unverified.
What actually happened
The pattern repeated with enough consistency to be worth naming. A sample:
- An invariant required detecting duplicate work under concurrency. The test only replayed the scenario serially — a condition where the bug can't occur.
- An invariant required that a certain kind of value could never be constructed outside an approved path. The test relied on a compile-time access restriction, which does nothing to stop the same value being constructed at runtime through a different route.
- A guard was supposed to refuse invalid input and accept valid input. The test suite proved the refusal case in detail — and never once verified the guard could accept anything at all. It turned out the guard always refused. The proof passed because the property it was supposed to demonstrate was, structurally, never exercised.
- A uniqueness rule was supposed to apply per logical group. The test checked global uniqueness instead — a strictly different and, in this case, wrong property, one that let two genuinely independent groups collide.
None of these were subtle coding errors. Each test read as reasonable in isolation. The gap only became visible when someone asked, of each one: "what exactly does this prove, and is that actually the invariant?"
Two of the seven were more than weak tests — they were real bugs an adversarial reviewer had to think like an attacker to find, because the original tests never modeled that kind of actor. Fixing them meant replacing checks with structural guarantees: durable claims instead of check-then-act races, runtime registries instead of type-level annotations, database-level uniqueness constraints instead of code that was supposed to remember to look.
The lesson
A test asserting a weaker property than the invariant requires will still go green. That's the trap: the failure mode isn't a red build, it's a suite that looks complete and proves less than it appears to. The most instructive case here had two named, documented proofs both passing — while testing nothing, because the code path they exercised could never actually be satisfied either way.
Six of the seven defects would have been caught by asking one question at the point each invariant was written: does this hold under concurrency? At runtime, not just compile time? Across independent instances? Through every entry point, not just the intended one? At the actual boundary the invariant is meant to protect? "X can never happen" is a claim with several dimensions, and a test that checks one dimension is not evidence for the others.
The broader principle
Rigor concentrated entirely on production code, with test code reviewed only for coverage percentage or "does it pass," leaves exactly this blind spot open. A test is itself a claim, and claims can be wrong in the same ways code can: too narrow, too weak, checking the wrong thing, passing vacuously. Auditing only the implementation while treating the test suite as ground truth means the one thing meant to catch implementation bugs never gets checked itself.
The other quiet cost showed up in how these defects got fixed. The first fix for a weak test often closed only the specific instance reported, not the underlying class — so the same shape of gap resurfaced at the next enforcement point and had to be found again. Treating "where else does this same property need to hold?" as a required question at fix time, not an afterthought, would have collapsed several rounds of rework into one.
How to apply it
- When reviewing a test tied to an invariant, ask what it actually proves, not what it was intended to prove. Read the assertion, not the test name or comment.
- For any "this can never happen" rule, check it against concurrency, runtime (not just compile-time) enforcement, cross-instance behavior, and every entry point — not just the one the happy-path test uses.
- Require a positive case alongside every refusal or rejection test. A guard that only proves it rejects has not proven it can also accept — and a guard that always refuses is not doing its job even though its "rejects invalid input" test passes.
- When you fix a weak test, look for the same weakness anywhere else the same property is supposed to hold, and say explicitly where you looked. Closing an instance without closing the class guarantees a repeat.
- Where possible, push invariants like uniqueness or immutability down into the substrate (database constraints, storage-level enforcement) rather than trusting application code to remember the rule on every path. Enforcement that can't be bypassed is stronger than enforcement that has to be replicated correctly everywhere.
