THE FOLD / RESPAWN / THE CONTINUE / THE TEST THAT DID NOT RUN
THE TEST THAT DID NOT RUN
identical to a test that passes
1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION
A harness captured an exit code with
LIT verified live: 2 of 7 cases never executed while the harness reported 0 failures, indistinguishable from a green suite; apparent coverage of 100% against real coverage of 71.4%, overstated by 28.6 points with no visible symptom; if each live case catches a distinct defect class with probability q, real detection falls short at every q tested (91.8%→83.2%, 99.2%→96.9%, 100.0%→99.8%) with a worst gap of 8.6 points; and when nothing is broken, both suites are green in 20,000 of 20,000 runs.
${PIPESTATUS[0]}. The shell was sh, not bash, so it returned Bad substitution — and two of seven test cases silently did not run. The harness reported zero failures, which is precisely what a fully passing suite reports. David’s line for it: a test that does not run looks identical to a test that passes. There is no symptom to notice, because the absence of a failure is the same signal in both worlds.LIT verified live: 2 of 7 cases never executed while the harness reported 0 failures, indistinguishable from a green suite; apparent coverage of 100% against real coverage of 71.4%, overstated by 28.6 points with no visible symptom; if each live case catches a distinct defect class with probability q, real detection falls short at every q tested (91.8%→83.2%, 99.2%→96.9%, 100.0%→99.8%) with a worst gap of 8.6 points; and when nothing is broken, both suites are green in 20,000 of 20,000 runs.
2 HOW IT WAS WEAVED · AI + HUMAN
David (human) caught it and re-ran under
AVAN (AI) is being precise about what the simulation shows, because the honest result is less dramatic than the framing invites and more useful. At high per-test detection rates the coverage loss is small — at q = 0.7 the gap is only 0.2 points, because five strong tests already catch nearly everything. The damage is worst in the middle, at q = 0.3, where it reaches 8.6 points. So the danger is not that a partially-dead suite misses everything; it is that the miss is modest and completely invisible, which is the combination that survives review. A suite failing loudly gets fixed. This one reported success for as long as nobody counted the cases.
bash -c, after which all seven exit paths were exercised for real. Seated at THE CONTINUE, because the suite carried on and reported success while two of its cases were not there.AVAN (AI) is being precise about what the simulation shows, because the honest result is less dramatic than the framing invites and more useful. At high per-test detection rates the coverage loss is small — at q = 0.7 the gap is only 0.2 points, because five strong tests already catch nearly everything. The damage is worst in the middle, at q = 0.3, where it reaches 8.6 points. So the danger is not that a partially-dead suite misses everything; it is that the miss is modest and completely invisible, which is the combination that survives review. A suite failing loudly gets fixed. This one reported success for as long as nobody counted the cases.
3 ONE DIMENSION
Seven cases. Two of them are not there, and the report is identical.
4 TWO DIMENSIONS · INTERACTIVE
Kill cases silently and watch the report refuse to change.
5 THREE DIMENSIONS + AVAN’S INVERSE
The green forward object: a suite with holes in it, all of them green.
AVAN’s addition (the inverse-companion): the forward reading is “make sure your tests run.” The inverse is that a green suite is evidence of two things at once and cannot separate them. It says either the code is correct or the test did not look, and the report has no channel for the difference. Every passing build carries that ambiguity; usually the second reading is so unlikely it can be ignored, and the moment a harness breaks quietly it becomes the likelier one without any signal that the balance shifted. Read backwards, this is why suites need to count themselves — not to catch bugs, but to distinguish silence from consent.
LIT 2 of 7 cases never executed while the harness reported 0 failures, indistinguishable from a green suite; apparent coverage of 100% against real coverage of 71.4%, overstated by 28.6 points with no visible symptom; if each live case catches a distinct defect class with probability q, real detection falls short at every q tested (91.8% to 83.2%, 99.2% to 96.9%, 100.0% to 99.8%) with a worst gap of 8.6 points; and when nothing is broken both suites are green in 20,000 of 20,000 runs
FIG The honest result is LESS dramatic than the framing invites and more useful. At high per-test detection the coverage loss is small — at q=0.7 the gap is only 0.2 points, because five strong tests already catch nearly everything. The damage is worst in the middle, 8.6 points at q=0.3. So the danger is not that a partially-dead suite misses everything; it is that the miss is modest and completely invisible, which is the combination that survives review.
FIG The honest result is LESS dramatic than the framing invites and more useful. At high per-test detection the coverage loss is small — at q=0.7 the gap is only 0.2 points, because five strong tests already catch nearly everything. The damage is worst in the middle, 8.6 points at q=0.3. So the danger is not that a partially-dead suite misses everything; it is that the miss is modest and completely invisible, which is the combination that survives review.
◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of THE CONTINUE · David Lee Wise (ROOT0), with AVAN