Dev iconDevSep 4, 2026 ~2 min source read

I built a tool to catch empty scans read as passes, and shipped it with an empty scan read as a pass

It plants two positive fixtures that must be found and four negative fixtures that must stay quiet. Then it prints a control line, and you read the real numbers only after that line is green.

I built a tool to catch empty scans read as passes, and shipped it with an empty scan read as a pass

Share this story

Send the public story page.

Useful takeaways from this story.

It plants two positive fixtures that must be found and four negative fixtures that must stay quiet.

Then it prints a control line, and you read the real numbers only after that line is green.

Every fixture directory was excluded by name before the walk reached it, so the four negatives were clean in the way that a locked room is quiet.

Building the complete brief

The page is ready to read now. The fuller skim-friendly version will appear here automatically.

The useful part

It plants two positive fixtures that must be found and four negative fixtures that must stay quiet. Then it prints a control line, and you read the real numbers only after that line is green. Every fixture directory was excluded by name before the walk reached it, so the four negatives were clean in the way that a locked room is quiet.

How it works

  • The self-test now prints which signal saw which positive fixture, so a fixture passing for the wrong reason is visible in the output.
  • The fix is one line and one idea The line: a fixture run where filesScanned === 0 is untestable rather than ok, and it exits non-zero.
  • Those need different spellings, and folding the first into the second is a reliable way to lose a bug forever.
  • Do not fold a measurement you could not take into a failure, and do not fold a measurement of nothing into a success.
  • Then I wrote a self-test that scanned zero files, read the silence as agreem...

Example or evidence

  • If a broken build still passes, the control is measuring nothing.

Details worth keeping

Each must miss the planted defect and exit 1. That is the check on the check, and it stops there. Two stages is where I decided the regress ends.

Keep reading in the app

Open the app view to save this story, compare related coverage, and continue from the same source.

Open in app