From ea6c279fbefc7509182b766d24f08a7b83e21f30 Mon Sep 17 00:00:00 2001 From: Arnel Robles Date: Tue, 25 Aug 2026 22:11:09 +0800 Subject: [PATCH] Pin the test universe: a glob that collects beyond the project lies in both directions Co-Authored-By: Claude Fable 5 --- DELIVERY.md | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/DELIVERY.md b/DELIVERY.md index 519fb5c..24755ca 100644 --- a/DELIVERY.md +++ b/DELIVERY.md @@ -307,6 +307,25 @@ broken; the release job simply never tagged. The costs are real anyway: `git log not resolve, so *what has landed since we shipped* cannot be answered from the repository, and a contributor whose work merged has nothing that tells them it reached users. +### Pin the test universe + +The test runner's file glob defines what "the suite" is. Pin it to the project's own directories +(explicit roots or ignore patterns for anything that is not the project: vendored checkouts, +worktrees, tool scratch space), and treat an unexplained jump in the collected-test count as a +finding, not a bonus. + +**Caught:** in rnxORM, Jest's default `**/test/**` glob silently collected an abandoned git +worktree under `.claude/worktrees/` — a dead branch with its own diverged copy of the source. The +run reported 605 tests where the project had 286, with 6 failures coming from code that was not +shipping and green passes from suites that proved nothing about `main`. It had been that way across +multiple sessions; a human comparing the count against the documented suite size caught it, nothing +in the pipeline did. The same glob had been fixed once before on a branch that was later abandoned, +so the fix was lost too — the gate is the config line in the repository, not the memory of having +fixed it. + +A number on the test summary that nobody can reconcile with the suite inventory is the same smell +as a metric with no denominator. + --- ## Say what you actually measured