diff --git a/DELIVERY.md b/DELIVERY.md index 5370d45..b25e60d 100644 --- a/DELIVERY.md +++ b/DELIVERY.md @@ -73,6 +73,65 @@ subject's own header comment. **A test that passes against the bug it names is not a test.** +**And check the mutation, not just the test.** Proving an allocation gate could fail meant making +the code allocate. Boxing a `bool` was the obvious way, the gate stayed green, and for a moment that +looked like a broken gate. The JIT had elided the box, so nothing allocated and the gate was right. +`GC.KeepAlive(new object())` turned it red immediately. A mutation the compiler optimises away +proves nothing in either direction, which is the same trap one level up. + +### A file on your disk is not a file in the repository + +Run the checks against a fresh checkout, not your working tree. Anything that reads files from the +repo, a link checker, a docs test, a template renderer, is checking your machine unless CI runs it. + +**Caught:** eight package guides were written, the docs index linked all eight, and every test +passed locally. `.gitignore` had an unanchored `packages/` rule meant for a NuGet folder, which also +matched `docs/packages/`, so `git add -A` skipped all eight without a word and the branch shipped a +documentation index pointing at nothing. The link test passed on the laptop because the files were +there. It failed on the first CI run, on a clean clone, which is the only place the difference is +visible. + +**`git add -A` is silent about what it ignored.** That silence is the failure, not the rule. + +### Absence of output is not absence of the thing + +A diagnostic step that prints nothing looks the same as a diagnostic step +reporting nothing is there. If a check exists to answer a question, make it +assert the answer rather than print material for a human to read. + +**Caught:** a CI matrix leg was added to run the suite on .NET 10, with a comment +explaining that installing only that SDK left the net8.0 assemblies no 8.0 +runtime to load, so they would roll forward. A step printed `dotnet --info` for +evidence. Its `sed` pattern was case-sensitive, matched nothing, and printed only +a version header. Grepping that empty output for a .NET 8 runtime found none, +which read as confirmation and went into the pull request as verified. + +The hosted image ships .NET 8, 9 and 10, `setup-dotnet` adds to them rather than +replacing them, and roll-forward never engages when an exact match is present. +Both legs had been running the net8.0 assets on .NET 8.0.30. The second leg +tested nothing the first did not, which is the exact failure the matrix was added +to avoid. + +Replacing the print with an assertion caught it on the first run. The real fix was +to multi-target the test projects, because targeting the framework is the only +thing that moves the runtime. + +**A step whose output nobody fails on is a comment.** + +### Assert the middle, not just the edges + +A test that checks the first and last element of a collection, or its count, will pass while +everything between them is destroyed. + +**Caught:** a pooled buffer was returned to `ArrayPool` with `clearArray: true` and then copied out +of, one line too late, so every item collected before the first buffer doubling came back blank: 16 +of 17, 512 of 2000. Two tests covered that code path. One asserted the count and the final element, +and the final element is written after the loss. The other used a collection type that took a +different branch entirely. Both were green. Asserting every index, at sizes that straddle each +doubling, fails immediately. + +**Count is not content.** + ### Gate the promise the project makes Every gate above is generic. This one is not, and it is the one most worth spending effort on: