rings_matrix: an empty matrix is not a clean build - #3070
Merged
Conversation
Closes #3069 discover() drops a directory unless the name starts `ring-`, ends `-rust`, and holds a Cargo.toml. Any of the three is one rename away. With the matrix empty the build job is skipped -- `if: needs.discover.outputs.count != '0'` -- and a SKIPPED job is green, so the run concluded success having compiled nothing. The trigger is the same commit. rings-rust.yml filters on `rings/ring-*-rust/**`, so the rename touches paths under the old name, matches the filter, RUNS the workflow, and collects a green tick for it. The script now refuses an empty population and exits 2, naming what it looked for and where. Verified under GitHub's own shell flags (bash --noprofile --norc -eo pipefail) that `MATRIX="$(python3 scripts/ci/rings_matrix.py)"` aborts the step on a non-zero exit, so `discover` goes red instead of the build going green-by-skip. Today the population is 17 and unchanged. This file has been here before, and its workflow header says so: seven master runs between 2026-05-23 and 2026-08-20 had all 17 crate jobs failing and every one concluded `success`. That was closed per job -- each matrix job writes its own verdict row now. This is the same door one step earlier: with zero jobs there are no verdict rows to write. THE CONTROL CAUGHT A FALSE PASS, which is the part worth reading. The first harness emptied $GITHUB_OUTPUT instead of pointing it at a file, so all three defect arms failed on line 2's redirect to "" rather than on line 1's exit 2 -- and all three "passed". The control that asserts SUCCESS on a real tree is what found it. Each defect arm now also asserts that `rings_matrix: REFUSED` is what stopped the step, so an arm cannot pass on any failure at all. Three mutants: removing the refusal fails 12 assertions, refusing always fails the 4 control assertions, refusing but exiting 0 fails 3. Exit 2 rather than 1: nothing failed to compile, the population was never built. Prior art checked rather than assumed -- pytest reserves exit code 5 for "No tests were collected" as a public-API outcome distinct from 1, 2, 3 and 4, which makes this repository's 2-for-everything coarser than the field's by one distinction. Recorded, not churned.
gHashTag
enabled auto-merge (squash)
September 3, 2026 22:28
Contributor
Contributor
|
📓 NotebookLM Notebook linked to this PR
This notebook contains session context, decisions, and artifacts for this work. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #3069
A crate rename made this workflow compile nothing and report success
discover()drops a directory unless the name startsring-, ends-rust, and holds aCargo.toml. Any of the three is one rename away.A skipped job is green.
discoverprintedDiscovered 0 ring-*-rust crate(s).and succeeded, and the run concluded success having compiled nothing.The trigger is the same commit.
paths: rings/ring-*-rust/**— the rename touches paths under the old name, matches the filter, runs the workflow, and collects a green tick for it.It has been here before
From the workflow's own header:
Closed per job — each matrix job writes its own verdict row now. This is the same door one step earlier: with zero jobs there are no verdict rows to write.
After
Discovered 17 ring-*-rust crate(s).-rustrings_matrix: REFUSED …, exit 2, step abortsCargo.tomlrings/at allVerified under GitHub's own shell flags —
bash --noprofile --norc -eo pipefail— thatMATRIX="$(python3 scripts/ci/rings_matrix.py)"aborts the step on a non-zero exit, sodiscovergoes red rather than the build going green-by-skip.The control caught a false pass
Worth more than the fix. The first harness emptied
$GITHUB_OUTPUTinstead of pointing it at a file, so all three defect arms died on line 2's redirect to""rather than on line 1's exit 2 — and all three "passed". The control asserting success on a real tree is what found it.Each defect arm now also asserts that
rings_matrix: REFUSEDis what stopped the step, so an arm cannot pass on any failure at all.The step body is extracted from the workflow, not restated, so a YAML change that stops the failure propagating is caught; the extraction is itself asserted.
Exit 2, and a note on the convention
Nothing failed to compile — the population was never built.
t27c corpusrefuses a spec tree with no specs the same way (#3025), andscripts/triuses 2 for an unbuilt compiler (#3045).Prior art, checked rather than assumed: pytest reserves exit code 5 for "No tests were collected", a public-API outcome distinct from 1 (tests failed), 2 (interrupted), 3 (internal error) and 4 (usage error). So this repository's 2-for-everything is coarser than the field's by one distinction — "the instrument is missing" and "the population is empty" share a code here and have different codes there. Recorded rather than churned; changing it is a decision for the repository, not a side effect of this fix.