fix(ci): make the honesty gate say what it measured - #2982
Merged
Conversation
Closes #2981 `rings-rust.yml`'s header states its purpose: a non-blocking honesty gate that "surfaces real per-crate compile state without yet enforcing it". The non-blocking half is deliberate and stays. The other half did not happen -- the `summary` job printed the crate COUNT and a pointer to `rings/COMPILE_STATUS.md`, and never read what the matrix had just measured. Measured, every master run of this workflow: 2026-08-28 conclusion=success 0 of 19 jobs failed 2026-08-20 conclusion=success 17 of 19 jobs failed 2026-08-06 conclusion=success 17 of 19 jobs failed 2026-06-14 conclusion=success 17 of 19 jobs failed 2026-06-13 conclusion=success 17 of 19 jobs failed 2026-06-01 conclusion=success 17 of 19 jobs failed 2026-06-01 conclusion=success 17 of 19 jobs failed 2026-05-23 conclusion=success 17 of 19 jobs failed 2026-05-22 conclusion=success 0 of 19 jobs failed Seven master runs, three months, in which every ring crate failed to compile. All seven concluded `success`. And `COMPILE_STATUS.md` -- the file this gate's summary points readers at, which calls itself "the honest, living per-crate compilation status", last updated 2026-05-22 -- said throughout that they compile. The crates were repaired by 2026-08-28 and neither instrument had ever said they were broken. `continue-on-error: true` is NOT the defect and is not touched. The workflow says why in its own header, and a gate that lands red on the default branch is one nobody can merge past. The defect is that the state was computed 17 times per run and thrown away. Each matrix job now appends its own verdict line to `$GITHUB_STEP_SUMMARY` using `steps.<id>.outcome` -- the step result taken BEFORE `continue-on-error` is applied, so a `failure` there means the crate really did not build: - `ring-100-rust` -- ok (check: success, test: success) - `ring-104-rust` -- DID NOT BUILD (check: failure, test: skipped) A list line rather than a table row, because the matrix jobs finish in no fixed order and GitHub concatenates their summaries in completion order: a header written anywhere would not stay above the rows. The step was exercised on all four outcome combinations before shipping. `rings/COMPILE_STATUS.md` gains a note saying it is hand-maintained, when it was last updated, and that where it and a run disagree, the run was measured and the file was remembered. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
|
📓 NotebookLM Notebook linked to this PR
This notebook contains session context, decisions, and artifacts for this work. |
Contributor
PR DashboardGenerated at: 2026-08-30 19:26:22 UTC
Summary
Seal Status
|
Contributor
|
📓 NotebookLM Notebook linked to this PR
This notebook contains session context, decisions, and artifacts for this work. |
Contributor
PR DashboardGenerated at: 2026-08-30 19:27:23 UTC
Summary
Seal Status
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #2981
rings-rust.yml's header states its own purpose: a non-blocking honesty gate that"surfaces real per-crate compile state without yet enforcing it". The non-blocking half
is deliberate and stays. The other half did not happen — the
summaryjob printed thecrate count and a pointer to
rings/COMPILE_STATUS.md, and never read what the matrixhad just measured.
Measured — every master run of this workflow
Seven master runs, three months, every ring crate failing to compile — all seven green.
And
COMPILE_STATUS.md, the file this gate points readers at, which calls itself "thehonest, living per-crate compilation status" and was last updated 2026-05-22, said
throughout that they compile. Two instruments, both silent. The crates were repaired by
2026-08-28 and neither had ever said they were broken.
What is deliberately not touched
continue-on-error: true. The workflow says why in its own header, and a gate that landsred on the default branch is one nobody can merge past. The defect is that the state was
computed 17 times per run and thrown away.
The repair
Each matrix job appends its own verdict line, using
steps.<id>.outcome— the stepresult taken before
continue-on-erroris applied, so afailurethere means thecrate really did not build:
A list line, not a table row: matrix jobs finish in no fixed order and GitHub
concatenates their summaries in completion order, so a header written anywhere would not
stay above the rows. The step's shell was exercised on all four outcome combinations
(
success/success,failure/skipped,success/failure,cancelled/cancelled) beforeshipping.
rings/COMPILE_STATUS.mdgains a note saying it is hand-maintained, when it was lastupdated, and that where it and a run disagree, the run was measured and the file was
remembered.
How it was found, and what the skeptic corrected
A six-lens adversarial sweep — 14 of 14 agents returned, 0 errors, checked before reading
the result as a sweep at all. The skeptic corrected the finding twice: that
continue-on-errorhas a written reason (so it is not the defect), and that the compilebreakage is already repaired (so this repair is preventive, not urgent). Both corrections
are in the text above.
Also seen, not repaired here
Both
--lockedflags in the same job are dead text: no ring crate has a trackedCargo.lock, socargo check --all-targets --lockedfails at startup for all 17 cratesand the
||swallows it, every run. Cosmetic — the fallback does the real work — but itmeans each run's log carries 17 startup errors that mean nothing.
🤖 Generated with Claude Code