The amnesty carries a number, not just an identity - #2772
Merged
Conversation
W699. `parse-no-discard`'s failure message has always carried a magnitude --
parser reached EOF but DISCARDED 208 top-level token(s)
-- and the ledger threw it away. An entry was `(path, phase)`, so a spec could go
from discarding one token to discarding 682 without moving a gate, and the two
parser fixes earlier today that recovered 1 292 tokens could not be priced: the
population was 87 either way.
`parse-no-discard` entries now carry `discard_tokens`, and the ratchet compares
it. Three rules, matching the ones already there rather than inventing new ones:
* MORE is a failure -- the regression signal that did not exist
* LESS is also a failure, for the same reason an unexpected PASS is one:
unclaimed slack is where the next regression hides. Re-bless to pin it.
* NO READING is treated as worse, never as an improvement. A spec that stopped
being measured and a spec that discards nothing look identical from the
ratchet's side, and a map defaulting to zero would report every unreadable
spec as a triumph.
Blessing writes what the run measured, so lowering is automatic on a re-bless and
raising is a diff a human reads. 87 entries pinned, 32 485 tokens total.
Seen failing, on purpose, twice: shifting one pinned value by -5 gives
DISCARD WORSENED with `(+5)`, by +5 gives DISCARD IMPROVED with `(-5)`. Five unit
tests cover worsened, improved, unpinned, no-reading, and exact match.
CORRECTION. I have written three times -- in a commit, a PR body and the loop
dashboard -- that "the phase that would notice sits in the suite's BLOCKED
column, so nothing reports it". That is false. `parse-no-discard` reports 87
PRIMARY corpus failures and 0 blocked; what sits in BLOCKED is
`no-vacuous-invariant`, at 72. The mechanism is not blocking, it is amnesty: the
ledger's 178 entries are 91 `parse` plus 87 `parse-no-discard`, so every spec
that discards is excused by name. That is a sharper finding than the one I
reported, and it is what this commit acts on.
Two stale claims in docs/CORPUS-RATCHET.md, both in the section listing what a
green run does NOT cover -- the last place a stale claim should sit, since it is
read exactly when someone is deciding how far to trust a pass:
* "`parse-complete` is not among the phases, and appending `))) ... (((` to a
corpus spec leaves the ratchet CLEAN". Re-verified by doing it: the ratchet
now names `specs/account/auth.t27 [parse]` as an unexpected failure.
* "5 standing unit-test failures". Measured today: `cargo test --no-fail-fast`
is 2429 passed, 0 failed.
New: `tri discard top --n 15` ranks the discarding specs against their pinned
bound. A count of 87 does not say where to start; `specs/isa/ternary_deque.t27`
at 1 873 tokens does. It reads `t27c parse-complete` rather than re-implementing
the accounting.
And the field, for once shaped the same way: Notion's eslint ratchet records
per-file allowances and DECREASES them automatically as issues are fixed. One
decision apart from this -- there an improvement silently tightens the bound,
here it fails until someone blesses it. Same choice `xfail_strict` makes: an
automatic tightening is invisible in review.
cargo test --no-fail-fast 2429 passed, 0 failed
suite --ratchet RATCHET: CLEAN, exit 0
check_seal_coverage.py exit 0
check_specs_generate.py exit 0
Refs #2754
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
|
📓 NotebookLM Notebook linked to this PR
This notebook contains session context, decisions, and artifacts for this work. |
Contributor
PR DashboardGenerated at: 2026-08-29 00:55:04 UTC
Summary
Seal Status
|
…ts first (156-158) I confused "blocked" with "amnestied" three times in one day; they look alike from a green run and the fix for each is the opposite of the other. An amnesty keyed by identity cannot see magnitude, and the third rule for adding one -- no reading is WORSE, never an improvement -- is the lesson a Default::default() on a lookup silently gets wrong. And the "what this does NOT cover" section is the one read when someone is deciding how far to trust a pass, which is why a stale claim there costs the most. Two were stale; both re-verified by running them. Refs #2754 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
|
📓 NotebookLM Notebook linked to this PR
This notebook contains session context, decisions, and artifacts for this work. |
Contributor
PR DashboardGenerated at: 2026-08-29 00:55:57 UTC
Summary
Seal Status
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #2754
parse-no-discard's failure message has always carried a magnitude —— and the ledger threw it away. An entry was
(path, phase), so a spec could go from discarding one token to discarding 682 without moving a gate. The blindness runs both ways: the two parser fixes earlier today recovered 1 292 tokens and nothing could price them, because the population was 87 either way.Three rules, matching the ones already there
parse-no-discardentries now carrydiscard_tokens:{ "path": "specs/isa/ternary_deque.t27", "phase": "parse-no-discard", "discard_tokens": 1873 }Blessing writes what the run measured, so lowering is automatic on a re-bless and raising is a diff a human reads. 87 entries pinned, 32 485 tokens.
Seen failing on purpose, twice. Shifting one pinned value by −5 gives
DISCARD WORSENED … (+5); by +5 givesDISCARD IMPROVED … (-5) -- re-bless to pin it. Five unit tests cover worsened, improved, unpinned, no-reading and exact match.Correction
I have written three times — in a commit, a PR body, and the loop dashboard — that "the phase that would notice sits in the suite's BLOCKED column, so nothing reports it." That is false.
parse-no-discardreports 87 primary failures and 0 blocked. What sits in BLOCKED isno-vacuous-invariant. The mechanism is not blocking, it is amnesty: the ledger's 178 entries are 91parse+ 87parse-no-discard, so every spec that discards is excused by name. That is a sharper finding than the one I reported, and it is what this PR acts on.Two stale claims in the doc that says what a green run does not cover
The last place a stale claim should sit — it is read exactly when someone is deciding how far to trust a pass.
parse-completeis not among the phases, and appending))) … (((leaves the ratchet CLEAN"UNEXPECTED FAILURES: 1 / + specs/account/auth.t27 [parse]cargo test --no-fail-fastis 2429 passed, 0 failedFinding the next one
A count of 87 does not say where to start. It reads
t27c parse-completerather than re-implementing the accounting — a second implementation of a measurement is a second number to disagree with the first.The field, for once shaped the same way
Notion's eslint ratchet records per-file allowances and decreases them automatically as issues are fixed. One decision apart from this: there an improvement silently tightens the bound, here it fails until someone blesses it. Same choice
xfail_strictmakes — an automatic tightening is invisible in review, and nobody notices when the tool tightens the wrong thing.🤖 Generated with Claude Code