fix(tests): give each test its own scratch directory, and a gate for the class by gHashTag · Pull Request #2955 · gHashTag/t27

gHashTag · 2026-08-30T14:52:59Z

The defect

Four test binaries computed their scratch directory from a key every test in the
binary shares, and each test deletes the whole directory on the way out:

let dir = temp_dir().join(format!("t27-scaffold-{}-{}", process::id(), src.len()));
std::fs::create_dir_all(&dir)?;
...
let _ = std::fs::remove_dir_all(&dir);

Every test in one binary runs in one process, so the pid is constant and the key
is really src.len(). Two tests whose sources happen to be the same length get
the same directory. Under the default parallel runner one erases the spec another
is mid-read of, t27c prints nothing, and the assertion reports an empty result.

verilog_imported_enum.rs is the degenerate case: its key interpolates only the
pid, so all three of its tests shared one directory.

Measurement, not inference

Printing the paths of a single scaffold_c run:

	distinct directories	for
before	3 (four tests sharing one)	6 tests
after	6	6 tests

A probe asserting the directory was fresh fired 8 runs out of 8 on
const_width, pointer_param_c and verilog_imported_enum. The collision is
not occasional — it happens every run, and only the timing of the delete decides
whether a test dies. scaffold_c failed about one run in three; the other three
had simply not fired yet.

scaffold_c passed the first time it was written, and this shape has been on
master since. A green run does not clear a race.

The fix

Key by an AtomicUsize counter: unique per call, which is what a directory its
own user deletes requires.

Not fixed here: bootstrap/tests/struct_order_c.rs, the fifth carrier — it is
fixed in #2949, and touching it in both branches would only make a conflict.

`tri harness scratch`

New command, with --gate and --self-check, so the class cannot come back
silently. It looks for the conjunction — more than one #[test], a scratch
path under temp_dir(), a remove_dir_all of it, and a key with no per-call
component — because any one of those alone is fine.

Historical control. Run against the files as they stand on master it reports
exactly the five the probe convicted, and nothing else.

Two false readings it had to survive, both found by counterexample rather than by
review:

Its first rule asked whether the format! call contained a {. That is true
of every single-line format call ever written, so a key interpolating only
process::id() looked variable — it reported nothing on
verilog_imported_enum.rs while the probe was firing 8 out of 8 on it. It now
judges the arguments.
Judging only the arguments then convicted backend_behaviour.rs, whose key is
format!("...-{tag}") — an inline capture, with a distinct tag per test and
no argument list at all. Inline captures are now read as arguments.

Grepping the symptom instead of the construct also convicts verilog_r_si_1.rs,
where src.len() is an argument to String::with_capacity and has nothing to do
with a path.

The self-check is a five-way negative control: a planted collision must be seen;
a pid-only key must be seen; counter-keyed, inline-capture and single-test files
must not be flagged. If any leg fails the command exits non-zero and says the
clean run claims nothing.

Deliberately not wired into CI in this PR — --gate stays red until #2949
lands and removes the fifth carrier. Wiring it belongs in the commit after both.

Closes #2954 · Refs #2949

🤖 Generated with Claude Code

…the class Four test binaries computed their scratch directory from a key every test in the binary shares, and each test deletes the WHOLE directory on the way out: let dir = temp_dir().join(format!("t27-scaffold-{}-{}", process::id(), src.len())); ... let _ = std::fs::remove_dir_all(&dir); One process per binary, so the pid is constant and the key is really `src.len()`. Two tests whose sources are the same length get one directory, and under the parallel runner one erases the spec another is mid-read of: t27c prints nothing and the assertion reports an empty result. verilog_imported_enum interpolates only the pid, so all three of its tests shared one directory. Measured rather than inferred. Printing the paths of one scaffold_c run: 6 tests produced THREE directories before, six after. A probe asserting the directory was fresh fired 8 runs out of 8 on const_width, pointer_param_c and verilog_imported_enum -- the collision happens every run and only the timing of the delete decides whether a test dies. scaffold_c failed about one run in three; the others had not fired yet. It passed the first time it was written. A green run does not clear a race. Keyed by an AtomicUsize counter now: unique per call, which is what a directory its own user deletes requires. struct_order_c is the fifth carrier and is fixed in #2949; touching it in both branches would only make a conflict. `tri harness scratch` (--gate, --self-check) keeps the class from coming back. It looks for the conjunction -- more than one #[test], a scratch path under temp_dir(), a remove_dir_all of it, and a key with no per-call component -- because any one alone is fine. Against master it reports exactly the five the probe convicted and nothing else. Two false readings it had to survive, both found by counterexample: Its first rule asked whether the format! call contained a `{`, true of every single-line format call, so a pid-only key looked variable and it reported nothing on verilog_imported_enum while the probe fired 8 of 8. It judges the arguments now. Judging only the arguments then convicted backend_behaviour, whose key is format!("...-{tag}") -- an inline capture, distinct per test, with no argument list at all. Inline captures are read as arguments now. Grepping the symptom rather than the construct also convicts verilog_r_si_1, where src.len() sizes a String and never touches a path. The self-check is five-way: a planted collision and a pid-only key must be seen; counter-keyed, inline-capture and single-test files must not be flagged. A failing leg exits non-zero and says the clean run claims nothing. --gate is deliberately NOT wired into CI here: it stays red until #2949 removes the fifth carrier. Wiring it belongs in the commit after both. Closes #2954 Refs #2949 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

github-actions · 2026-08-30T14:53:12Z

📓 NotebookLM Notebook linked to this PR

Notebook ID: b83263109fb055dc
URL: https://notebooklm.google.com/notebook/b83263109fb055dc

This notebook contains session context, decisions, and artifacts for this work.

github-actions · 2026-08-30T14:53:14Z

PR Dashboard

Generated at: 2026-08-30 14:53:12 UTC

fix(tests): give each test its own scratch directory, and a gate for the class #2955 fix(tests): give each test its own scratch directory, and a gate for the class | Created: 2026-08-30 | Updated: 2026-08-30 | 23 failing / 27 total
fix(zig): declare assert_eq, and give the prelude one emitter (+60 build-obj / +28 deep, 0 regressions) #2953 fix(zig): declare assert_eq, and give the prelude one emitter (+60 build-obj / +28 deep, 0 regressions) | Created: 2026-08-30 | Updated: 2026-08-30 | 2 failing / 36 total
fix(c): forward-declare structs so a self-referencing one compiles (+4, 0 regressions) #2949 fix(c): forward-declare structs so a self-referencing one compiles (+4, 0 regressions) | Created: 2026-08-30 | Updated: 2026-08-30 | 3 failing / 36 total
Land the formal verification harness on master (it has never run) #2727 Land the formal verification harness on master (it has never run) | Created: 2026-08-27 | Updated: 2026-08-27 | 5 failing / 24 total
chore(deps)(deps): bump thiserror from 2.0.18 to 2.0.20 #2693 chore(deps)(deps): bump thiserror from 2.0.18 to 2.0.20 | Created: 2026-08-24 | Updated: 2026-08-28 | 3 failing / 34 total
chore(deps)(deps): bump hyper from 1.10.1 to 1.11.0 #2692 chore(deps)(deps): bump hyper from 1.10.1 to 1.11.0 | Created: 2026-08-24 | Updated: 2026-08-28 | 3 failing / 33 total
chore(deps)(deps): bump base64 from 0.22.1 to 0.23.1 #2691 chore(deps)(deps): bump base64 from 0.22.1 to 0.23.1 | Created: 2026-08-24 | Updated: 2026-08-28 | 3 failing / 34 total
chore(deps)(deps): bump serde from 1.0.228 to 1.0.229 #2690 chore(deps)(deps): bump serde from 1.0.228 to 1.0.229 | Created: 2026-08-24 | Updated: 2026-08-28 | 3 failing / 34 total
chore(deps)(deps): bump uuid from 1.23.2 to 1.25.0 #2689 chore(deps)(deps): bump uuid from 1.23.2 to 1.25.0 | Created: 2026-08-24 | Updated: 2026-08-28 | 3 failing / 34 total
Unbreak FPGA CI and correct two unbacked hardware claims #2081 Unbreak FPGA CI and correct two unbacked hardware claims | Created: 2026-08-11 | Updated: 2026-08-20 | 5 failing / 27 total

Summary

Status	Count
Total Open PRs	10
PRs with Failing Checks	9
PRs with All Checks Green	1
READY	0
FAILING	9
PENDING	0

Seal Status

⚠️ STALE -- sha256(compiler.rs)=4d74b20cbd04 != manifest seal=87e5cbd3ad94.
The committed NMSE numbers were certified against an older compiler.rs.
Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

Refs #2954 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

github-actions · 2026-08-30T14:54:05Z

📓 NotebookLM Notebook linked to this PR

Notebook ID: b83263109fb055dc
URL: https://notebooklm.google.com/notebook/b83263109fb055dc

This notebook contains session context, decisions, and artifacts for this work.

github-actions · 2026-08-30T14:54:12Z

PR Dashboard

Generated at: 2026-08-30 14:54:10 UTC

fix(tests): give each test its own scratch directory, and a gate for the class #2955 fix(tests): give each test its own scratch directory, and a gate for the class | Created: 2026-08-30 | Updated: 2026-08-30 | 16 failing / 27 total
fix(zig): declare assert_eq, and give the prelude one emitter (+60 build-obj / +28 deep, 0 regressions) #2953 fix(zig): declare assert_eq, and give the prelude one emitter (+60 build-obj / +28 deep, 0 regressions) | Created: 2026-08-30 | Updated: 2026-08-30 | 3 failing / 3 total
fix(c): forward-declare structs so a self-referencing one compiles (+4, 0 regressions) #2949 fix(c): forward-declare structs so a self-referencing one compiles (+4, 0 regressions) | Created: 2026-08-30 | Updated: 2026-08-30 | 25 failing / 27 total
Land the formal verification harness on master (it has never run) #2727 Land the formal verification harness on master (it has never run) | Created: 2026-08-27 | Updated: 2026-08-27 | 5 failing / 24 total
chore(deps)(deps): bump thiserror from 2.0.18 to 2.0.20 #2693 chore(deps)(deps): bump thiserror from 2.0.18 to 2.0.20 | Created: 2026-08-24 | Updated: 2026-08-28 | 3 failing / 34 total
chore(deps)(deps): bump hyper from 1.10.1 to 1.11.0 #2692 chore(deps)(deps): bump hyper from 1.10.1 to 1.11.0 | Created: 2026-08-24 | Updated: 2026-08-28 | 3 failing / 33 total
chore(deps)(deps): bump base64 from 0.22.1 to 0.23.1 #2691 chore(deps)(deps): bump base64 from 0.22.1 to 0.23.1 | Created: 2026-08-24 | Updated: 2026-08-28 | 3 failing / 34 total
chore(deps)(deps): bump serde from 1.0.228 to 1.0.229 #2690 chore(deps)(deps): bump serde from 1.0.228 to 1.0.229 | Created: 2026-08-24 | Updated: 2026-08-28 | 3 failing / 34 total
chore(deps)(deps): bump uuid from 1.23.2 to 1.25.0 #2689 chore(deps)(deps): bump uuid from 1.23.2 to 1.25.0 | Created: 2026-08-24 | Updated: 2026-08-28 | 3 failing / 34 total
Unbreak FPGA CI and correct two unbacked hardware claims #2081 Unbreak FPGA CI and correct two unbacked hardware claims | Created: 2026-08-11 | Updated: 2026-08-20 | 5 failing / 27 total

Summary

Status	Count
Total Open PRs	10
PRs with Failing Checks	7
PRs with All Checks Green	3
READY	0
FAILING	7
PENDING	0

Seal Status

⚠️ STALE -- sha256(compiler.rs)=4d74b20cbd04 != manifest seal=87e5cbd3ad94.
The committed NMSE numbers were certified against an older compiler.rs.
Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

Three conflicts, and each one is resolved by regenerating rather than by choosing a side: FROZEN_HASH -- recomputed from the merged compiler.rs, which now carries both this branch's forward declarations and master's Zig prelude. 338 seal files -- the merged emitter's output is what they must describe, so they are recomputed. `tri seals drift` reads 0 across three runs with a binary the tool itself reports FRESH; the first reading of 338 was taken before the rebuild finished and is not the measurement. .claude/skills/ci-gates/SKILL.md -- master had taken 382-386 for another agent's sections while this branch used 382-389. Master's numbering is kept EXACTLY as it stands and mine are appended as 387-394. The last time a skill conflict was resolved here the other side was renumbered, which broke every cross-reference into it; the rule is that the side already on master does not move. The commit messages and PR body on this branch still say 382-389. They describe the sections by title, and the titles did not change. `tri harness scratch` now reports NONE: struct_order_c was the fifth carrier of the shared-scratch-directory class and it is fixed here, so #2955's gate can be wired in a follow-up as its PR body said. t27c 0 failing test binaries, tri 391 passed. Refs #2948 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

…ratch gate (#2958) * fix(ci): count the files this gate does not read, and wire the scratch gate `check_pr_branch_filters.py` printed merge-critical workflows checked: 15 workflow files present: 49 explicitly not merge-critical: 4 CLEAN: no merge-critical workflow filters pull_request by branch. 15 + 4 against 49, printed side by side and never subtracted. Thirty files were read by nothing and the last line still said CLEAN. Two of the thirty carried `pull_request: branches: [master]` -- the very defect this check exists to detect. A branches filter on pull_request means the gate does not run at all when a PR targets any other base, so a stacked PR shows a green check list with that gate absent from it. One of the two is corpus-ratchet, the expected-failure ledger. Three parts: the two filters are removed -- `paths:` selects by what changed and stays, `branches:` selects by where the change is headed and is a hole; the third bucket is printed with its arithmetic closing against the file count, and the clean line says how many files remain unread instead of promising more than it checked; MAX_UNCLASSIFIED = 27, down only. Twenty-seven files cannot be classified in the commit that finds them, and a gate red on the day it lands teaches everyone to ignore red. The ceiling buys that the NEXT workflow cannot land unread. The same read runs over the unclassified files and is REPORTED, not failed: whether one should block a merge is a human call, whether anybody looked is not. That count is 0 and is printed as a zero rather than omitted. Controls, each seen failing on purpose: a 28th unclassified workflow gives UNCLASSIFIED ROSE 27 -> 28; a name in both lists gives IN BOTH LISTS; restoring the filter on corpus-ratchet gives BRANCH-FILTERED MERGE-CRITICAL WORKFLOWS. All restored, exit 0. Also here: harness-scratch.yml, which #2955 held back because the gate stayed red until #2949 removed the fifth carrier of that class. Both have landed -- on master the command reports `none` and `--gate` exits 0. It carries no `paths:` filter and a push trigger on master, and its header says why: emit-bitexact-gate.yml is pull_request with a paths filter and no push, so it has never run on master, and when a change made it fail there was no baseline to compare against. Historical control: against the tree before #2955 the gate exits 1 and names all five carriers. The remaining 27 are a work list, not a verdict; l1-traceability.yml is the one worth deciding first. Closes #2957 Refs #2954 Refs #2919 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(skill): 395 -- a guard's clean line is a claim about a population The two list sizes and the file count were printed three lines apart and never subtracted: 15 + 4 against 49 present, so 30 workflow files were read by nothing and the summary still said CLEAN. Two of the thirty carried the defect the check exists to detect. Make the parts sum out loud; name the remainder as a third state; run the same read over it and report rather than fail; ceiling not refusal so the next addition cannot land unread; and let the final line state the scope it actually earned. Refs #2957 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

docs(now): entry for the scratch-directory race

43f5fa3

Refs #2954 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

gHashTag merged commit c22013d into master Aug 30, 2026
36 checks passed

This was referenced Aug 30, 2026

Gate Topology reported CLEAN over 19 of 49 workflow files, and two of the unread 30 carried its own defect #2957

Closed

fix(ci): count the files Gate Topology does not read, and wire the scratch gate #2958

Merged

Navigation Menu

fix(tests): give each test its own scratch directory, and a gate for the class - #2955

fix(tests): give each test its own scratch directory, and a gate for the class#2955
gHashTag merged 2 commits into
masterfrom
w792

gHashTag commented Aug 30, 2026

Uh oh!

github-actions Bot commented Aug 30, 2026

Uh oh!

github-actions Bot commented Aug 30, 2026

Uh oh!

github-actions Bot commented Aug 30, 2026

Uh oh!

github-actions Bot commented Aug 30, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

gHashTag commented Aug 30, 2026

The defect

Measurement, not inference

The fix

tri harness scratch

Uh oh!

github-actions Bot commented Aug 30, 2026

Uh oh!

github-actions Bot commented Aug 30, 2026

PR Dashboard

Summary

Seal Status

Uh oh!

github-actions Bot commented Aug 30, 2026

Uh oh!

github-actions Bot commented Aug 30, 2026

PR Dashboard

Summary

Seal Status

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

`tri harness scratch`