fix(pi): a 5s test deadline cannot cover a cold Python launch on Windows - #496
Merged
Conversation
The pi security-evidence test fails on windows-latest with "Test timed out in 5000ms" - a harness timeout, not an assertion. Seen three times on 2026-07-26 across two branches and both pinned Pi runtimes, while ubuntu-latest and macos-latest passed every one of those runs. It fails on main too (run 30203161506), so it is not branch-specific. The script is not the slow part: `test_pi_security.py --contract-only` runs in about 70ms, and it imports nothing this repository changed recently. What does not fit in 5s is a cold CPython process launch on a loaded Windows runner, and the whole cost lands inside the test's budget because the spawn is synchronous. The deadline was provisioned for the script's runtime when what it has to cover is a process creation. So the timeout is what is wrong, not the code under test, and it is now platform-aware rather than uniformly larger - a genuine hang on the fast platforms still surfaces as a hang. Both tests in the file that spawn Python synchronously get it, so the second does not resurface separately (AC-4). No assertion is touched: the secret-bearing ambient-value checks are unchanged. This is not cosmetic. `[GATE ] | [REPO] | Merge readiness` aggregates this suite, and the release preflight requires that gate green for the EXACT commit being tagged (#385), so a flake here blocks the release lane for that SHA and trains readers to re-run a red gate without reading it. It cost two merge attempts on PR #494. No version bump: plugins/ca-pi/tools/** is outside the shipped payload (payload_scope.py), and this changes a test only. Closes #495 Claude-Session: https://claude.ai/code/session_01Y8UPz5RZwgnda3NbAVfd6L
The audit log is resolved as an append-only union: main's committed order first, then this session's rows. Taking either side outright would drop real gate events, and gate-events.log is an audit artifact (H-05), not a scratch file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The pi security-evidence test gets a platform-aware deadline. Both tests in the file that spawn Python synchronously get it.
Why
test/security.test.ts:412fails onwindows-latestwithError: Test timed out in 5000ms— a harness timeout, not an assertion. Observed three times on 2026-07-26 across two branches and both pinned Pi runtimes, while ubuntu-latest and macos-latest passed every one of those runs. It also fails onmain(run 30203161506, commite0557558), so it is not branch-specific.Three lines of evidence that the code under test is not the problem:
test_pi_security.py --contract-onlyruns in 72ms locally._sloplib,_bashguardlib) — zero references.What does not fit in 5s is a cold CPython process launch on a loaded Windows runner, and the whole cost is billed against the test's own budget because
spawnSyncis synchronous. The deadline was provisioned for the script's runtime when what it has to cover is a process creation.Why it is not cosmetic
[GATE ] | [REPO] | Merge readinessaggregates this suite, and the release preflight requires that gate green for the exact commit being tagged (#385). So a flake here does not just cost a re-run — it blocks the release lane for that SHA, and it trains readers to re-run a red gate without reading it, which is the habit the gate exists to prevent. It cost two merge attempts on #494.How
PYTHON_SPAWN_TIMEOUT_MS = process.platform === "win32" ? 30_000 : 5_000, applied as the third argument to both Python-spawning tests. Platform-aware rather than uniformly larger, so a genuine hang on the fast platforms still surfaces as a hang. The reasoning lives in the code, not just here.No assertion is weakened or removed — the secret-bearing ambient-value checks are byte-identical.
Versions
No bump.
plugins/ca-pi/tools/**is outside the shipped payload perpayload_scope.py, and this changes a test only. Confirmed:payload_scope.py --plugin plugins/ca-pireportsunchanged.Locally:
tsc --noEmitclean,test/security.test.ts19/19 passed.Closes #495
https://claude.ai/code/session_01Y8UPz5RZwgnda3NbAVfd6L