feat(tri): tri gates empty -- what still passes when there is nothing to check - #3016
Merged
Conversation
… to check Runs every gate invocation the workflows write, verbatim, against a tree that holds the scripts and no data. Of 36 runnable invocations, 31 refuse and 5 pass; four of the five are self-contained self-tests and the fifth prints "tracked files read 0". This axis is healthy and the number is worth having written down. The population is the COMMAND LINE, not the script. By script name the same probe reads 12 of 38 -- seven of those twelve are invoked with --require, which is exactly the flag that turns a SKIP branch into a failure. A gate is what it is called with. Exclusions are printed with their reason: a line continued with a backslash, a line inside a command substitution, and a --self-check run whose subject is the gate rather than the tree. A population that silently shrinks is the defect this command exists to look for. Complements tools/check_gate_preconditions.py, which covers 6 of 19 tools/ scripts with hand-written expectations and says so; this measures the whole CI-invoked population behaviourally, scripts/ included. Refs #2994 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… tree that was not empty 441: 34 of 34 refuse (every one was 'timeout: command not found' on macOS), then 12 of 38 pass (measured by script name, where CI writes --require), then 5 of 36 (the invocations as written). A gate is what it is called with, and a clean audit is a result. 442: the probe's own scratch tree carried tools/withdrawn.txt and every baseline, so two gates read backwards. Caught by running them by hand and disagreeing with my own command. Refs #2994 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
|
📓 NotebookLM Notebook linked to this PR
This notebook contains session context, decisions, and artifacts for this work. |
Contributor
…4-445) The parallel session landed 441-443 while this branch was open. Refs #2994
Contributor
PR DashboardGenerated at: 2026-09-03 10:53:32 UTC
Summary
Seal Status
|
Contributor
|
📓 NotebookLM Notebook linked to this PR
This notebook contains session context, decisions, and artifacts for this work. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three readings of one question, and only the third measured the gate
Which gates report success over a tree with nothing in it?
34 of 34 refuse. Every gate copied into an empty directory, run under
timeout 25. Clean, plausible, entirely false:timeoutdoes not exist on macOS, so all 34 exited 127 withcommand not foundand the classifier binned every one as a refusal. The tell was the table itself — a classifier that puts 34 of 34 in one bucket is describing its input.12 of 38 pass. Real runs, by script name. Also wrong: CI does not invoke a script, it invokes a command line. Seven of those twelve are written
--requirein the workflow, which is exactly the flag that turns theirSKIP: t27c is not builtbranch into a failure.5 of 36 — the invocations as the workflows write them.
Four are self-contained self-tests, green anywhere by construction. The fifth prints
tracked files read 0before it says the tree is clean — a gate telling a reader it read nothing. This axis is healthy, and after three passes that each found a defect that is worth saying plainly rather than hunting until something breaks.What shipped
tri gates empty [--verbose]extracts everypython3 tools/…/bash scripts/…line the workflows write, verbatim, runs each in a tree holding the scripts and no data, and reports what still exits 0.Exclusions are printed with the reason, never dropped: a line continued with
\, a line inside a$( … )substitution, and a--self-checkrun whose subject is the gate rather than the tree. A population that silently shrinks is the defect this command exists to look for. A population of zero is a hard refusal, because it would print as "every gate refuses an empty tree".It complements
tools/check_gate_preconditions.py, which plants the same empty world for 6 of 19tools/scripts with hand-written expectations and already names the 13 it does not reach. This measures the whole CI-invoked population behaviourally,scripts/included, and agrees with an independent shell probe on all five.The defect in my own probe
The first version copied every file out of
tools/andscripts/, carryingtools/withdrawn.txtand every baseline into the "empty" tree. Not an error — a plausible table with two rows inverted:check_withdrawn_live.py, whose whole job is to refuse when its register is missing, found the register and passed. Caught by running the two by hand and disagreeing with my own command.An empty tree that carries the data is not an empty tree. The repair is an extension filter, and its test plants
gate.py,helper.sh,withdrawn.txt,baseline.jsonand demands exactly the two scripts arrive.Controls
cargo test -p tri--self-checkas runnablepython3 …counts as a gateSkill §441–442.
Refs #2994
🤖 Generated with Claude Code