feat(tri): vsim funnel -- how far each spec gets when its Verilog is RUN - #2993
Merged
Conversation
The one arm that can catch a defect whose nature is that it COMPILES is the one that simulates, and its gate has had no targets since specs/scratch was untracked in #2283 (#2987). Pointing it at the corpus by hand produced four defects in one sitting: #2988, #2989, #2990 and #2869. Repairing the gate is half a line and half a decision about 260 baselines for subjects that no longer exist. The measurement should not wait for it. Measured at 487003f over 650 specs: refused by gen-verilog-for-simulation 69 iverilog rejected the testbench 410 simulation exited non-zero 0 ran and REPORTED FAILURES 10 ran and produced a verdict 106 ran, exit 0, and said NOTHING 55 650 of 650 accounted for. Three properties worth naming: * is its own row. run_icarus_simulate bails only on a line saying FAILED, so `checked everything and passed` and `checked nothing` are the same exit code. A spec there is not passing; nothing asked it anything. * One TMPDIR per spec, because run_icarus_simulate keys its scratch file on the BASENAME alone and this corpus shares stems. * It refuses when iverilog or t27c is absent rather than printing a table it did not earn, and asks the OS rather than matching an error message (ci-gates 426). It reports and never gates: which of the 10 are compiler defects and which are spec defects is not a question a walker may answer. Refs #2987
The message for `tri vsim funnel` was passed inline to `git commit -m "..."`.
Double quotes do not stop command substitution, so the span naming the `silent`
row was executed and removed:
* `silent` is its own row. -> * is its own row.
zsh printed `command not found: silent`, which is the only reason it was seen.
The commit is pushed and this repository forbids force-pushing, so the message
stands with the hole and this is the correction. The missing word is `silent` --
the row for a spec that ran, exited 0, and printed no verdict line at all.
ci-gates 418 already says to quote a heredoc delimiter when the body carries a
backtick. That is too narrow: this went through the argument list instead. 427
names the class -- prose containing a backtick never reaches the shell as an
argument; write it to a file or pipe it through a QUOTED heredoc and let git
read it with `-F -`.
Third occurrence in this repository's log, which by its own rule means the cure
was wrong rather than that this is another case.
Refs #2987
Contributor
|
📓 NotebookLM Notebook linked to this PR
This notebook contains session context, decisions, and artifacts for this work. |
Contributor
# Conflicts: # .claude/skills/ci-gates/SKILL.md
…cking gate
Measured with a committed entry in the wrong shape:
tri hooks now-gate exit 0 "NOW gate PASSED"
t27c check-now exit 0 "build authorized"
scripts/verify.sh "gates:OK"
check_now_entry_shape exit 1 "FAIL: 1 of 3 entr(y/ies) do not say
anything checkable"
The last is the REQUIRED `check` context. docs/now/README.md documents the other
gate's three conditions and then says the same three are what the local tools
enforce; the shape condition -- a `# NOW -- <title> (YYYY-MM-DD)` first line and
a `## ` section heading -- is previewed by nothing. The two local tools also name
a different entry than the one the change adds: they answer "is the newest entry
fresh", not "is the added entry well formed".
Found by paying for it: this made `check` red on #2991.
Refs #2987
Contributor
|
📓 NotebookLM Notebook linked to this PR
This notebook contains session context, decisions, and artifacts for this work. |
Contributor
PR DashboardGenerated at: 2026-08-30 21:55:33 UTC
Summary
Seal Status
|
gHashTag
added a commit
that referenced
this pull request
Aug 30, 2026
`tri vsim funnel` shipped in #2993 with two arms returning `Stage::Gen` for states nobody had measured: a process that could not be spawned, and a failure whose message matched none of the four patterns. Both file a hole in this reader as a verdict about the spec -- the shape ci-gates 428 names, written in the same session and then present in my own code. * new `Unattributed` stage, with its own printed row * attribution extracted into `attribute_failure(&log)`, so the decision can be tested without spawning anything * two tests: a message matching no arm is Unattributed and NOT Gen; and each of the four arms is reachable by the compiler's own words Measured over 650 specs at d7f20a5: the new row reads 0 and every other row is identical (69 / 410 / 0 / 9 / 107 / 55 / 0). Nothing was being misfiled today -- the change is preventive, and the zero is printed beside the hits because a reader cannot otherwise tell "none" from "not looked for". Refs #2987
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The one arm of this compiler that can catch a defect whose whole nature is that
it compiles is the one that simulates. Its gate has had no targets since
specs/scratchwas untracked in #2283 —tri vsim funneldoes not repair that(#2987 is half a line and half a decision about 260 baselines for subjects that
no longer exist), it makes the measurement available while the decision waits.
Pointing it at the corpus by hand, once, produced four defects in one sitting:
#2988, #2989, #2990 and #2869.
Measured at
487003fb9, over 650 specsThe ten reporting failures right now:
fpga/bpsk,fpga/mac,fpga/spi,fpga/top_level,fpga/verification/build_verify,numeric/gft64,numeric/gft128,numeric/gft256,server/api,trinet/etx. (etxis theone #2991 repairs, so this list is nine the moment that lands.)
Three properties worth naming
silentis its own row.run_icarus_simulatebails only on a line sayingFAILED, so the testbench checked everything and passed and the testbench
checked nothing are the same exit code. A spec in that row is not passing —
nothing asked it anything.
run_icarus_simulatewrites<temp_dir>/t27c_icarus_<file_stem>.v, keyed on the BASENAME alone, and thiscorpus shares stems — two specs would overwrite each other's Verilog.
iverilogonPATHeveryspec lands in
elaband the table reads as a corpus-wide regression, so thecommand says so and exits non-zero. It asks the OS rather than matching an
error message — ci-gates 426, written after that exact guard failed in CI on
fix(verilog): a real value must not travel in an integer container (+6 declared tests, 0 regressions) #2991 an hour earlier.
It reports and never gates. Which of the ten are compiler defects and which are
spec defects is not a question a walker may answer, and #2987 says so.
Honesty
answer available:
: startingand: NOT CHECKEDare not verdicts, andcounting either would make the
silentrow permanently zero.stop.
rustfmtapplied to the new leaf module only;git statusconfirms no otherfile moved (the crate-root trap from an earlier pass).
silentto a backtick substitutionin
git commit -m "...". It is pushed and this repository forbidsforce-pushing, so the second commit carries the correction and ci-gates 427
closes the class that 418 named too narrowly.
Refs #2987
🤖 Generated with Claude Code