What
fpga-build.yml's conformance step, landed by #2403, cannot fail. Line 772:
if python3 tools/run_conformance_vvp.py "$m" "$v" build/fpga/conformance | tee "build/fpga/conformance/${m}.log"; then
echo "| $m | EXECUTED, all cases passed |" >> $GITHUB_STEP_SUMMARY
else
echo "| $m | FAILED (see log) |" >> $GITHUB_STEP_SUMMARY
fails=$((fails+1))
fiWithout pipefail the if tests tee's exit status. A failing runner takes the then
branch, fails stays 0, and the exit 1 at :784 never fires.
The step has no set -o pipefail, and the workflow has no defaults: run: shell: block, so
the default bash -e {0} applies — -o pipefail is added only for an explicit
shell: bash.
Measured, with a stub runner that fails the way a real mismatch would
$ cat runner.sh
#!/bin/sh
echo "case 3: MISMATCH expected 0x2a got 0x00"
exit 1
As landed on master:
case 3: MISMATCH expected 0x2a got 0x00
| mac | EXECUTED, all cases passed |
fails=0
STEP EXIT 0 — job green
With set -o pipefail, same runner:
case 3: MISMATCH expected 0x2a got 0x00
| mac | FAILED |
fails=1
::error::conformance execution failed
step exit 1
The gate reports "all cases passed" on the line directly below its own log saying
MISMATCH.
This is the defect #2242 removed, in the same file
#2242 fixed exactly this in fpga-formal, and the corrected pattern sits 115 lines
above the new one, at :638-640:
# Audit 2026-08-19: without pipefail the if tested tee's exit (always 0),
set -o pipefail
On the negative control
#2403's body reports a planted-fault control failing correctly, and I have no quarrel with
it: it demonstrates the runner detects a fault. It does not demonstrate the gate
fails, because the gate reads tee. Those are different claims and only the second is what
CI enforces. fpga-conformance going green is consistent with both a working gate and a
vacuous one — which is precisely why the control has to run through the workflow step, not
beside it.
Raised on the PR before merge (#2403 review comment) and merged unchanged; filing so it is
tracked rather than lost.
Not in dispute
The rest of #2403 is good and this issue does not touch it: registry-scoped execution with
the remainder printed as visible debt is the honest shape, and its own comment — "a thin
gate that is real beats a broad one that is vacuous" — is exactly the right doctrine.
Related: #2241, #2242, #2403, #2376.
What
fpga-build.yml's conformance step, landed by #2403, cannot fail. Line 772:Without
pipefailtheifteststee's exit status. A failing runner takes thethenbranch,
failsstays 0, and theexit 1at :784 never fires.The step has no
set -o pipefail, and the workflow has nodefaults: run: shell:block, sothe default
bash -e {0}applies —-o pipefailis added only for an explicitshell: bash.Measured, with a stub runner that fails the way a real mismatch would
As landed on master:
With
set -o pipefail, same runner:The gate reports "all cases passed" on the line directly below its own log saying
MISMATCH.
This is the defect #2242 removed, in the same file
#2242fixed exactly this infpga-formal, and the corrected pattern sits 115 linesabove the new one, at
:638-640:On the negative control
#2403's body reports a planted-fault control failing correctly, and I have no quarrel with
it: it demonstrates the runner detects a fault. It does not demonstrate the gate
fails, because the gate reads
tee. Those are different claims and only the second is whatCI enforces.
fpga-conformancegoing green is consistent with both a working gate and avacuous one — which is precisely why the control has to run through the workflow step, not
beside it.
Raised on the PR before merge (#2403 review comment) and merged unchanged; filing so it is
tracked rather than lost.
Not in dispute
The rest of #2403 is good and this issue does not touch it: registry-scoped execution with
the remainder printed as visible debt is the honest shape, and its own comment — "a thin
gate that is real beats a broad one that is vacuous" — is exactly the right doctrine.
Related: #2241, #2242, #2403, #2376.