The fourth vacuous green. fd245d2aa (#2242, Closes #2239) caught three jobs that were green while verifying nothing: fpga-conformance (bare -g2005 compiled 0/32 behind a ::warning), fpga-formal (continue-on-error: true, plus if sby | tee testing tee without pipefail), and fpga-lint's readiness step (run_synth_readiness printed NOT READY and exited 0). That audit fixed the reporting of fpga-lint's yosys loop — a lint failure now exit 1s. It did not ask whether the loop reaches the defects. It does not.
fpga-lint is not vacuous in the earlier sense: it runs, it can fail, and it did catch #2240. It is vacuous in a narrower and more durable way — its verdict is green on files that do not elaborate, and no amount of failing-harder fixes that, because the defective code is never reached.
Measured
One artifact, both loops, run by hand: the fpga-verilog artifact of fpga-build.yml run #1037 (head bba06449fe37c42e2ca021cc719138b818545d74), 32 .v files.
| loop |
tool |
result |
fpga-lint's exact loop — read_verilog -sv -DSIMULATION + hierarchy -top |
Yosys 0.63 |
pass=32 fail=0 |
fpga-conformance's exact loop — iverilog -g2012 -DSIMULATION |
Icarus 13.0 |
pass=4 fail=28 |
The four that compile: HwTypes, SPI_Master, ZeroDSP_TopLevel, ZeroDSP_UART.
This is not a local artifact. In run #1037 itself, fpga-lint concluded success and fpga-conformance concluded failure, against the same uploaded artifact, in the same run.
Why hierarchy misses it
Yosys elaborates a function body only where the function is called. These files almost never call their own functions. Counting definitions against call sites across the 32 files (regex over comment-stripped source: function [range] name; vs. any other name( occurrence):
485 functions defined, 79 with at least one call site, 406 with none.
31 of the 32 files are effectively function libraries: 5–33 functions each against exactly 1–2 always/assign blocks. Only zerodsp_top is a structural module (0 functions, 15 logic blocks). So hierarchy validates the module header and the one or two logic blocks, and walks past the rest.
Decisive check — inject a single call site into fifo.v:
wire [31:0] __probe_aw;
assign __probe_aw = addr_width(32'd0);
read_verilog -sv -DSIMULATION + hierarchy -top Fifo goes from silent-rc=0 to:
fifo_probe.v:174: ERROR: Non-constant expression in constant function
fifo.v:174 is d = cfg_depth; — the exact line Icarus reports as Unable to bind wire/reg/memory `cfg_depth' in `Fifo.addr_width.addr_width_body' . Same defect, same line. Reachable only through a call site.
Every Icarus error in the 28 is scoped to a *_body block — Fifo.pop.pop_body, ZeroDSP_MAC.mac_cycle.mac_cycle_body, Memory.add_read_port.add_read_port_body. That is precisely the region hierarchy does not enter.
Deepening the yosys pass does not work — measured, not assumed
The obvious remedy is to run yosys deeper. It buys nothing here, because depth is not the axis; reachability is. On fifo.v, all four of these exit 0:
hierarchy -top Fifo
hierarchy -top Fifo -check
hierarchy -top Fifo; proc
synth -top Fifo
Across all 32 files, full synth -top <module> is 31 pass / 1 fail. The single failure is zerodsp_top:
ERROR: Module `\ZeroDSP_TopLevel' referenced in module `\zerodsp_top' in cell `\u_top_level' is not part of the design.
— a missing-submodule artifact of compiling one file at a time, not a codegen defect. So promoting fpga-lint to full synthesis would flag 1 of the 28, for the wrong reason, and still certify the other 27 as clean. An elaboration gate that reaches these bodies has to be a tool that elaborates uncalled functions, which is what fpga-conformance already does with Icarus.
What is actually wrong
The per-row table is honest — its columns are literally | Module | read_verilog | hierarchy | Status |. Two things over-claim on top of it:
- the summary line,
**Result:** $pass/$total modules passed Yosys lint, which reads as a clean bill for the file;
- the job name
fpga-lint, which is the only thing most readers see in the check list.
"32/32 modules passed Yosys lint" is true of what ran and false of what it suggests. The 28 modules it certified do not elaborate.
Options (gate policy — owner's call)
- Minimum honest change, no new red. Reword the summary to state the depth, e.g.
32/32 parsed and resolved hierarchy (read_verilog + hierarchy; function bodies are not elaborated — see fpga-conformance), and rename the job to something like fpga-lint (parse+hierarchy). Costs nothing, ends the false read.
- Make the summary carry the conformance result so the two depths are never read apart.
- Add a real elaboration gate. It must be Icarus-class, not deeper yosys. It is red today for 28 modules and stays red until the emitter defects land — so this is a decision to accept a known red, not a fix.
Option 1 is the part that is unambiguously correct regardless of what is decided about 3.
Root cause of the 28
Emitter defects, tracked separately: #2275 (imported-enum members never declared; struct param field flattened to an unbound name — that issue already notes mac.v "passes yosys lint (read_verilog+hierarchy) but fails iverilog elaboration", for one module) and #2319 (TernaryWord sized 40 bits, correct is 3712). #2240 was the same family and is closed. This issue is not a request to fix those; it is about a gate that reports green across all 32 while they are open.
Refs #2239, #2242, #2241, #2275, #2319, #2240.
Reproduction
gh run download 32464326319 --repo gHashTag/t27 -n fpga-verilog -D ./art1037
cd ./art1037
for v in *.v; do
name=$(sed -n 's/^module \([A-Za-z0-9_]*\).*/\1/p' "$v" | head -1)
yosys -p "read_verilog -sv -DSIMULATION $v; hierarchy -top $name" -q 2>/dev/null \
&& echo "PASS $name" || echo "FAIL $name"
done
Swap the yosys line for iverilog -o /dev/null -g2012 -DSIMULATION "$v" for the other column.
Local tools: Yosys 0.63 (git sha1 70a11c6bf0e8dd669f56c7da3587f78b405138e2), Icarus Verilog 13.0 (stable, v13_0). CI installs both from apt on ubuntu-latest, so exact versions there may differ — but the divergence does not rest on my local versions: run #1037 shows it directly, fpga-lint green and fpga-conformance red on one artifact.
The fourth vacuous green.
fd245d2aa(#2242, Closes #2239) caught three jobs that were green while verifying nothing:fpga-conformance(bare-g2005compiled 0/32 behind a::warning),fpga-formal(continue-on-error: true, plusif sby | teetestingteewithoutpipefail), andfpga-lint's readiness step (run_synth_readinessprintedNOT READYand exited 0). That audit fixed the reporting offpga-lint's yosys loop — a lint failure nowexit 1s. It did not ask whether the loop reaches the defects. It does not.fpga-lintis not vacuous in the earlier sense: it runs, it can fail, and it did catch #2240. It is vacuous in a narrower and more durable way — its verdict is green on files that do not elaborate, and no amount of failing-harder fixes that, because the defective code is never reached.Measured
One artifact, both loops, run by hand: the
fpga-verilogartifact offpga-build.ymlrun #1037 (headbba06449fe37c42e2ca021cc719138b818545d74), 32.vfiles.fpga-lint's exact loop —read_verilog -sv -DSIMULATION+hierarchy -topfpga-conformance's exact loop —iverilog -g2012 -DSIMULATIONThe four that compile:
HwTypes,SPI_Master,ZeroDSP_TopLevel,ZeroDSP_UART.This is not a local artifact. In run #1037 itself,
fpga-lintconcluded success andfpga-conformanceconcluded failure, against the same uploaded artifact, in the same run.Why
hierarchymisses itYosys elaborates a function body only where the function is called. These files almost never call their own functions. Counting definitions against call sites across the 32 files (regex over comment-stripped source:
function [range] name;vs. any othername(occurrence):485 functions defined, 79 with at least one call site, 406 with none.
31 of the 32 files are effectively function libraries: 5–33 functions each against exactly 1–2
always/assignblocks. Onlyzerodsp_topis a structural module (0 functions, 15 logic blocks). Sohierarchyvalidates the module header and the one or two logic blocks, and walks past the rest.Decisive check — inject a single call site into
fifo.v:read_verilog -sv -DSIMULATION+hierarchy -top Fifogoes from silent-rc=0 to:fifo.v:174isd = cfg_depth;— the exact line Icarus reports asUnable to bind wire/reg/memory `cfg_depth' in `Fifo.addr_width.addr_width_body'. Same defect, same line. Reachable only through a call site.Every Icarus error in the 28 is scoped to a
*_bodyblock —Fifo.pop.pop_body,ZeroDSP_MAC.mac_cycle.mac_cycle_body,Memory.add_read_port.add_read_port_body. That is precisely the regionhierarchydoes not enter.Deepening the yosys pass does not work — measured, not assumed
The obvious remedy is to run yosys deeper. It buys nothing here, because depth is not the axis; reachability is. On
fifo.v, all four of these exit 0:Across all 32 files, full
synth -top <module>is 31 pass / 1 fail. The single failure iszerodsp_top:— a missing-submodule artifact of compiling one file at a time, not a codegen defect. So promoting
fpga-lintto full synthesis would flag 1 of the 28, for the wrong reason, and still certify the other 27 as clean. An elaboration gate that reaches these bodies has to be a tool that elaborates uncalled functions, which is whatfpga-conformancealready does with Icarus.What is actually wrong
The per-row table is honest — its columns are literally
| Module | read_verilog | hierarchy | Status |. Two things over-claim on top of it:**Result:** $pass/$total modules passed Yosys lint, which reads as a clean bill for the file;fpga-lint, which is the only thing most readers see in the check list."32/32 modules passed Yosys lint" is true of what ran and false of what it suggests. The 28 modules it certified do not elaborate.
Options (gate policy — owner's call)
32/32 parsed and resolved hierarchy (read_verilog + hierarchy; function bodies are not elaborated — see fpga-conformance), and rename the job to something likefpga-lint (parse+hierarchy). Costs nothing, ends the false read.Option 1 is the part that is unambiguously correct regardless of what is decided about 3.
Root cause of the 28
Emitter defects, tracked separately: #2275 (imported-enum members never declared; struct param field flattened to an unbound name — that issue already notes
mac.v"passes yosys lint (read_verilog+hierarchy) but fails iverilog elaboration", for one module) and #2319 (TernaryWordsized 40 bits, correct is 3712). #2240 was the same family and is closed. This issue is not a request to fix those; it is about a gate that reports green across all 32 while they are open.Refs #2239, #2242, #2241, #2275, #2319, #2240.
Reproduction
Swap the yosys line for
iverilog -o /dev/null -g2012 -DSIMULATION "$v"for the other column.Local tools: Yosys 0.63 (git sha1
70a11c6bf0e8dd669f56c7da3587f78b405138e2), Icarus Verilog 13.0 (stable,v13_0). CI installs both from apt onubuntu-latest, so exact versions there may differ — but the divergence does not rest on my local versions: run #1037 shows it directly,fpga-lintgreen andfpga-conformancered on one artifact.