The measurement in #3349 lived in a shell script in /tmp. The next session would rebuild it, or — likelier — take the largest first-error class again and get another +0.
tri one-away reads every spec, generates Rust, runs rustc, and buckets the failures by how many REAL errors they carry:
specs read 650
rustc accepted 338
did not GENERATE 69 (not zero errors -- a different fact)
rustc refused 243
real errors per failing spec, the summary line excluded:
1: 56
2: 37
3: 19
...
56 spec(s) carry exactly 1 error, and only those can be moved by one repair.
their sole class:
8 cannot find type `X` in this scope
7 expected one of `X`, ..., found `X`
5 expected type, found keyword `X`
5 mismatched types
4 use of moved value: `X`
The summary line is excluded, and that is the whole point. rustc ends with error: aborting due to N previous errors, which matches ^error like any diagnostic. Counting it inflates every bucket by one and turns 56 specs with one error into zero — the opposite conclusion, and the one written down first in #3349.
Six controls, one per claim, including that counting the summary would give the opposite answer. Five unit tests over the two pure functions.
Its 56 disagrees with the 55 measured by hand in #3349, and the difference is resolved rather than averaged: vsa/jones_polynomial.t27, the one spec the vec! repair in #3350 moved from two errors to one. The hand count ran against the binary from before that merge.
The measurement in #3349 lived in a shell script in
/tmp. The next session would rebuild it, or — likelier — take the largest first-error class again and get another +0.tri one-awayreads every spec, generates Rust, runs rustc, and buckets the failures by how many REAL errors they carry:The summary line is excluded, and that is the whole point.
rustcends witherror: aborting due to N previous errors, which matches^errorlike any diagnostic. Counting it inflates every bucket by one and turns 56 specs with one error into zero — the opposite conclusion, and the one written down first in #3349.Six controls, one per claim, including that counting the summary would give the opposite answer. Five unit tests over the two pure functions.
Its 56 disagrees with the 55 measured by hand in #3349, and the difference is resolved rather than averaged:
vsa/jones_polynomial.t27, the one spec thevec!repair in #3350 moved from two errors to one. The hand count ran against the binary from before that merge.