Conversation
§317 ended with "That is a five-line script, not an audit -- and it
would have found all four." No such script existed when I wrote that.
Built and measured:
naive diff of constructed variants against each list
-> 10 lists of 10 flagged. Almost every list is a legitimate
subset, so it finds everything, which is finding nothing.
the same grouped into families (control flow / binding / exit),
flagging a list that covers PART of a family
-> 3 of 10, and on the commit before #2875 it points at the exact
`has_body` line. But all three of its hits on a clean tree are
correct code: `StmtLocal | StmtAssign` collects NAMED bindings
and `StmtExpr` has no name; the loop arm at 10979 excludes
`StmtIf` on purpose.
two of the four are not NodeKind lists at all: `compound_binop` maps
operator strings and `expr_is_bool` is a match whose missing arm
nobody enumerated.
So: 1 of 4, three false positives. `tri kinds drift` also finds 1 of 4
and costs zero false positives, which is why that is the one that
shipped. §317 now carries these numbers instead of the prediction.
319 is the general form. A sentence in the conditional tense, inside a
document whose purpose is to hold measurements, takes authority from the
numbers around it -- and unlike a wrong claim about existing code, it
cannot be checked until somebody builds the thing. Either build it and
write the number, or write "untested" beside it.
Refs #2876
gHashTag
enabled auto-merge (squash)
August 29, 2026 20:08
Contributor
|
📓 NotebookLM Notebook linked to this PR
This notebook contains session context, decisions, and artifacts for this work. |
Contributor
PR DashboardGenerated at: 2026-08-29 20:08:41 UTC
Summary
Seal Status
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
§317, written one iteration ago, ended with:
No such script existed when I wrote that sentence. It was a prediction wearing the grammar of a measurement, in a document whose whole purpose is to hold measurements.
Built and measured
has_bodylinetri kinds drift(comment vs pattern)StmtIfarmThe naive diff finds everything, which is finding nothing: almost every list in the file is a legitimate subset of the constructed kinds.
The family version discriminates and does find the real defect on history — but all three of its hits on a clean tree are correct code:
StmtLocal | StmtAssignat 7242 and 7290 collects named bindings, andStmtExprhas no name;StmtForRange | StmtWhile | StmtForat 10979 sits under "Do NOT recurse into loop bodies" —StmtIfis absent on purpose.And two of the four defects are not NodeKind lists at all:
compound_binopmaps operator strings, andexpr_is_boolis amatchwhose missing arm nobody enumerated.Score: the enumeration-diff finds 1 of 4 and costs three false positives.
tri kinds driftfinds 1 of 4 and costs zero. That is why the second one is the one that shipped, and §317 now carries these numbers instead of the prediction.§319 — the general form
A sentence in the conditional tense, inside a document of findings, takes its authority from the numbers around it. And unlike a wrong claim about existing code — an issue reasoning from absence (§240), a hypothesis reported as a rule (#2830) — it cannot be checked at all until somebody builds the thing.
Refs #2876