Accepted is not the same as accepted on the whole spec - #2770
Merged
Conversation
The columns say "217 specs produce Zig that Zig takes". None of them has ever
said ON HOW MUCH OF THE SPEC, and 87 specs are accepted while part of their text
is thrown away -- the part being the bodies of invariants, which is to say the
assertions.
specs with tokens DROPPED 87 13.4%
... tokens dropped 32485
Accepted is not the same as accepted ON THE WHOLE SPEC.
`--per-spec` gains a `dropped` column, so the diff that names a moved row also
shows what that row throws away.
docs/PARSER_DISCARD_LANDSCAPE.md is the competitive read behind the design:
* rowan (rust-analyzer, Swift, C#, Kotlin) makes discarding UNREPRESENTABLE --
the source must be reconstructible from the tree, so a parse-complete
command cannot be needed. Cost: two tree layers and every consumer tolerating
error nodes.
* tree-sitter represents the dropped region as an ERROR node, and its own
`tree-sitter parse` still does not report an error when one is present
(tree-sitter#4049). Having the representation is not the same as reporting
it -- this repository's own lesson, arrived at from the other side.
* bison error productions put the skip in the grammar: auditable by reading,
silent about how much was skipped at runtime.
t27 sits closer to tree-sitter, and does one thing more than tree-sitter's CLI:
it counts the discard and ratchets it. The note names the cheap half of rowan's
property that was available and unused -- and this commit writes it.
Skill sections 150-153: when a column moves add the per-item dump; two node
shapes reaching one emitter arm; prove a wrapper is dead before removing it;
a sync command must recompute the truth rather than pick the newer file.
Refs #2754
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
PR DashboardGenerated at: 2026-08-29 00:03:06 UTC
Summary
Seal Status
|
Contributor
|
📓 NotebookLM Notebook linked to this PR
This notebook contains session context, decisions, and artifacts for this work. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #2754
The corpus table says "217 specs produce Zig that Zig takes". None of its columns has ever said on how much of the spec — and 87 specs are accepted while part of their text is thrown away. The thrown-away part is the bodies of invariants, which is to say the assertions.
--per-specgains adroppedcolumn, so the diff that names a moved row also shows what that row throws away.The competitive read: docs/PARSER_DISCARD_LANDSCAPE.md
Three designs that do not have this failure mode, and what each costs:
ERRORnode, addressable by query; zero-widthMISSINGnodes where insertion is the cheaper repairtree-sitter parsestill does not report an error when an ERROR node is present (#4049)The middle row is the one worth reading twice. Tree-sitter has the representation and still does not report it — the same defect this repository keeps finding from the other direction, in a project with far more users.
t27 sits closer to tree-sitter than to rowan: recovery happens and the discarded region is not in the AST. It does one thing more than tree-sitter's CLI — it counts the discard (
parse-complete) and ratchets it (parse-no-discardis a suite phase). The note is explicit that it is not a proposal to rewrite the parser as a lossless CST, and names the cheap half of rowan's property that was available and unused. This PR writes the third item on that list, so the document does not describe an intention.Skill sections 150–153
.v = {synccommand must recompute the truth, not pick the newer file — the rewrite is what found the 31 dangling pairs🤖 Generated with Claude Code