Skip to content

Accepted is not the same as accepted on the whole spec - #2770

Merged
gHashTag merged 1 commit into
masterfrom
w699-discard-landscape
Aug 29, 2026
Merged

Accepted is not the same as accepted on the whole spec#2770
gHashTag merged 1 commit into
masterfrom
w699-discard-landscape

Conversation

@gHashTag

Copy link
Copy Markdown
Owner

Refs #2754

The corpus table says "217 specs produce Zig that Zig takes". None of its columns has ever said on how much of the spec — and 87 specs are accepted while part of their text is thrown away. The thrown-away part is the bodies of invariants, which is to say the assertions.

  ALL FOUR accept               63    9.7%

  specs with tokens DROPPED     87   13.4%
    ... tokens dropped       32485
  Accepted is not the same as accepted ON THE WHOLE SPEC.

--per-spec gains a dropped column, so the diff that names a moved row also shows what that row throws away.

The competitive read: docs/PARSER_DISCARD_LANDSCAPE.md

Three designs that do not have this failure mode, and what each costs:

how the dropped text is handled cost
rowan — rust-analyzer, Swift, C#, Kotlin discarding is unrepresentable: the source must be perfectly reconstructible from the tree, errors and all two tree layers; every consumer must tolerate error nodes rather than assume a well-formed AST
tree-sitter kept as an ERROR node, addressable by query; zero-width MISSING nodes where insertion is the cheaper repair recovery costs are opaque by its own documentation — and tree-sitter parse still does not report an error when an ERROR node is present (#4049)
bison error productions the grammar names its own recovery points designed in per-construct; silent about how much was skipped at runtime

The middle row is the one worth reading twice. Tree-sitter has the representation and still does not report it — the same defect this repository keeps finding from the other direction, in a project with far more users.

t27 sits closer to tree-sitter than to rowan: recovery happens and the discarded region is not in the AST. It does one thing more than tree-sitter's CLI — it counts the discard (parse-complete) and ratchets it (parse-no-discard is a suite phase). The note is explicit that it is not a proposal to rewrite the parser as a lossless CST, and names the cheap half of rowan's property that was available and unused. This PR writes the third item on that list, so the document does not describe an intention.

Skill sections 150–153

  • when a column moves, add the command that names the rows — one flag beat a hand-rolled harness that had already lied once
  • two node shapes reaching one emitter arm, and only one of them was read
  • prove a wrapper is dead before removing it: 0 of 156 C-accepted specs contained .v = {
  • a sync command must recompute the truth, not pick the newer file — the rewrite is what found the 31 dangling pairs

🤖 Generated with Claude Code

The columns say "217 specs produce Zig that Zig takes". None of them has ever
said ON HOW MUCH OF THE SPEC, and 87 specs are accepted while part of their text
is thrown away -- the part being the bodies of invariants, which is to say the
assertions.

    specs with tokens DROPPED     87   13.4%
      ... tokens dropped       32485
    Accepted is not the same as accepted ON THE WHOLE SPEC.

`--per-spec` gains a `dropped` column, so the diff that names a moved row also
shows what that row throws away.

docs/PARSER_DISCARD_LANDSCAPE.md is the competitive read behind the design:

  * rowan (rust-analyzer, Swift, C#, Kotlin) makes discarding UNREPRESENTABLE --
    the source must be reconstructible from the tree, so a parse-complete
    command cannot be needed. Cost: two tree layers and every consumer tolerating
    error nodes.
  * tree-sitter represents the dropped region as an ERROR node, and its own
    `tree-sitter parse` still does not report an error when one is present
    (tree-sitter#4049). Having the representation is not the same as reporting
    it -- this repository's own lesson, arrived at from the other side.
  * bison error productions put the skip in the grammar: auditable by reading,
    silent about how much was skipped at runtime.

t27 sits closer to tree-sitter, and does one thing more than tree-sitter's CLI:
it counts the discard and ratchets it. The note names the cheap half of rowan's
property that was available and unused -- and this commit writes it.

Skill sections 150-153: when a column moves add the per-item dump; two node
shapes reaching one emitter arm; prove a wrapper is dead before removing it;
a sync command must recompute the truth rather than pick the newer file.

Refs #2754

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-08-29 00:03:06 UTC

Summary

Status Count
Total Open PRs 8
PRs with Failing Checks 7
PRs with All Checks Green 1
READY 0
FAILING 7
PENDING 0

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=3abc96dcebd6 != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

@github-actions

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

@gHashTag
gHashTag merged commit b277390 into master Aug 29, 2026
33 checks passed
@gHashTag
gHashTag deleted the w699-discard-landscape branch August 29, 2026 00:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant