Add comprehensive TS vs Go functional differences analysis#48
Merged
Add comprehensive TS vs Go functional differences analysis#48
Conversation
Documents 30 functional differences between the authoritative TypeScript implementation and the Go port, categorized by severity. Critical issues include non-functional error handling in Go, forced deep-merge on duplicate keys, and silent swallowing of invalid string escapes. https://claude.ai/code/session_01EnvP7joAa9P9S8YAfyDYTq
Adds 10 new findings from the parser comparison: empty/whitespace input short-circuiting, missing IGNORE token filtering, bad token not intercepted mid-parse, conditional token consumption, rule subscriber timing, unknown rule silent failure, missing result.fail/parse.prepare, no parent context merging, and trailing content detection gaps. Total: 40 differences. https://claude.ai/code/session_01EnvP7joAa9P9S8YAfyDYTq
Align Go LexMatcher signature with the TypeScript version by passing the current parsing rule to custom matchers: type LexMatcher func(lex *Lex, rule *Rule) *Token This threads the rule from ParseAlts through lex.Next() and nextRaw() to custom matchers, enabling context-sensitive lexing (e.g. checking rule.K, rule.U, rule.N, or rule.State). Previously matchers only received *Lex, requiring fragile shared closure state for context-dependent token production. Addresses difference #35 from TS_GO_DIFFERENCES.md. https://claude.ai/code/session_01EnvP7joAa9P9S8YAfyDYTq
Major changes across all core files to resolve functional differences: **Error handling (rule.go, parser.go)** - alt.E errors now propagate via ctx.ParseErr instead of being discarded - Parser loop checks ParseErr and converts to JsonicError **Grammar (grammar.go)** - pairval respects MapExtend flag (overwrites when false) and MapMerge function - safe.key blocks __proto__/constructor keys - list.property guard on elem.Open pair-in-list alternate - All alternates tagged with "json" or "jsonic" group tags for Exclude() **Lexer (lexer.go)** - Removed NaN/Infinity from default value keywords (now text, matching TS) - Invalid \x/\u/unknown escapes produce error tokens instead of silent corruption - Added string.abandon (fallthrough on error) and string.replace support - Added text.modify pipeline, number.exclude, line.single, comment eatline - Number matcher pushes trailing fixed tokens as lookahead (subMatchFixed) **Parser (parser.go)** - Removed whitespace-only input short-circuit - Rule subscribers fire BEFORE process (was after), matching TS - Explicit trailing content check after parse loop - Added parse.prepare hooks and result.fail sentinel checking **Options (options.go)** - Added MapMergeFunc, StringOptions.Abandon/Replace, TextOptions.Modify - Added LineOptions.Single, NumberOptions.Exclude, CommentDef.EatLine - Wired all new options through buildConfig **Rule (rule.go)** - Added AltModifier (H field) and dynamic PF/RF/BF function fields - Token consumption remains conditional (required for error detection) **Utility (utility.go)** - deep() preserves base when over is Undefined sentinel - ModList uses delete-move-filter order (was delete-filter-move) All tests pass. https://claude.ai/code/session_01EnvP7joAa9P9S8YAfyDYTq
Shared TSV test files (test/spec/alignment-*.tsv) validate identical behavior between TS and Go for: NaN/Infinity as text, safe.key on objects vs arrays, map merge with extend, number-text tokenization, auto-close structures, empty/comment input, and error cases. Go alignment_test.go adds direct tests for option-dependent features: map.extend=false, map.merge func, string.abandon/replace/escape errors, number.exclude, line.single, comment.eatline, text.modify, list.property guard, Exclude (alt.g tags), result.fail, parse.prepare, deep() with Undefined, ModList order, lex/rule subscribers, error propagation, trailing content detection, and finish rule control. TS alignment.test.js mirrors the same test coverage for the TS side. Also fixes safe.key to only block __proto__/constructor on arrays (matching TS where Object.create(null) objects have no prototype). https://claude.ai/code/session_01EnvP7joAa9P9S8YAfyDYTq
- rule() callback receives RuleSpec where the alt array is at rs.def.open,
not rs.open (which is a method)
- sub() takes {lex: fn} object, not a direct callback
https://claude.ai/code/session_01EnvP7joAa9P9S8YAfyDYTq
Check for absence of 'jsonic' tag rather than requiring exactly 'json', since TS grammar alts use comma-separated tags like 'json,map'. https://claude.ai/code/session_01EnvP7joAa9P9S8YAfyDYTq
Some TS grammar alts have undefined g values; guard with typeof check. https://claude.ai/code/session_01EnvP7joAa9P9S8YAfyDYTq
Use JSON roundtrip comparison to handle Jsonic's null-prototype objects. https://claude.ai/code/session_01EnvP7joAa9P9S8YAfyDYTq
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Documents 30 functional differences between the authoritative TypeScript
implementation and the Go port, categorized by severity. Critical issues
include non-functional error handling in Go, forced deep-merge on duplicate
keys, and silent swallowing of invalid string escapes.
https://claude.ai/code/session_01EnvP7joAa9P9S8YAfyDYTq