Skip to content

fix(schema): release unused parse context ids from the global registry - #1673

Merged
ssalbdivad merged 1 commit into
arktypeio:mainfrom
isaacwasserman:budapest/arktype-1584-fix
Oct 1, 2026
Merged

ssalbdivad merged 1 commit into
arktypeio:mainfrom
isaacwasserman:budapest/arktype-1584-fix

Conversation

@isaacwasserman

Copy link
Copy Markdown
Contributor

Fixes the leak in #1584 when an equivalent type is created again and again.

Before this change:
Each parse gets a new id in nodesByRegisteredId. When the result node comes from a cache, it has a different id and nothing can refer to the new one, but the id stayed in the global registry. Repeatedly creating an equivalent type (e.g. type("string") or T.pick("a")) grew memory without limit.

After this change:
These unused entries are pruned.

This change ensures that subsequent creation of identical schemas will not bloat the cache. However, even with this change, the cache will still grow monotonically as many distinct schemas are created. I will try to address this in a later PR.

Each parse gets a new id in nodesByRegisteredId. When the result node comes
from a cache, it has a different id and nothing can refer to the new one,
but the id stayed in the global registry. Repeatedly creating an equivalent
type (e.g. type("string") or T.pick("a")) grew memory without limit.

Remove the id when the result node does not use it.

Fixes arktypeio#1584 for equivalent types. Distinct types (cyclic types, inline
morphs/predicates) are still retained through other global caches.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

  • Selective retention of parse context ids (ark/schema/scope.ts) — after a parse completes, the freshly allocated context id is kept only when the result node actually adopted it; otherwise it is deleted from the global nodesByRegisteredId. Previously both node() and parseDefinition() stored the bound node under the new context id unconditionally, so every cache-hit parse of an equivalent schema left an orphan id behind. The helper's invariant (an id can only be referenced by an alias child that makes the enclosing node cyclic, and cyclic nodes are forced onto ctx.id via withId) holds for every path I exercised.
  • Preassigned ids preserved — when the input context already carries an id (reachable for sequence nodes), the old unconditional assignment is retained. I confirmed via instrumentation that this branch is live, not dead code.
  • Regression test (ark/type/__tests__/registry.test.ts) — warms caches, then asserts the registry size is stable across 100 repeated instantiations for keyword/expression/object/validation/pick/and/or/array, plus a check that cyclic this types still resolve. I verified it genuinely fails against main (pick grew 3597 → 3197, etc.) and passes here.

Verification performed: full pnpm test passes (1775), prettier --check passes on both files, and repeated-creation probes across Record<string, this>, this[], index-signature, tuple, union, nested, and optional this shapes all resolve.

ℹ️ Equivalent recursive and generic schemas still grow the registry

The pruning fixes the flat cases completely — 50 repeats of type("string") now add 0 ids (50 before). But identical recursive schemas are still not fully reclaimed: 50 repeats of type({ box: "this | undefined" }) add ~300 ids here (vs ~400 on main), and repeated generic instantiation also still grows. This is not a regression — the change reduces growth in every case I measured — and the PR description already scopes out continued growth, but the caveat is about distinct schemas; worth noting that equivalent recursive schemas remain affected, so a follow-up will still be needed for that class.

Technical details
# Recursive/generic parse ids still leak

## Affected sites
- `ark/schema/scope.ts:708` — `withId(node, ctx.id)` allocates a fresh id for every re-parse of a cached cyclic node (the cached node's alias children reference the prior id), so each creation registers a new id.
- `ark/type/__tests__/registry.test.ts` — the no-growth assertion is only applied to non-recursive cases; the cyclic `it` only checks resolution, not registry stability.

## Measurement
- head: `type({ box: "this | undefined" })` ×50 → +300 entries; base → +400.
- head: generic instantiation ×50 → +300; base → +650.
- head/base: `type("string")` ×50 → +0 / +50.

## Required outcome (if addressed in a follow-up)
- Reclaim ids for cache-hit cyclic/generic parses too, or document the remaining bound explicitly.

Pullfrog  | View workflow run | Using deepseek-v4.1-flash (free via Pullfrog for OSS) | 𝕏

@ssalbdivad ssalbdivad left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for tracking this down, super clean fix! approving so 2.x gets it now.

fyi 3.0 already handles this more broadly (contexts only stay registered if an alias actually references them), so no need for the distinct-schemas follow-up. will carry your tests forward though.

lots more coming in 3.0 too, especially way faster transforms/defaults 👀

@ssalbdivad
ssalbdivad merged commit 254328c into arktypeio:main Oct 1, 2026
7 checks passed
isaacwasserman added a commit to isaacwasserman/arktype that referenced this pull request Oct 1, 2026
A WeakRef keeps its target alive until the current job ends. Each parse put
its new context into a WeakRef, so in a synchronous loop each context stayed
in memory until the loop ended (about 190 bytes per type("string") call).
This undid part of the fix in arktypeio#1673 for synchronous code.

- add WeakValueMap.setStrong for short-lived values
- createParseContext holds the context strongly. The parse replaces it with
  its node or releases it, as before
- release the context if its parse throws, and release generic parameter
  contexts after their parse (both stayed in the registry before)
- scope alias contexts stay weak in the registry, because the scope holds them

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@ssalbdivad ssalbdivad mentioned this pull request Oct 1, 2026
4 tasks done
ssalbdivad added a commit that referenced this pull request Oct 1, 2026
Bumps all publishable packages and documents the changes merged since 2.2.6:

- fix `this[]` in self-referential object types (#1618, fixes #1406,
  @yharaskrik)
- parse string.date.epoch.parse input as milliseconds (#1669, @breken-ai)
- keep uppercase UUIDs valid in string.uuid JSON Schema (#1670,
  @breken-ai)
- render nested bigints correctly in error messages (#1626, fixes #1477,
  @chatman-media)
- release unused parse ids from the global registry (#1673, fixes #1584
  for equivalent types, @isaacwasserman)
- fix crashes on Hermes and Turbopack (#1674, fixes #1645 and #1643)

#1590 only adds tests, so it has no changelog entry.

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done (merged or closed)

Development

Successfully merging this pull request may close these issues.

2 participants