Schema compiler - #7908
Draft
gcanti wants to merge 55 commits into
Draft
Conversation
🦋 Changeset detectedLatest commit: ae1a082 The changes in this PR will be included in the next version bump. This PR includes changesets to release 30 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Contributor
Bundle Size AnalysisGenerated from PR build output; treat the content below as untrusted.
|
gcanti
force-pushed
the
schema-compiler
branch
2 times, most recently
from
September 7, 2026 06:51
035d2b3 to
3ab7d8a
Compare
Remove schema annotation overrides and parsing concurrency, share interpreter and JIT helpers, and reuse option-independent generated functions. Update regression coverage, migration annotations, and runtime and bundle snapshots.
gcanti
force-pushed
the
schema-compiler
branch
from
September 8, 2026 07:43
2b481c5 to
ae1a082
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Runtime compilation
Schema offers experimental, opt-in JIT and AOT compilation. Both work through
the normal
SchemaParserAPIs for decoding, encoding, type guards, and construction; schemasremain composable and do not acquire a separate compiled type.
To enable JIT globally, import its side-effect entrypoint during startup:
Compilation is lazy: the import enables it, but parsers initialize their
operations only when first used. If the environment forbids
new FunctionorJIT compilation fails, Schema falls back to interpreted parsing without retrying
the failed compilation. This also applies to lazy checkpoints, without repeating
earlier transformations or middleware. Exceptions during parsing keep their
normal behavior; they do not trigger a restart in the interpreter.
One cache, interchangeable parsers
SchemaCompileruses oneWeakMap<AST, Entry>for interpreted, JIT, AOT, andmanually installed decoders. Installed decoders are
CompiledDecoderobjects,not just decoding functions. Each entry provides lazy
decodeEffectandmakeEffectoperations, either installed or interpreted. Decoders can also supply
validateand
is, as described below. The cache never stores parsing results.The registry adds a
parseEffectfunction to coordinatevalidateanddecodeEffect,and lazily prepares interpreted construction when the installed bundle omits
makeEffect.Both constructor and decoder functions are cached on that same entry. It also adds
an
originflag, either"interpreted"or"installed". This flag controlsreplacement during installation, not validation: selective JIT preserves already
installed descendants but can replace interpreted ones. Callers of
setsupplyonly the decoder operations, not these internal fields.
On first use, a parser reuses the cached entry or creates and caches a compiled
or interpreted decoder. Children use the same cache, so an interpreted parent
can have compiled children. Internal decoding resolves the entry's raw parser;
the public
parseEffectboundary materializes its successful output. Constructionresolves
makeEffectinstead. Selective compilation remains active for childreneven when their parent uses an interpreted constructor. All operations are lazy,
so recursive children resolve after their parent entry has been installed. No
separate constructor cache or recursive placeholder cache is needed.
On first use of a Declaration, its declared type parameters are registered through
the entry lookup carried by the same resolver before its callback runs. Their
operations remain lazy. This lets callbacks using public parsers find selectively
compiled children; new ASTs created inside a callback follow the normal registry
policy.
Choose how to populate it:
SchemaJITCompiler/enableSchemaJITCompiler.enable(ast)SchemaCompiler.set(ast, decoder)These modules live under
effect/unstable/schema. Selective installation takesan AST: use
schema.astfor decoding,SchemaAST.flip(schema.ast)for encoding,and
SchemaAST.toType(schema.ast)for type guards and construction. The exact returned AST isthe cache key. Different AST objects have separate entries, even if structurally
equal; operations using the same AST object share an entry.
Install before the first execution of parsers you want to accelerate.
Creating a parser earlier is fine. Late installation is safe, but a parser that
already captured an entry keeps it, even when later calls change parse options.
This includes
make,makeOption, andmakeEffect: constructing a value canpopulate the entry before its first decode. A later global JIT import does not
upgrade it. Explicit
setstill replaces the whole entry for new consumers;omitting
makeEffectfrom a replacement restores interpreted construction for them.Parsing behavior and constraints
Every entry provides the complete
decodeEffectoperation. Two optional fast pathsavoid work that is unnecessary for successful decoding or boolean validation.
All operations initialize independently when first needed:
is, optionalbooleanvalidate, optionalSchemaCompiler.invaliddecodeEffect, requiredEffect<value, SchemaIssue, R>makeEffect, optionalEffect<value, SchemaIssue, R>decodeEffectis required so every entry can produce output and explain failures,even without any fast paths. It also handles transformations, middleware, and
asynchronous work when the AST requires them. It can be compiled or interpreted;
calling
decodeEffectdoes not necessarily mean returning to the interpreter.validateis optional because a synchronous, diagnostic-free first pass is notalways supported or safe to repeat. In particular, ASTs containing encodings
omit it so a later failure cannot repeat transformations or middleware.
isis optional because preserving validation semantics can require constructingoutput. For example, a check on a Struct must see the reconstructed object with
extra properties removed. Such a schema uses
validate, ordecodeEffectifvalidateis unavailable, instead of an output-freeis. Omitting either fastpath removes an optimization, not parsing capability.
For decoding,
entry.parseEffecttriesvalidatewhen available. Success alreadycontains the output, so no detailed pass is needed.
invalidcontains no errorlocation or explanation, so failure requires one detailed
decodeEffectpass. Thisfavors valid inputs at the cost of traversing invalid inputs again, only where
repetition is safe. Without
validate, or for themissingsentinel, it callsdecodeEffectdirectly. Interpreter, JIT, and AOT implementations supply the operationswithout implementing this dispatch. The synchronous decode and encode adapters
share a direct version of it, returning successful
validateoutput without anintermediate Effect. Encoding uses the flipped AST; when it equals the original,
both adapters use the same entry and execution path. Detailed traversal does not
restart fast validation at every child.
Construction calls
entry.makeEffectdirectly, withoutisorvalidate.It has different semantics from decoding: defaults are applied to Struct fields,
tuple/array elements, and Record values before the child is constructed. They do
not become defaults for that AST used as a root, a Union member, or a Record key.
Union selection stays conservative for missing discriminants. Class construction
preserves recognized instances; otherwise it constructs the source and creates
the instance. Ordinary Declaration callbacks retain their own parsing choices.
Using only construction does not initialize the decoder or validators. JIT
preparation failures select the interpreted implementation for the affected
operation, independently of decoding or construction that already works. If a
lazy child's compilation fails after a default has run, only that child falls
back. Neither default effects nor Class constructors are replayed. Normal Union
branch attempts remain unchanged, so more than one branch's defaults can run.
A composed Struct decoder can compile its fields, including transformations,
then apply the Struct's checks to the decoded output. Both stages read the same
AST, and only the complete decoder is installed. Checks run after stripping and
successful field decoding, without replaying transformations.
SchemaParser.is(schema)andSchema.is(schema)checktoType(schema.ast)with default parse options: excess properties are ignored and checks are enabled.
The guard uses
is, thenvalidateifisis unavailable, convertinginvalidto
falsewithout a diagnostic pass. If neither exists, it uses ordinarydecoding and converts the outcome to a boolean; non-schema failures still throw.
For configurable type-side validation, use a decoding API with
Schema.toType(schema).The following constraints apply:
ParseOptionsapply throughout parsing, without recompilation.Annotations cannot override them. Children parse sequentially; use Effect
concurrency combinators for independent operations or inside transformations.
have no observable side effects; property getters must be deterministic and
safe to repeat. Declaration parsers have the same constraint and must be
synchronous.
decodeEffectdirectly. Transformations and middlewarerun once, with their validation checkpoints resolved through the shared cache.
Local checkpoints preserve the original AST for checks and issues.
supported descendants can still compile. Parsing exceptions follow normal
Effect defect behavior.
For custom
setimplementations, every operation must honor the activeParseOptions. Returninvalidonly for invalid input, never to decline anoptimization;
validatemust not calldecodeEffectand discard its issues.User checks may themselves allocate issues. An absent optional input reaches
decodeEffectormakeEffectasSchemaCompiler.missing, distinct from a presentundefined. When a field produces no value, propagatemissingas an Effectsuccess: the parent omits optional fields or reports a missing required key.
Public root adapters reject a final
missingrather than exposing it.Installation trusts the decoder to implement its AST and does not mutate the
supplied object. Operation accessors are evaluated once, on demand, with that
object as their receiver; missing optional operations are cached too. Installed
decodeEffectandmakeEffectfunctions return ordinary Effects.Ahead-of-time compilation
SchemaAOTCompiler.compile(asts)accepts a readonly array of ASTs and returnsa JavaScript ES module exporting
install(asts). Share the root array betweenthe build script and the application:
Generate the module at build time:
Install it before first parser execution:
Neither generation nor importing the generated module installs decoders.
installregisters the roots and supported statically reachable dependencies.Use
[ast]for one schema; an empty array is a no-op. Repeated roots and shareddependencies are emitted and installed once by identity.
JIT and AOT share generation rules and runtime support. Generated modules import
effect/unstable/schema/SchemaCompiler/runtime, not the code generator, andwork without dynamic function construction. Validators and composed Struct
decoders and constructors are static. Detailed diagnostics, transformation
orchestration, and Array, Record, Union, leaf, and Class constructors specialize
lazily in shared runtime support. AOT does not precompute every operation.
Constructor default links and Class source schemas are included among reachable
dependencies and read from the runtime ASTs without executing defaults during generation.
Keep these installation requirements in mind:
array length, root order, AST structure, and sharing to match the build-time
roots; it does not check compatibility.
not serialization, preserving their identity.
unsupported nodes use the interpreter, with no late JIT required.
when they differ from the supplied roots. As with
set,late installation does not update parser closures that captured older entries.
Performance snapshot
Construction
Measured on 2026-09-08, Node 24.12.0, V8 13.6, Apple M3, macOS arm64.
Median ns/op through
SchemaParser.make; nine isolated rounds, 500 ms measurementand 150 ms warmup. Interpreted/JIT use calibrated batches; AOT uses batch 256
with dynamic code generation disabled and a separate fixture call site.
Schema setup is outside measurement except in the last row, which also includes
installation for AOT.
Retained parser heap, KiB/schema, for
Struct({ a: String, b: Number.withConstructorDefault(succeed(1)) }).Five isolated processes per cell retain 1,000 distinct schemas and their public
parsers after first use with
{ a: "a", b: 1 }. Imports, static AOT modules,64 warmup schemas, and schema construction precede the parser heap reading.
Two forced GCs run at each reading. Costs include public parser closures and AOT
installation, but exclude schema construction, about 3 KiB/schema.
These are retained V8 heap values, not peak or all native executable-code memory.
Decoding and type guards
Measured on 2026-09-08 on the same source revision as the construction snapshot.
All measurements use public
SchemaParserAPIs on Node 24.12.0, V8 13.6,Apple M3, macOS arm64. The current snapshot includes every scenario in the
schema-compilersuite, not just the Moltar objects.Moltar
Median ns/op, batch 256. Interpreted/JIT: five processes per case, 300 ms
measurement and 100 ms warmup. AOT: nine processes, 500 ms measurement and
150 ms warmup, with dynamic code generation disabled. AOT uses the same data and
worker with a different fixture call site. Sub-10 ns differences between JIT and
AOT are not a general ranking.
Schema creation plus first use, median µs/op, five processes per case. AOT was
not measured in this fixture. Creation and first use are measured together.
Other schema shapes
All 31 scenarios in
schema-compiler, median ns/op from five processes per case,300 ms measurement, 100 ms warmup, and shared calibrated batches.
declaration-set-validchecked-transformed-struct-validchecked-transformed-struct-invalidsync-decode-validsync-encode-validstrict-record-1024-validstrict-record-4096-validstrict-record-4096-invalidarray-100-validarray-100-invalid-lasttuple-rest-validoptional-struct-validrecord-validtemplate-record-validstruct-with-record-validnumber-record-validtransformed-key-record-validencoding-checked-struct-validliteral-100-valid-lastliteral-100-invalidtagged-union-100-valid-lasttagged-union-100-invalidchecked-string-validtemplate-literal-validtransformation-struct-validtransformation-root-validtransformation-root-invalidtransformation-uppercase-validtransformation-output-invalidmiddleware-struct-validrecursive-node-validMemory and first-use CPU
Seven isolated processes per mode, each retaining 1,000 distinct schemas for
Struct({ name: String, count: Number, active: Boolean, tags: Array(String) }),their inputs, and public
decodeUnknownSyncfunctions. The inputs contain{ name: "a", count: 1, active: true, tags: ["a", "b"] }.Two forced GCs run before construction, after construction, and after first use.
Retained heap, B/schema. Construction includes schemas, inputs and public parser
closures. First use includes AOT installation and lazy parser initialization.
Imports and the static AOT module precede the first reading and are excluded.
These are retained V8 heap values, not peak memory, RSS, or all native code memory.
First use, median µs/schema within the 1,000-schema batch, including AOT
installation. Schema construction, imports and GC are outside the timed region.
Process CPU includes background work and can exceed wall time. Warm CPU is not
measured separately.
Coverage of the accompanying revision comparison: all 191 Effect cases in the
runtimeperf registry, including the 54 interpreter diagnostics and 31 Arbitrary
cases not tabulated here. AOT throughput coverage includes the six Moltar paths and six construction cases.
Peak memory and native executable-code memory are not covered. Current measurements are shown here without deltas;
revision-comparison reports retain their paired observations and confidence intervals.