Phase 7a: serialization — documentation and phase record

.claude/skills/knowledge-lookup/SKILL.md

@@ -203,7 +203,7 @@ gives no hint. Never take the entry count as the prefix's ID count.
 | Pipeline composition and execution context | `--topic pipeline,execution-context,cancellation-and-timeouts --section rules --brief` and `--prefix CTX,RECOV,PIPE --section rules --brief` | `--prefix-info PIPE`, `--gaps CTX,RECOV,PIPE` | live |
 | Observability, configuration and redaction | `--topic observability,configuration,redaction-and-security --section rules --brief` and `--prefix CFG,OBS --section rules --brief` | `--prefix-info CFG`, `--prefix-info OBS`, `--gaps CFG,OBS` | live |
 | Resilience: retry, redirect and authentication | `--topic retry-and-resilience,redirect-handling,authentication,cancellation-and-timeouts --section rules --brief` and `--prefix RETRY,REDIR,AUTH --section rules --brief` | `--prefix-info RETRY`, `--gaps RETRY,REDIR,AUTH,RECOV` | live |
-| Serialization, SSE and pagination | `--topic serde,sse-streaming,pagination --section rules --brief` and `--prefix SERDE,SSE,PAGE --section rules --brief` | `--prefix-info SERDE`, `--gaps SERDE,SSE,PAGE` | live |
+| Serialization, SSE and pagination | `--topic serde,sse-streaming,pagination --section rules,constraints,conclusions --brief` and `--prefix SERDE,SSE,PAGE --section rules,constraints,conclusions --brief` (a `--section rules` reading alone misses `SERDE-17`, `SERDE-24`, `SERDE-25` and `SERDE-30`, filed under Constraints and Conclusions; phase 7a's finding) | `--prefix-info SERDE`, `--gaps SERDE,SSE,PAGE` | live |
 | Transport and async-runtime adapters | `--topic transport-adapter,cancellation-and-timeouts,concurrency-and-async --section rules --brief` and `--prefix TRANSPORT,ASYNC --section rules --brief` | `--prefix-info TRANSPORT`, `--gaps TRANSPORT,ASYNC` | live |
 | Styleguide-vs-design conflicts | `--section conflicts --brief` | resolve via a note under notes/ | live |
 

README.md

@@ -15,7 +15,7 @@ not compete with `faraday` or `httpx` on the easiest way to fetch a JSON endpoin
 
 ## Status
 
-**Phases 0, 1, 2, 3a, 3b, 4a, 4b, 4c, 5a, 5b, 5c, 6a, 6b, 6c, 7b and 7c are built.** Nothing is published. The repository holds six gems under
+**Phases 0, 1, 2, 3a, 3b, 4a, 4b, 4c, 5a, 5b, 5c, 6a, 6b, 6c, 7b, 7c and 7a are built.** Nothing is published. The repository holds six gems under
 `gems/`, every one at `0.0.0`. `dexpace-core` carries the HTTP domain model — the frozen,
 validated wire types every later phase stands on (`docs/sdk-documentation/http.md`) — the seam
 layer: the provider registry, the transport and codec seams, the core-owned async pivot,
@@ -75,17 +75,24 @@ pagination layer: the page value that owns one live response, the strategy contr
 built-in strategies over a caller-supplied extractor and never a codec, the byte-for-byte query
 splice, the eager-closing item view and the single-use, look-ahead page view over one private walk,
 the blocking engine, the non-blocking engine driven through `Future#on_settle` as a re-arm
-trampoline, the fetcher front-end, and `URL.resolve` (`docs/sdk-documentation/pagination.md`); the
-operation-lifecycle and transport groups of the HTTP-tracer vocabulary are emitted by nothing yet,
-and nothing talks to a socket yet. The other five are still skeletons — a namespace, a `VERSION`, a gemspec, a
-signature mirror and a smoke suite:
+trampoline, the fetcher front-end, and `URL.resolve` (`docs/sdk-documentation/pagination.md`) — and the
+serialization layer: the witness protocol with its decode context, three container combinators, a named
+boolean witness and an ISO-8601 witness, the three-state PATCH type with its decode combinator, the
+native-form encode walk that makes tri-state PATCH structural, `Body.serialized`, and the two response
+handlers phase 3b's `TypedResponse` was built to take (`docs/sdk-documentation/serde.md`).
+`dexpace-serde-json` holds the JSON codec over one private `JSON::Coder` per instance, the
+`json >= 2.19.9` floor in its gemspec and asserted at require time, and the seam registration under
+`:json` — the first gem here with a third-party dependency, and the first the three zero-dependency
+gates have run against with one; the operation-lifecycle and transport groups of the HTTP-tracer
+vocabulary are emitted by nothing yet, and nothing talks to a socket yet. The other four are still
+skeletons — a namespace, a `VERSION`, a gemspec, a signature mirror and a smoke suite:
 
 | Gem | Namespace | Runtime dependencies today |
 |---|---|---|
 | `dexpace-core` | `Dexpace` | none |
 | `dexpace-transport-net_http` | `Dexpace::Transport::NetHTTP` | `dexpace-core` |
 | `dexpace-transport-async_http` | `Dexpace::Transport::AsyncHTTP` | `dexpace-core` |
-| `dexpace-serde-json` | `Dexpace::Serde::JSON` | `dexpace-core` |
+| `dexpace-serde-json` | `Dexpace::Serde::JSON` | `dexpace-core`; `json >= 2.19.9` |
 | `dexpace-async-thread` | `Dexpace::Async::Thread` | `dexpace-core` |
 | `dexpace-conformance` | `Dexpace::Conformance` | `dexpace-core` |
 

Steepfile

@@ -39,7 +39,16 @@ target :serde_json do
   check "gems/dexpace-serde-json/lib"
   signature "gems/dexpace-serde-json/sig", "gems/dexpace-core/sig"
   library "json"
-  configure_code_diagnostics(D::Ruby.default)
+  # The one relaxation, on this target alone (phase 7a): rbs 4.2.0's stdlib json signatures
+  # declare JSONError, GeneratorError, ParserError, State, generate and parse -- and no
+  # `JSON::Coder`, the per-instance engine json 2.19.9 added and this adapter is built on -- while
+  # json 3.0.2 ships no sig/ of its own for `rbs collection` to pick up. So the codec's one
+  # `::JSON::Coder.new` is a Ruby::UnknownConstant that steep's default warning severity turns
+  # into a red gate. Downgraded to :information here, never a line-level ignore (no precedent in
+  # lib/) and never on core's strict target; the engine is typed `untyped` in the gem's sig
+  # (NFR-11 admits no `::JSON` type there either). Re-tighten to D::Ruby.default at the first rbs
+  # release that declares JSON::Coder.
+  configure_code_diagnostics(D::Ruby.default.merge({ D::Ruby::UnknownConstant => :information }))
 end
 
 target :async_thread do

docs/README.md

@@ -105,9 +105,10 @@ domain model, phase 2's seam layer, phase 3a's byte-streaming layer, phase 3b's
 4a's execution context, phase 4b's recovery layer, phase 4c's stage pipeline, phase 5a's
 configuration layer, phase 5b's logging facade and redaction, phase 5c's tracing and metrics
 layer, phase 6a's retry layer, phase 6c's authentication layer, phase 6b's redirect layer, phase
-7b's server-sent-events layer and phase 7c's pagination layer; the other five are
+7b's server-sent-events layer, phase 7c's pagination layer and phase 7a's serialization layer;
+`dexpace-serde-json` holds phase 7a's JSON codec and declares `json >= 2.19.9`; the other four are
 phase 0's skeletons, a namespace and a `VERSION`. The as-built documentation lands in
-[`sdk-documentation/`](./sdk-documentation/) as each gem gains code; the sixteen pages written so far are [`sdk-documentation/quality-gates.md`](./sdk-documentation/quality-gates.md), because the
+[`sdk-documentation/`](./sdk-documentation/) as each gem gains code; the seventeen pages written so far are [`sdk-documentation/quality-gates.md`](./sdk-documentation/quality-gates.md), because the
 gate set is the thing phase 0 built, [`sdk-documentation/http.md`](./sdk-documentation/http.md),
 because the domain model is the thing phase 1 built,
 [`sdk-documentation/seams.md`](./sdk-documentation/seams.md), because the seam layer is the thing
@@ -133,10 +134,12 @@ the thing phase 6c built,
 [`sdk-documentation/redirect.md`](./sdk-documentation/redirect.md), because the redirect layer and
 the two `standard` constructors are the things phase 6b built,
 [`sdk-documentation/sse.md`](./sdk-documentation/sse.md), because the server-sent-events layer and
-the serde-boundary gate are the things phase 7b built, and
+the serde-boundary gate are the things phase 7b built,
 [`sdk-documentation/pagination.md`](./sdk-documentation/pagination.md), because the pagination
 layer — the page value, the strategies, the two views, the two engines and the fetcher front-end —
-is the thing phase 7c built.
+is the thing phase 7c built, and
+[`sdk-documentation/serde.md`](./sdk-documentation/serde.md), because the serialization layer and the
+JSON codec are the things phase 7a built.
 
 ## Keeping this file true
 

docs/first-release.md

@@ -300,12 +300,17 @@ find it in a design document.
   from the other side. **Two things owed before release**, which is why this is an entry and not only a
   ledger row: the documented behaviour of a typed response handler on a body above
   `MAX_MATERIALIZED_BYTES` must be stated in `docs/sdk-documentation/`, so a caller streaming a large
-  JSON response meets a documented limit rather than an `::IOError`; and phase 8's adapters must each
-  be checked for whether their library offers a pull parser that would satisfy the clause — phase 8's
-  segmentation design already records that none of its three does. **Where it is carried.** `7a`'s
-  checklist marks `SERDE-27` with the clause named and cites this entry; the deviation row is
-  `7a P7-1`, consolidated into design §10 and audited by `docs/deviations.md`. Cites `SERDE-27`,
-  `SEAM-21`, `IO-9`, `BODY-32`.
+  JSON response meets a documented limit rather than an `::IOError` — **done 2026-09-20**, when phase 7a
+  was built: `docs/sdk-documentation/serde.md` states it under "`#load` materialises the whole text",
+  and the codec's suite asserts the `StreamError` propagates unwrapped
+  (`gems/dexpace-serde-json/test/dexpace/serde/json/codec_load_test.rb`, the `R1/P7-1` case); and phase
+  8's adapters must each be checked for whether their library offers a pull parser that would satisfy
+  the clause — phase 8's segmentation design already records that none of its three does, and this half
+  stays open until phase 8 lands. **Where it is carried.** `7a`'s checklist marks `SERDE-27` with the
+  clause named and cites this entry; the deviation row is `7a P7-1`, consolidated into design §10 and
+  audited by `docs/deviations.md`; as built, the codec drains through 3a's `#read_utf8` and the ceiling
+  it reads is the configured `Dexpace::IO.max_materialized_bytes`. Cites `SERDE-27`, `SEAM-21`, `IO-9`,
+  `BODY-32`.
 
 ### SHOULD- and MAY-level requirements declined for v1
 

docs/knowledge/notes/serde.md

@@ -0,0 +1,11 @@
+# serde — notes
+
+Hand-written. `../harvested/serde.md` is what the documents say; this file is what the
+implementation found, and it wins. Each entry names the harvested entry it answers by that entry's
+stable key.
+
+## Reference
+- **A decoded JSON `String` can be UTF-8-tagged and invalid, and neither layer that produces it raises.** Beside `serde/b5e5efc8`, which puts the strictness burden on the witness: a witness asserting `String` cannot catch this, because an invalid-UTF-8 `String` *is* a `String`. Measured on Ruby 3.2.11, 3.3.12, 3.4.10 and 4.0.6 against json 2.19.9 (the gemspec floor) and 3.0.2 (the bundle's): `JSON.parse(%Q({"a":"\xff"}).b)["a"]` returns a `String` whose `#encoding` is UTF-8, whose `#bytes` is `[255]` and whose `#valid_encoding?` is `false`. Phase 3a's `#read_utf8` and `#read_string(encoding)` retag and apply no replacement policy, by their own stated contract, so nothing between the wire and the witness validates. Phase 7a's `Dexpace::Serde::JSON::Codec#load` therefore calls `#valid_encoding?` on the drained text and raises `Dexpace::Serde::DeserializationError` before parsing (`7a P7-6`). A *transcode* is the wrong repair: `"\xc3\xa9".b.encode(::Encoding::UTF_8, ::Encoding::BINARY)` raises `Encoding::UndefinedConversionError` for a valid two-byte `é`, because BINARY has no character semantics to convert from — so the retag-then-transcode recipe of `docs/knowledge/notes/io-and-byte-streams.md` (its key io-and-byte-streams/6eb5155f, a note and not a harvested rule) is retag, then **validate**, and the transcode step applies only when a declared charset is not UTF-8.
+  <sub>review · `docs/work/mvp/phase7/phase7a/2026-09-10-phase7a-serialization-design.md` · high · sha:manual-phase7a-utf8-validation</sub>
+- **`JSON::Coder`'s constructor and its tolerance of an unknown option both changed between json 2.19.9 and 3.0, so an adapter that forwards caller options verbatim is configured differently on the two, silently on one and loudly on the other.** Adds to `serde/d3bef411` and `serde/b36d4403`, which describe the private-engine and fresh-factory contract the `Coder` makes satisfiable literally (`7a P7-4`), and to `serde/d15ade64`, the reason the floor is a gemspec line: measured on 2026-09-20 on 3.4.10 with json 2.19.9 and on 3.2.11 and 4.0.6 with json 3.0.2, `JSON::Coder.instance_method(:initialize).parameters` is `[[:opt, :options], [:block, :as_json]]` on 2.19.9 and `[[:key, :object_class], [:key, :array_class], [:key, :on_load], [:keyrest, :options], [:block, :as_json]]` on 3.0.2; `Coder.new(max_nestng: 4).dump([1])` returns `"[1]"` on 2.19.9 and raises `ArgumentError: unknown keyword: max_nestng` on 3.0.2; `Coder.new(encoders: {})` is accepted on 2.19.9 and refused on 3.0.2; `Coder.new(nil)` works on 2.19.9 and raises on 3.0.2; a duplicate key is accepted last-wins on 2.9.1, a warning on 2.19.9 and a `ParserError` on 3.0.2, while `Coder.new(allow_duplicate_key: false)` raises `ParserError` on both floored versions; and `Coder.new.dump(Object.new)` and `.dump(Time.at(0))` raise `GeneratorError` on both **without** `strict: true` — the `Coder` is strict on its own account, and `strict: true` is documentation of intent rather than a switch. Phase 7a's codec therefore validates options against its own allowlist, constructs the `Coder` with keywords only, fixes `allow_duplicate_key: false` unless the caller opts in, and never forwards `encoders:`. Design open question 4's premise ("`Coder.new` accepts unknown options silently") is true at the floor and false at 3.0.
+  <sub>review · `docs/work/mvp/phase7/phase7a/2026-09-10-phase7a-serialization-checklist.md` · high · sha:manual-phase7a-coder-option-drift</sub>

docs/sdk-documentation/architecture.md

@@ -8,18 +8,19 @@ recovery layer and the error trail, phase 4c the stage pipeline, phase 5a the co
 layer and the clock, phase 5c the tracing and metrics layer, phase 5b the logging facade and
 redaction, phase 6a the retry layer — one policy core and its two stacks — phase 6c the
 authentication layer, phase 6b the redirect layer with the two `standard` pipeline
-constructors, phase 7b the server-sent-events layer with the serde-boundary gate and phase 7c the
-pagination layer; the adapters are still to come, so this page is still a stub: it names the
+constructors, phase 7b the server-sent-events layer with the serde-boundary gate, phase 7c the
+pagination layer and phase 7a the serialization layer with the JSON codec; the adapters are still to
+come, so this page is still a stub: it names the
 pages this tree will eventually hold and where each one's content will come from, so the plan for
-the documentation exists before the documentation does. Sixteen pages are real already, because
+the documentation exists before the documentation does. Seventeen pages are real already, because
 their subjects are: [`quality-gates.md`](./quality-gates.md), [`http.md`](./http.md),
 [`seams.md`](./seams.md), [`io.md`](./io.md), [`body.md`](./body.md),
 [`execution-context.md`](./execution-context.md), [`recovery.md`](./recovery.md),
 [`pipelines.md`](./pipelines.md), [`configuration.md`](./configuration.md),
 [`tracing-and-metrics.md`](./tracing-and-metrics.md),
 [`logging-and-redaction.md`](./logging-and-redaction.md), [`retry.md`](./retry.md),
-[`auth.md`](./auth.md), [`redirect.md`](./redirect.md), [`sse.md`](./sse.md) and
-[`pagination.md`](./pagination.md).
+[`auth.md`](./auth.md), [`redirect.md`](./redirect.md), [`sse.md`](./sse.md),
+[`pagination.md`](./pagination.md) and [`serde.md`](./serde.md).
 Once `dexpace-core` and the first adapters ship, this page becomes the same kind of front door the
 sibling Node SDK's `docs/sdk-documentation/architecture.md` is — package by package, seam by seam
 — and the entries below turn from plain text into real links, one at a time, as each page is
@@ -165,6 +166,18 @@ layer phase 7b shipped; derives from `docs/sdk-design-ruby/07-pagination-sse-and
 §7.2 and §7.1, read together with entries 6 and 18 of
 `docs/sdk-design-ruby/10-deliberate-deviations-from-the-reference-contract.md`.
 
+[serde.md](./serde.md) — the serialization layer and the JSON codec: the witness protocol — the
+predicate pair, the decode context with its RFC 6901 pointer and one raise site, the three container
+combinators, the named boolean witness and the ISO-8601 witness with its stated precision domain —
+the three-state PATCH type and its decode combinator, the native-form encode walk and the `OMIT`
+sentinel that make tri-state PATCH structural, `Body.serialized`, the two response handlers supplied
+into `TypedResponse`, and `dexpace-serde-json`'s codec — its private engine per instance, its option
+allowlist, its failure model, the four encode profiles, the stream rule, and the materialisation
+ceiling a large body meets. Written against the layer and the gem phase 7a shipped; derives from
+`docs/sdk-design-ruby/03-seam-by-seam-idiomatic-mapping.md` §3.4 and
+`docs/sdk-design-ruby/07-pagination-sse-and-serialization.md` §7.3, read together with entries 12, 13
+and 14 of `docs/sdk-design-ruby/10-deliberate-deviations-from-the-reference-contract.md`.
+
 [quality-gates.md](./quality-gates.md) — every blocking gate this SDK runs, what each protects,
 and how to run it locally. Written against the build phase 0 shipped; derives from
 `docs/sdk-design-ruby/09-toolchain-and-quality-gates.md`.
@@ -175,7 +188,8 @@ the seam's contract itself is already on [seams.md](./seams.md).
 
 write-a-serde.md — implementing the `Serde` seam: the serializer/deserializer pair, the
 `Tristate` PATCH convention, and the four encode profiles. Derives from
-`docs/sdk-design-ruby/07-pagination-sse-and-serialization.md`.
+`docs/sdk-design-ruby/07-pagination-sse-and-serialization.md`; the as-built reference implementation
+it will walk through is on [serde.md](./serde.md).
 
 [pagination.md](./pagination.md) covers what write-a-paging-strategy.md was planned to hold —
 the strategy contract (`#parse(response, template) -> Info`), the three built-ins over a