Repository navigation
Phase 8c: asynchronous transport — documentation and phase record - #98
Merged
Merged
Conversation
Wahbeh-Mohammad
added
type:feature
Contributor
Author
Review record for the phase 8c stack (#96 → #97 → #98)4 independent reviews, each by a fresh agent with no memory of the previous one, each re-running every gate itself on every tip (all eighteen individually at the code tip on 4.0.6; the full
Mutations are counted per row — an a/b split is two; the reviewer's prose counts named mutations where the two differ (round 0: the brief's 30 plus six extras; round 1: 42 named; round 2: 50 named). Nothing was skipped. Round 0 → fixed in round 1
Round 1 → fixed in round 2
Round 2 → fixed in round 3
Round 3 (final) — approve, three nits folded into the reconcile pass's chore commit
|
dexpace-transport-async_http alone needs Ruby >= 3.3, because async-http and its whole closure declare it (phase 8c's P8-36). The floor is one row in VERSIONS, `ruby floor:dexpace-transport-async_http 3.3`, and DexpaceVersions.ruby_floor(gem) with a KeyError fallback to the global floor is what the gemspecs, gates:versions and gates:gemspec_audit read. DexpaceVersions.gem_supported?(gem, ruby) is what the Gemfile, test:gems and gates:clean_bundle consult, so a row below a gem's floor skips that gem with the count printed instead of failing on Bundler's refusal. Two gate fixtures, per_gem_floor_ahead under versions/ and gemspec_audit/, are the deliberately failing inputs: a gemspec whose floor is below its VERSIONS row.
Phase 8c's reference asynchronous transport over async-http ~> 0.104: .build over a client map keyed by (reactor, origin) and bounded at MAX_ORIGINS, .using over a caller's own client with retries already zero, the Adapter behind both with its Closeable latch, one exchange task per call under the caller's own task and one with_timeout budget, the queue-marshalled cancellation bridge whose transient watcher acts on the reactor's thread (a Task#cancel from a foreign OS thread raises and cancels nothing), the RFC 7230 token predicate applied before dispatch on both protocols with DropPolicy's once-per-name, bounded reporting, the ten framing headers never copied, the wire-boundary re-validation's second call site, the lazy pull-shaped ResponseBody whose native close goes through close_quietly, the classifier that asks the token first, the lenient inbound mapper, the default TLS context offering h2 by ALPN, and the require-time registration under :async_http. Core gains Configuration::Keys::TRANSPORT_CONNECTION_LIMIT; the two key pins move, and the async seam's four in-process bare-require pins become a child-process suite, as 8a's registration did to the sync seam's. The :async_http Steep target relaxes UnknownConstant to :information as :serde_json's does, since nothing in the closure ships a sig/. The surface manifests are regenerated once: core gains the one key, this gem its twenty-two public rows. test/support/async_http_warmup.rb spends Ruby 4.0's once-per-process IO::Buffer warning at test-helper load, before the fatal-warning hook can see it.
Phase 8c's portable rows join dexpace-conformance as two private groups: Asynchronous (TRANSPORT-7, 9, 21, 23) and HeaderDrops (TRANSPORT-12, 13), each written against the suite contract's primitives alone, so the suite is thirty-four assertions in seven groups; TRANSPORT-8 stays an adapter's own row, which PREAMBLE now says beside 8a's two omissions. Every head a script writes carries Connection: close and write_response takes close: (true by default): WireServer closes after one exchange, and without the header a client that pools keep-alive connections re-used one the server had already closed and read EOF on its next request one time in two against the async adapter. Net::HTTP builds a client per call and reads the header as nothing; the one keep-alive fixture passes close: false for its first response and every 8a count is unchanged. The two size pins (lifecycle_test.rb, 8a's driver) move to thirty-four, with TRANSPORT-12 and 13 vacuous by measurement there.
Phase 8c, review round 0's R0-2. The :async_http Steep target's comment and the phase record said ruby/gem_rbs_collection carries nothing for this gem's closure, and that the plan's `- name: async / ignore: true` row would ignore nothing. Measured otherwise: the collection carries gems/async/2.12, whose Task declares #stop and neither #cancel nor .current?, and `rbs collection install` installs it the moment the workspace gem's own `ignore: true` is lifted -- rbs cuts its dependency walk at an ignored gem, which is the only reason the walk never reached async on the committed tree. With the row in place and the workspace gem un-ignored the walk reaches the closure (38 gems) and still installs no async; with the gem ignored the lock is unchanged and steep stays green. The row is the manager's decision of 2026-09-21 restored on its true reason, and the Steepfile comment now states that reason.
The gemspec's comment said `~> 0.104` admits every 0.104.x and 0.105.x release. A two-segment pessimistic constraint is `>= 0.104, < 1`: it admits every 0.x release from 0.104 on, 0.200.0 and 0.999.9 included (measured with Gem::Requirement#satisfied_by? on 4.0.6). The constraint is the one the design chose; only the comment misdescribed it (review round 1, R1-3). No code changes.
The TRANSPORT-7 row's comment said a streaming adapter surfaces the cancellation from the body read. It does only when the cancel lands after the consumer's read has blocked: the server's signal fires as the head leaves its socket, and the cancel reaches the adapter either before it has checked its token on the delivered head (the send surfaces it, the in-flight path) or after (the read does). Measured against the async-http adapter with its delivered-response close deleted (review round 1, mutation 37): half the runs passed through the send path and half hung in the read. The row proves the in-flight clause on every adapter and the delivered-body clause only when the race falls that way; the adapter's own suite pins the body path with the consumer signalling from inside the read, and its driver bounds `around:` (the tests branch, R1-1). The contract has no primitive that tells the two apart for an eager and a streaming adapter alike, so the comment states the race rather than the row pretending to settle it. Comment only; no code changes.
A token cancel in flight while the exchange finishes raised ClosedQueueError out of Cancellation::Source#cancel on the canceller's thread (review round 2, R2-1). Source#cancel and Completer#settle both steal their hook list under their mutex, flip the state and run the hooks outside it; an exchange whose check-after-resume saw the flag in that window settled the pivot cancelled and closed its queue through release_watch, whose detach reached a list the source no longer held, and the adapter's hook then pushed onto the closed queue. Hooks.notify hands the first hook failure back to the caller, so an ordinary caller's own hook registered first -- or the canceller merely being descheduled between the flip and the push -- turned a legitimate cancel into a raise on the cancelling thread while the future read cancelled (reproduced deterministically on 4.0.6 and 3.3.12). Both pushes now go through Exchange#signal, which rescues ClosedQueueError: a hook that runs after the exchange ended has nothing left to do, the pivot is settled cancelled with the reason and the token reads cancelled, so the raise is swallowed there and nowhere else. A closed? check first would be the same race one instruction later. The watcher's comment also states that the CancelledError wrapped into Task#cancel(cause:) names the reason for whoever reads the task and is read back by nothing in the adapter -- the pivot is settled before the watcher acts (R2-4). The sig mirror gains the private method.
Sixteen suites for dexpace-transport-async_http: the ten unit suites beside each lib file, matrix_facts_test.rb re-running the eight async 2.46.0 facts the adapter's shape rests on per row, and the four behavioural ones -- cancellation_test.rb (the token, the future and a foreign OS thread reaching a blocked exchange; a cancel racing and following a delivery; a blocked body read woken), parent_cancellation_ test.rb (TRANSPORT-8's pair measured live against a silent server, the R13 close discipline), dispatch_conformance_test.rb (TRANSPORT-23 and ASYNC-22 over HTTP/1.1, plaintext HTTP/2 and TLS HTTP/2 by ALPN and across two threads, ASYNC-7's checkpoint abort, the standard async pipeline, a clean stderr) and wire_grammar_test.rb (TRANSPORT-12/13 and the wire-boundary re-validation dispatched over both protocols, with each library's antecedent measured) -- plus conformance_test.rb, the suite's second driver, which waives TRANSPORT-14 and TRANSPORT-27 by id and materialises a body inside the per-thread reactor it opens, because under async-http a response cannot outlive the reactor that produced it. The doubles are all AsyncHTTP-prefixed (one test:gems process): the recording sink and body, the holding and silent raw servers, the in-process async-http server fixture over three protocol shapes, and AsyncHTTPReactor, whose reactor_over closes a holding fixture inside the reactor so a defective adapter's blocked exchange fails an assertion rather than hanging the run, and whose assert_exchange_released proves the watcher is gone after an undelivered settlement. dexpace-conformance's two new groups are proven in both directions against RawWireTransport, which gains six defects and a drop mode; the two gate suites gain the per-gem floor cases over their fixtures.
Phase 8c, review round 0's R0-1, R0-3 and R0-4. R0-1 (blocking): the connection limit's "default 8" in clients_test.rb and the timeout's 60-second default in adapter_test.rb were pinned through Dexpace::Configuration.build, whose env_source: defaults to the real process environment, so both moved under an exported TRANSPORT_CONNECTION_LIMIT=3 / REQUEST_TIMEOUT=5s; the ASYNC-22 http1 bound in dispatch_conformance_test.rb moved the same way under TRANSPORT_CONNECTION_LIMIT=64 (16 connections where at most 8 were pinned, measured). Every configuration those cases build now comes from the new AsyncHTTPHermeticConfiguration support module -- Sources::NONE on the environment tier -- and all three suites stay green with the three variables exported. R0-3: the retag case fed BINARY chunks only, so dropping `chunk.b` from ResponseBody#each survived; the first chunk is now a UTF-8 literal and the yielded encodings are pinned, and that mutant runs red. R0-4: Exchange#net's request_cancel-unless-settled had no test; a deterministic case parks the adaptation-failure exit arm inside a native close that waits on a queue and cancels the parent there, so Async::Cancel leaves the rescue arm before Errors.settle ran and only the ensure's net settles the pivot -- removing it leaves the future pending until the bounded wait expires, and that mutant runs red too. No sleep anywhere.
Review round 1, R1-1. The body-path cancellation test passed with the watcher's close of the delivered response deleted (mutation 37): its own with_timeout fired inside the native read five seconds later, and the adapter's token-first classifier turned that Async::TimeoutError into the CancelledError(:reader_cancelled) the test expected, with the body closed. The test now refutes Async::TimeoutError as the error's cause -- a close under a blocked read surfaces as a bare IOError, which is what the classifier saw -- so the mutant fails in 5 s on 4.0.6 and 3.3.12 while the header's "never on elapsed time" rule still holds. The conformance driver's `around:` opened its reactor with no bound, so the same mutant hung the portable TRANSPORT-7 row (one run in two: the cancel races the client's head parse, and the other half takes the in-flight path and passes). A `with_timeout` around the assertion's body is not the fix: raised into the assertion's own fiber it meets the same classifier and the row PASSES thirty seconds late (measured). The driver now runs each assertion as a child task under `finished: false` and bounds the PARENT's wait at AROUND_BOUND (30 s); on expiry it cancels the child -- Async::Cancel, which no classifier converts, and body_string's ensure releases the connection so the reactor drains -- and raises the driver's own Failure, so the row flunks by name instead of hanging the run, with nothing written to stderr.
…kind Review round 2's R2-1, R2-2 and R2-3. R2-1: cancellation_test.rb gains the deterministic race -- an ordinary caller's hook registered before the adapter's parks the canceller between Source#cancel's flag flip and the adapter's hook, a fake client answers a 204 once the flag is up, check-after-resume settles the pivot cancelled and the exchange closes its queue, and only then does the adapter's hook run. Source#cancel must return true on the canceller's thread; against the pre-fix push it returned ClosedQueueError, and the canceller is unparked from an ensure so no assertion path leaves the thread parked. R2-2: request_mapper_test.rb's forged request takes method: and body:, and the body-forbidden clause is asserted over a forged GET and HEAD carrying a body -- the only shape that reaches the guard, since HTTP-7 makes a built GET's body nil -- with a forged POST as the control that shows the fixture carries its body through. Removing the guard now fails by name where it survived every suite. R2-3: the ASYNC-20 case reads the delivered response's watcher while the body is open -- one per response, transient -- and asserts it after the body's release, so a watcher spawned without transient: true fails in milliseconds by name instead of holding the reactor open until the run is killed. watcher_tasks joins the reactor helper beside assert_exchange_released, which now reads through it.
The checklist, one row per ID from what was built: ten own rows, nine implemented and ASYNC-21 not applicable with its one honourable property asserted, eleven cross-reference rows, the matrix facts re-run on three interpreters, thirty-three guard rows run red on 4.0.6 and 3.3.12 with three equivalent mutants recorded by measurement, the audit groups, forty-one departures from the plan's text, the findings routed and the two postponed items marked landed. The design's As-built addendum adds P8-91 to P8-102, numbered from the band the charter fixes. docs/sdk-documentation/transport-async_http.md is the twentieth as-built page, every example run on 4.0.6 and 3.3.12; conformance.md's counts, preamble and report examples are re-run with the two new groups; architecture.md, the gem README (the ASYNC-7 section, the reactor rule, the sync_over caveat, the async_over hazard and the timeout unit), README.md and docs/README.md point at the page. CLAUDE.md's built-phases paragraph gains the gem and core's key, its floor sentence names this gem's 3.3, its counts move, and its constraints list gains one line on what a reactor means for a response, a cancel and a close. first-release.md changes in existing entries only; two knowledge-note entries record what async 2.46.0 and async-http 0.105.0 measured that the design did not; the roadmap gains the phase's status note and one inbound bullet.
…mple Phase 8c, review round 0's docs half. R0-2: checklist deviation 9, the design's As-built row P8-99 and the roadmap's status note said the rbs collection carries nothing for this gem's closure; it carries async/2.12, which installed the moment the workspace gem's own ignore was lifted, and the row now on the code branch is recorded with that reason. R0-3 and R0-4: the two extra mutants review round 0 left surviving are rows 34 and 35 of "Guards run red", each with the guard that runs it red, the count sentence and the roadmap note updated, and guard 35 cited on TRANSPORT-8's row. R0-1: the audit-group row for configuration states that every default the suite pins is read through the new AsyncHTTPHermeticConfiguration support module, with the measurement that forced it, and the module joins the doubles list. R0-5: the as-built page's CRLF example is captioned as the model's own refusal and followed by a request-shaped object that met no builder, refused by the adapter's re-validation with the fixture recording nothing; `sink` and `drops` join the page's shorthand list. R0-7: the page's close paragraph states what a caller holding an open streaming response sees after the adapter closes, with the library's message as measured. Every changed example was re-run as one script on 4.0.6 and 3.3.12. Beside the findings, docs/first-release.md's native-extension sentence no longer confines the openssl compile to the 3.3 row: a networked resolve picks the newest openssl gem on 3.4 and 4.0 too.
Review round 1 (R1-1, R1-2), the documentation half. The checklist's TRANSPORT-7 row now says what the body-path test proves and how -- the wake's cause refuted as the test's own bound -- and that the portable row proves the delivered-body clause only when its cancel races past the adapter's token check (half the runs on 4.0.6, a third on 3.3.12, against a mutant with the watcher's close deleted); guard 37 joins the table, the guards paragraph counts its rows from the table (thirty-six of thirty-nine red, three equivalent) instead of restating the round-0 arithmetic, deviation 42 records the driver's child-task bound and why a with_timeout around the assertion's body is not it, and the race is routed to phase 10's inbound list as a dated bullet. The design's As-built addendum gains the same property beside its two stale facts. The counts the reviewer measured: the tests tip is 3,898 runs and 73,490 assertions on every six-gem row (round 0's 3,897 / 73,482 stood in the roadmap's status note), the honest RuboCop run inspects 708 files there, and the async_http manifest is 25 rows, a gain of twenty-three (the checklist, the roadmap note and P8-102 said 24 and twenty-two). conformance.md's driver paragraph names the bound, and the roadmap gains a dated paragraph for the two review rounds.
Review round 2's four findings recorded where each lives. The checklist's TRANSPORT-7 and ASYNC-6 rows state that a token cancel in flight while the exchange finishes never raises back into the caller's Source#cancel -- the source runs its hooks after stealing them, and Exchange#signal is total over the queue's close (deviation 43, guard 51) -- and that the CancelledError wrapped into Task#cancel(cause:) names the reason on the task's own Async::Cancel for whoever reads the task and is read back by nothing in the adapter (guard 46, equivalent by measurement). Guard rows 44 (the watcher's transience, asserted by name), 47v (the body-forbidden guard over a forged request) and 51 join the table: thirty-nine of forty-three rows red, four equivalent, on 4.0.6 and 3.3.12. The design's P8-91 row and its fact-8 note, the concurrency-and-async note's second fact and CLAUDE.md's bridge clause say the same instead of calling the wrap load-bearing; the as-built page gains the consumer-facing sentence (a cancel racing the exchange's own end is a no-op on the canceller's side). The roadmap's status note gains the round-2 paragraph, its guard arithmetic, the forty-third departure and the measured counts: 3,900 runs and 73,502 assertions on every six-gem row.
Carries what the replayed 8c commits cannot: the roadmap's dated reconciliation paragraph, the "Reconciled 2026-09-22" note at the head of 8c's checklist, the three documentation nits review round 3 left (guard row 18's status, guard rows 8 and 17's citations, and the code tip's coverage as measured), and CLAUDE.md's checklist count re-derived to twenty for the combined tree.
Wahbeh-Mohammad
force-pushed
the
32-phase-8c-asynchronous-transport-tests
branch
from
September 22, 2026 11:04
a56336e to
c51da2c
Compare
Wahbeh-Mohammad
force-pushed
the
32-phase-8c-asynchronous-transport-docs
branch
from
September 22, 2026 11:04
47c6afc to
0fdedd9
Compare
Wahbeh-Mohammad
changed the base branch from
32-phase-8c-asynchronous-transport-tests
to
main
September 22, 2026 11:05
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #32. Third PR of phase 8c's stack — the phase record and the documentation — on top of #97. Umbrella #29: 8c is the last lane — it closes by hand once both stacks have landed.
What lands
13 files.
docs/work/mvp/phase8/phase8c/2026-09-11-phase8c-asynchronous-transport-checklist.md— ten own rows (nine ✅;ASYNC-21N/A with its pull property asserted onResponseBody) plus eleven cross-reference rows and forty-three guard rows (forty red, three recorded equivalents: 14, 29 and 46; row 18 red by name through the round-2 hook-race case after round 3's R3-1); sections: requirement rows, what was built, matrix facts re-run on every interpreter, guards run red, audit groups run (seven), deviations from the plan (forty-three), findings routed, postponed work. Its head carries the dated "Reconciled 2026-09-22" note and the re-derived combined-tree figures.asyncfacts and the portable row's race stated as notes. The consolidation into design §10 is a human's —docs/sdk-design-ruby/is frozen, anddocs/deviations.mdwaits for phase 10.docs/sdk-documentation/transport-async_http.md(new; the twenty-first as-built page — every fence executed on 4.0.6 and 3.3.12, 61 printed values identical but for the ephemeral port),conformance.md(the counts,PREAMBLE, the two reports, the added rows),architecture.md, the gem's README,README.md,docs/README.md(both new pages linked).CLAUDE.md— the built-phases sentence names 8a, 8b and 8c and says the whole of phase 8 is built (8c landing second); no skeleton remains; both lanes' gem sentences, constraints and counts in merge order; twenty checklists, twenty-one pages, 220lib/dexpace/files, the nineteenprivate_constanttest-mirror exceptions.docs/first-release.md— existing entries only: the supported-Ruby note's 0.105.0 and openssl-on-3.3 clause, the conformance line's async-http status, theP8-9box ticked with the waivers stated, thegates:bounded_mapblocker's status.docs/knowledge/notes/concurrency-and-async.mdandtransport-adapter.md— two new Reference entries (the threeTask#cancel/Kernel#Asyncfacts; the reactor-exit drain,P8-37as built, the h2 double release and the 4.0 warning), none edited.EOFError, the portableTRANSPORT-7race), and the 2026-09-22 reconciliation paragraph carrying round 3's three nits and the re-derived counts.The reconcile pass's one chore commit (
chore: reconcile the 8c stack onto main after phase 8b) carries what no replayed 8c commit could: the checklist's dated note, the roadmap's reconciliation paragraph, the CLAUDE.md checklist count re-derived to twenty, and round 3's three nits — guard row 18 rewritten red-by-name with its close-count measurement kept as its own sentence and the guards arithmetic moved to forty of forty-three / three; guard rows 8 and 17's stale second citations corrected (the tls variant uses a callerssl_context; row 17 now namesadapter_test.rb's already-cancelled-token case); and the roadmap's code-tip coverage written as the measured 97.52%. The six shared files —CLAUDE.md,README.md,docs/README.md,architecture.md,docs/first-release.mdand the roadmap — were reconciled inside the replayed 8c commits and carry both lanes' content; every file only one lane touched is byte-identical to that lane's tip.docs/product-spec/,docs/sdk-design-ruby/,docs/knowledge/harvested/,docs/deviations.mdand every other phase's documents are untouched.Verification
Docs tip:
bundle exec rake(eighteen gates) green on 4.0.6 — 4,020 runs / 74,125 assertions / 0 failures / 7 skips / 99.88%, honest RuboCop 725 files clean;ruby .claude/skills/housekeeping/probe.rbexit 0 (all eight checks);ruby scripts/verify_knowledge_structure.rbOK; the housekeeping and knowledge tooling suites green (108/0 and 92/0); everyrubyfence oftransport-async_http.mdrun as one script on 4.0.6 and 3.3.12 and identical,conformance.md's changed fences on both, and 8b'sasync-thread.mdblocks once more on 4.0.6;surface:regeneratea no-op with the counts core 1,336 / async_http 25 / async-thread 14 / conformance 100 / net_http 17 / serde-json 15.Known follow-ups
The two routed phase-10 inbound bullets (the stale
asyncfacts plus the upstream Console noise; the portableTRANSPORT-7race) and the recorded equivalents — see the code PR's list.