Repository navigation
Phase 8c: asynchronous transport — the async-http adapter and the per-gem floor - #96
Merged
Merged
Conversation
This was referenced Sep 22, 2026
Wahbeh-Mohammad
changed the base branch from
31-phase-8b-async-runtime-adapter-docs
to
main
September 22, 2026 10:37
dexpace-transport-async_http alone needs Ruby >= 3.3, because async-http and its whole closure declare it (phase 8c's P8-36). The floor is one row in VERSIONS, `ruby floor:dexpace-transport-async_http 3.3`, and DexpaceVersions.ruby_floor(gem) with a KeyError fallback to the global floor is what the gemspecs, gates:versions and gates:gemspec_audit read. DexpaceVersions.gem_supported?(gem, ruby) is what the Gemfile, test:gems and gates:clean_bundle consult, so a row below a gem's floor skips that gem with the count printed instead of failing on Bundler's refusal. Two gate fixtures, per_gem_floor_ahead under versions/ and gemspec_audit/, are the deliberately failing inputs: a gemspec whose floor is below its VERSIONS row.
Phase 8c's reference asynchronous transport over async-http ~> 0.104: .build over a client map keyed by (reactor, origin) and bounded at MAX_ORIGINS, .using over a caller's own client with retries already zero, the Adapter behind both with its Closeable latch, one exchange task per call under the caller's own task and one with_timeout budget, the queue-marshalled cancellation bridge whose transient watcher acts on the reactor's thread (a Task#cancel from a foreign OS thread raises and cancels nothing), the RFC 7230 token predicate applied before dispatch on both protocols with DropPolicy's once-per-name, bounded reporting, the ten framing headers never copied, the wire-boundary re-validation's second call site, the lazy pull-shaped ResponseBody whose native close goes through close_quietly, the classifier that asks the token first, the lenient inbound mapper, the default TLS context offering h2 by ALPN, and the require-time registration under :async_http. Core gains Configuration::Keys::TRANSPORT_CONNECTION_LIMIT; the two key pins move, and the async seam's four in-process bare-require pins become a child-process suite, as 8a's registration did to the sync seam's. The :async_http Steep target relaxes UnknownConstant to :information as :serde_json's does, since nothing in the closure ships a sig/. The surface manifests are regenerated once: core gains the one key, this gem its twenty-two public rows. test/support/async_http_warmup.rb spends Ruby 4.0's once-per-process IO::Buffer warning at test-helper load, before the fatal-warning hook can see it.
Phase 8c's portable rows join dexpace-conformance as two private groups: Asynchronous (TRANSPORT-7, 9, 21, 23) and HeaderDrops (TRANSPORT-12, 13), each written against the suite contract's primitives alone, so the suite is thirty-four assertions in seven groups; TRANSPORT-8 stays an adapter's own row, which PREAMBLE now says beside 8a's two omissions. Every head a script writes carries Connection: close and write_response takes close: (true by default): WireServer closes after one exchange, and without the header a client that pools keep-alive connections re-used one the server had already closed and read EOF on its next request one time in two against the async adapter. Net::HTTP builds a client per call and reads the header as nothing; the one keep-alive fixture passes close: false for its first response and every 8a count is unchanged. The two size pins (lifecycle_test.rb, 8a's driver) move to thirty-four, with TRANSPORT-12 and 13 vacuous by measurement there.
Phase 8c, review round 0's R0-2. The :async_http Steep target's comment and the phase record said ruby/gem_rbs_collection carries nothing for this gem's closure, and that the plan's `- name: async / ignore: true` row would ignore nothing. Measured otherwise: the collection carries gems/async/2.12, whose Task declares #stop and neither #cancel nor .current?, and `rbs collection install` installs it the moment the workspace gem's own `ignore: true` is lifted -- rbs cuts its dependency walk at an ignored gem, which is the only reason the walk never reached async on the committed tree. With the row in place and the workspace gem un-ignored the walk reaches the closure (38 gems) and still installs no async; with the gem ignored the lock is unchanged and steep stays green. The row is the manager's decision of 2026-09-21 restored on its true reason, and the Steepfile comment now states that reason.
The gemspec's comment said `~> 0.104` admits every 0.104.x and 0.105.x release. A two-segment pessimistic constraint is `>= 0.104, < 1`: it admits every 0.x release from 0.104 on, 0.200.0 and 0.999.9 included (measured with Gem::Requirement#satisfied_by? on 4.0.6). The constraint is the one the design chose; only the comment misdescribed it (review round 1, R1-3). No code changes.
The TRANSPORT-7 row's comment said a streaming adapter surfaces the cancellation from the body read. It does only when the cancel lands after the consumer's read has blocked: the server's signal fires as the head leaves its socket, and the cancel reaches the adapter either before it has checked its token on the delivered head (the send surfaces it, the in-flight path) or after (the read does). Measured against the async-http adapter with its delivered-response close deleted (review round 1, mutation 37): half the runs passed through the send path and half hung in the read. The row proves the in-flight clause on every adapter and the delivered-body clause only when the race falls that way; the adapter's own suite pins the body path with the consumer signalling from inside the read, and its driver bounds `around:` (the tests branch, R1-1). The contract has no primitive that tells the two apart for an eager and a streaming adapter alike, so the comment states the race rather than the row pretending to settle it. Comment only; no code changes.
A token cancel in flight while the exchange finishes raised ClosedQueueError out of Cancellation::Source#cancel on the canceller's thread (review round 2, R2-1). Source#cancel and Completer#settle both steal their hook list under their mutex, flip the state and run the hooks outside it; an exchange whose check-after-resume saw the flag in that window settled the pivot cancelled and closed its queue through release_watch, whose detach reached a list the source no longer held, and the adapter's hook then pushed onto the closed queue. Hooks.notify hands the first hook failure back to the caller, so an ordinary caller's own hook registered first -- or the canceller merely being descheduled between the flip and the push -- turned a legitimate cancel into a raise on the cancelling thread while the future read cancelled (reproduced deterministically on 4.0.6 and 3.3.12). Both pushes now go through Exchange#signal, which rescues ClosedQueueError: a hook that runs after the exchange ended has nothing left to do, the pivot is settled cancelled with the reason and the token reads cancelled, so the raise is swallowed there and nowhere else. A closed? check first would be the same race one instruction later. The watcher's comment also states that the CancelledError wrapped into Task#cancel(cause:) names the reason for whoever reads the task and is read back by nothing in the adapter -- the pivot is settled before the watcher acts (R2-4). The sig mirror gains the private method.
Wahbeh-Mohammad
force-pushed
the
32-phase-8c-asynchronous-transport
branch
from
September 22, 2026 11:04
c05ada2 to
1dcaeca
Compare
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #32. First PR of phase 8c's three-PR stack — the code — built off
mainata7cfeb6(phases 0–7 and 8a) concurrently with 8b #31, reviewed and approved there, and reconciled onto 8b's tree (5755267) once 8b approved first — rebased with every commit preserved, re-proven at the rebased tip; this PR's base is 8b's docs branch, so the stack lands in order. The first transport that runs inside a reactor, and the asynchronous half of the wire-boundary re-validation phase 1 postponed.What lands
dexpace-transport-async_httpbecomes a real gem — 61 files —TRANSPORT-7,-8,-9,-12,-13,-21,-23,ASYNC-6,-21,-22(nine ✅,ASYNC-21N/A with its one honourable property asserted onResponseBody), plus eleven cross-reference rows (HTTP-17/18/XCUT-18,XCUT-14,TRANSPORT-1–-29's second driver,ASYNC-7,OBS-19,SEAM-5/6/16,SEAM-24,CFG-7,NFR-2/3/11/13,NFR-10/14).Dexpace::Transport::AsyncHTTP— the entry file's six constants (DEFAULT_TIMEOUT_SECONDS,DEFAULT_CONNECTION_LIMIT,MAX_ORIGINS,REGISTRY_KEY,FRAMING_HEADERS,ALPN_PROTOCOLS),.build(timeout:, logger:, drop_policy:, connection_limit:, ssl_context:, configuration:),.using(client, logger:, drop_policy:),.defaultand the require-timeAsyncTransport.register(:async_http, …, core: "~> 0.0"); ten files underasync_http/—adapter.rbanddrop_policy.rbpublic, the eightprivate_constantsclients,endpoints,errors,exchange,request_body,request_mapper,response_body,response_mapper— every one mirrored insig/and every one butexchange.rbintest/.P8-92) —Fiber.scheduleridentity at the call,MAX_ORIGINS(32) capping pairs, the drain evicting closed reactors first and releasing every evicted client; the cross-check's hang (oneAsync::HTTP::Clientshared by two threads each in their ownSync) is what forced it.P8-91): the token's hook andFuture#cancelsettle the pivot and push the reason; a transient watcher task pops it on the reactor's thread and cancels the in-flight exchange or closes the delivered response; both hooks push throughExchange#signal, which swallows theClosedQueueErrorof a push after the exchange's own end (round 2's R2-1).Async::Task#cancelfrom a foreign OS thread raisesNoMethodErrorand cancels nothing — measured on 3.3.12, 3.4.10 and 4.0.6.P8-40):HeaderSyntax.token?applied before dispatch over HTTP/1.1, plaintext h2 and TLS h2; the seventeen delimiter bytesHTTP-17admits and the grammar refuses are the antecedent.DropPolicy—EVERY/ONCE_PER_NAME/QUIET, the folded-name latch bounded at 64 — lands phase 5b's postponedOBS-19.RequestMapper#revalidate!is the second wire-boundary call site; noContent-Typeis invented (P8-97).ResponseBody— pull-per-demand, one native read per yield, a memoised#source, mid-stream failures asStreamError, and the native close throughclose_quietlybecauseasync-http0.105.0 double-releases an unread h2 body (P8-101).dexpace-conformancegains groups six and seven,Asynchronous(TRANSPORT-7,-9,-21,-23) andHeaderDrops(TRANSPORT-12,-13) — 28 → 34 assertions in seven groups,PREAMBLEnamingTRANSPORT-8as the third thing a green run does not prove — andScriptswritesConnection: closeon every head with aclose:keyword for the one keep-alive fixture (P8-96), with every 8a count unchanged.VERSIONS'ruby floor:dexpace-transport-async_http 3.3,DexpaceVersions.ruby_floor(gem)/.gem_supported?,gates:versionsandgates:gemspec_auditreading it, theGemfile/test:gems/gates:clean_bundleskips, twoper_gem_floor_aheadfixtures); theSteepfile's:async_httptarget relaxed as:serde_jsonis, withlibrary "openssl", "uri"and therbs_collection.yamlrow ignoring the staleasync/2.12by name (R0-2);test/support/async_http_warmup.rbparking Ruby 4.0's once-per-processIO::Bufferwarning (P8-98); one core widening,Configuration::Keys::TRANSPORT_CONNECTION_LIMIT, with its two pins flipped, core's four registration-invalidated pins moved to a child process, and both surface manifests regenerated (core 1,335 → 1,336,async_http2 → 25).Decisions taken in the open, against the plan's text
Ledger rows P8-91–P8-102 in the design's As-built addendum and the checklist's "Deviations from the plan" (forty-three items). Beyond those above:
TRANSPORT-8lives in this gem's own suite (the runtime-originated cancel has no portable antecedent) and six portable assertions plus thePREAMBLEsentence are the substitute (P8-93); a response does not outlive the reactor that produced it, so the driver's foreign-thread settle materialises the body inside its reactor (P8-94); the floor is oneVERSIONSrow read by gemspec, gates and loaders (P8-95);P8-37as built retires every pooled resource beforepool.close, becausepool.closealone drains and waits exactly asClient#closedoes (P8-100); the public surface is.build/.usingover.owning/.borrowing,DropPolicypublic, no_Clientinterface (P8-102). Two of the design's thirteen facts are stale onasync2.46.0 —cause:drops a Symbol, andKernel#Asyncinside a task is the task's child — corrected in a new knowledge-note entry and routed to phase 10's inbound list.Layering
Each tip is green under every gate on its own tree — all eighteen. This branch is green on the SimpleCov floor too: 97.57% on 4.0.6 at the reconciled tip (3,833 runs, 73,334 assertions, 0 failures, 3 skips — the
net_httpdriver'sTRANSPORT-18vacuity plus the two new groups' rows measured vacuous there; 97.52% / 3,713 / 3 at the reviewed tip, whereExchange#signal's rescue branch was the three uncovered lines); the 3.2.11 matrix row is green with the gem absent by its floor (3,824 runs, 3 skips, 95.62%,gates:clean_bundle"5 gem(s)"). The tests PR takes the same tree to 99.88% — 4,020 runs / 74,125 assertions / 7 skips: thenet_httpdriver's three and the async driver's four (twoTRANSPORT-14assertions and oneTRANSPORT-27waived by id,TRANSPORT-18vacuous).Verification
Exchange#signalswallowing a hook's push after the exchange's own end (round 2's R2-1); everything else code-side was a comment or configuration correction (the rbs-collection row and Steepfile comment, round 0's R0-2; the gemspec's~> 0.104comment, round 1's R1-3; the portable row's race comment, round 1's R1-1). Round 3's three nits were documentation, folded into the reconcile pass's chore commit.Kernel#Asyncis the current task's child on 2.46.0), 29 (a raise inside#dispatch's fence is still a settlement) and 46 (thecause:wrap is read back by nothing), with row 18 red since the round-2 hook-race case caught it. Forty-three guards run red and recorded in the checklist.c05ada2/a56336e/47c6afc, on 8b's tree): all eighteen gates individually at the code tip on 4.0.6 and the matrix set on 3.2.11; the full default task at the tests and docs tips; three extra seeds identical; 8b'scomposed_transport_test.rbgreen by name (10 runs / 45 assertions) against theConnection: closefixture;surface:regeneratea no-op; probe clean.Known follow-ups from the reviews (not blocking a gate)
Kernel#Asyncdelegates toTask.current.async), guard 29 (equivalent by construction), guard 46 (thecause:wrap names the reason for a debugger and nothing reads it back); guard 18 is red through the round-2 case and its close-count measurement stands separately.TRANSPORT-14and a waiver is by id; a per-assertion waiver protocol is 8a's design's to change.TRANSPORT-7row proves its delivered-body clause by chance against a streaming adapter (the cancel races the client's head parse; half the runs on 4.0.6, a third on 3.3.12 against the mutant) — stated in the row's comment, the checklist and the design addendum, and routed to phase 10's inbound list; the adapter's own body-path test is the deterministic proof.AsyncHTTPServerFixture::QuietServerswallows a peer's mid-headEOFErrorfrom async-http's server (seen once in ~100 whole-suite runs; upstream report on the same inbound list); the three configuration-default pins build throughAsyncHTTPHermeticConfigurationbecauseConfiguration.buildreads the real environment.docs/first-release.mdcarries thegates:bounded_mapstatus (the map is bounded, the gate does not exist until phase 9), the conformance line and theP8-9box;OBS-29has no route and says so; the 3.4.10 row's scratch bundle resolved the installedopenssl4.0.2 rather than a fresh resolve.