diff --git a/CLAUDE.md b/CLAUDE.md index bd02b86..0c0168f 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -13,16 +13,16 @@ three-state PATCH — solved exactly once, and it deliberately does not compete Work here is **spec-driven, not feature-driven**. `docs/product-spec/` is normative: 645 numbered requirements across 19 prefixes. Before implementing anything, find the requirement IDs it must satisfy. -**Phases 0, 1, 2, 3a, 3b, 4a, 4b, 4c, 5a, 5b, 5c, 6a, 6b, 6c, 7b, 7c, 7a, 8a and 8b are built — the whole of +**Phases 0, 1, 2, 3a, 3b, 4a, 4b, 4c, 5a, 5b, 5c, 6a, 6b, 6c, 7b, 7c, 7a, 8a, 8b and 8c are built — the whole of phase 6, the whole of phase 7, whose three sub-phases were built concurrently off one base and landed in that -order, 7a last (umbrella #25 closes by hand), the first of phase 8's three sub-phases, built off the same -base as phase 7's and reconciled onto the tree that holds all three, and the second, built off the tree that -holds 8a concurrently with 8c; the domain model, the seam layer, the +order, 7a last (umbrella #25 closes by hand), and the whole of phase 8, whose three sub-phases were built +8a first, off the same base as phase 7's and reconciled onto the tree that holds all three, then 8b and 8c +concurrently off the tree that holds 8a, 8c landing second; the domain model, the seam layer, the byte-streaming layer, the body layer, the execution context, the recovery layer, the stage pipeline, the configuration layer, the tracing and metrics layer, the logging facade with its redaction, the retry layer, the authentication layer, the redirect layer, the server-sent-events layer, the pagination layer, the -serialization layer, the synchronous transport, the conformance suite and the thread-pool executor are the -only domain code.** Six gems exist under `gems/`, every one at `0.0.0`. `dexpace-core` carries the HTTP domain model — `Dexpace::Request`, +serialization layer, the synchronous transport, the conformance suite, the thread-pool executor and the +asynchronous transport are the only domain code.** Six gems exist under `gems/`, every one at `0.0.0`. `dexpace-core` carries the HTTP domain model — `Dexpace::Request`, `Response`, `Headers`, `Status`, `Method`, `Protocol`, `MediaType`, `Query`, `RequestOptions`, `HeaderName`, the `HeaderSyntax`, `PercentEncoding` and `URL` function modules, and the construction contract `Dexpace::Model` / `Dexpace::Builder` under one error root, `Dexpace::Error` @@ -236,17 +236,32 @@ and `DEFAULT_NAME`, and `Dexpace::Closeable`'s latched `#close` that drains with `Events::INSTRUMENTATION_SHUTDOWN` once — the `private_constant` `Timer` with its `Entry`, `Pool::Job`, the two private field keys, `RejectedError`, `REQUIRED_CORE` and the require-time version-skew assertion made directly because there is no executor registry -(`docs/work/mvp/phase8/phase8b/2026-09-11-phase8b-async-runtime-adapter-checklist.md`); -the remaining gem's `lib/` still holds its namespace module and a `VERSION` constant and nothing else. The -synchronous transport is the first thing here that talks to a socket, and the pool is the first executor on -the async path — the two meet in `dexpace-async-thread`'s composed suite over a real socket. The workspace root +(`docs/work/mvp/phase8/phase8b/2026-09-11-phase8b-async-runtime-adapter-checklist.md`); plus phase 8c's one +addition to core, `Configuration::Keys::TRANSPORT_CONNECTION_LIMIT`, the tenth key, **and the workspace's +sixth real gem**, the asynchronous transport in `dexpace-transport-async_http`, +`Dexpace::Transport::AsyncHTTP` — `.build(timeout:, logger:, drop_policy:, connection_limit:, ssl_context:, +configuration:)` over a client map the adapter owns, one `Async::HTTP::Client` per (reactor, origin) bounded at +`MAX_ORIGINS`, and `.using(client, logger:, drop_policy:)` over a caller's own, the `Adapter` behind both with +`.owning` / `.borrowing` and `REACTOR_MESSAGE`, the public `DropPolicy` with `MAX_TRACKED_NAMES`, `EVERY`, +`ONCE_PER_NAME`, `QUIET` and `MODES`, the six constants `DEFAULT_TIMEOUT_SECONDS`, `DEFAULT_CONNECTION_LIMIT`, +`MAX_ORIGINS`, `REGISTRY_KEY`, `FRAMING_HEADERS` and `ALPN_PROTOCOLS`, the eight `private_constant`s `Clients`, +`Endpoints`, `Errors`, `Exchange`, `RequestBody`, `RequestMapper`, `ResponseBody` and `ResponseMapper`, the RBS +interface `_Release`, and the require-time `AsyncTransport.register(:async_http, …)` — and, in +`dexpace-conformance`, the two private groups `Asynchronous` and `HeaderDrops` that make the suite thirty-four +assertions in seven, with `PREAMBLE` naming `TRANSPORT-8` and `Scripts.write_response` taking `close:` +(`docs/work/mvp/phase8/phase8c/2026-09-11-phase8c-asynchronous-transport-checklist.md`); +the synchronous transport is the first thing here that talks to a socket, and the pool is the first executor +on the async path — the two meet in `dexpace-async-thread`'s composed suite over a real socket. Both +transports talk to a socket; the async one needs a running reactor on the calling thread and creates none. The workspace root carries the `Gemfile`, `Rakefile`, `Steepfile`, `rbs_collection.yaml`, `.rubocop.yml`, `.yardopts`, `VERSIONS` and the eighteen blocking gates — phase 0's seventeen (`docs/work/mvp/phase0/2026-09-05-phase0-scaffold-and-quality-gates-checklist.md`) and phase 7b's `gates:serde_boundary`. Beyond that, what exists is the specification, the port design, the process tooling and the register, -`docs/deviations.md`. Ruby **>= 3.2** is the floor (`required_ruby_version` in every gemspec, asserted by -`gates:versions`); CI runs a 3.2 / 3.3 / 3.4 / 4.0 matrix; every Ruby fact in the design was verified against +`docs/deviations.md`. Ruby **>= 3.2** is the floor (`required_ruby_version` in every gemspec but one, asserted by +`gates:versions`; `dexpace-transport-async_http` alone declares **>= 3.3**, read from `VERSIONS`' per-gem +`floor:` row, because `async-http`'s whole closure does — phase 8c's `P8-36` — and the 3.2 row installs, tests +and clean-bundles the other five); CI runs a 3.2 / 3.3 / 3.4 / 4.0 matrix; every Ruby fact in the design was verified against 3.4.10 and the ones the gates and the domain model rest on were re-verified against 3.2.11, 3.4.10 and 4.0.6. Top-level namespace is `Dexpace`. Gem names are hyphenated and map segment-for-segment onto the constant path: @@ -258,7 +273,7 @@ MVP gems (`docs/sdk-design-ruby/02-gem-and-workspace-layout.md` §2.1): |---|---|---| | `dexpace-core` | `Dexpace` | **none** | | `dexpace-transport-net_http` | `Dexpace::Transport::NetHTTP` | `dexpace-core`; `net-http` (a default gem) | -| `dexpace-transport-async_http` | `Dexpace::Transport::AsyncHTTP` | `dexpace-core`; `async-http` | +| `dexpace-transport-async_http` | `Dexpace::Transport::AsyncHTTP` | `dexpace-core`; `async-http ~> 0.104` | | `dexpace-serde-json` | `Dexpace::Serde::JSON` | `dexpace-core`; `json >= 2.19.9` | | `dexpace-async-thread` | `Dexpace::Async::Thread` | `dexpace-core` | | `dexpace-conformance` | `Dexpace::Conformance` | `dexpace-core` | @@ -1111,6 +1126,29 @@ Each is one line plus the chapter to read before touching the area. `class RecordingSink` in an adapter gem re-assigns core's `RecordingSink::Entry`, an "already initialized constant" warning `NFR-6` makes fatal at load time; the gem's own `rake test` never sees it. The adapter gem's double is `NetHTTPRecordingSink` for that reason (8a's checklist, departure 35). +- **Under `async-http` a response does not outlive the reactor that produced it, every call needs a + reactor on the calling thread, and a cancellation from another OS thread is marshalled through a queue** + — `Sync { }` returns only when the reactor has no non-transient work left, and on its way out it stops + the pool's transient gardener, whose `ensure` drains the pool and waits on every busy connection; the + connection behind an unread body is busy, so a `Sync` that hands a streaming response out past its own + end never returns (`TRANSPORT-29`'s eight-thread assertion found it; the 8c driver materialises the body + inside its per-thread reactor, `P8-94`). `Adapter#call` outside `Async::Task.current?` settles + `SeamError` through the future and creates no reactor (`P8-39`); `Async::Task#cancel` from a foreign + thread raises `NoMethodError` and cancels nothing, so the token's hook pushes onto a `Thread::Queue` + that a transient watcher task under the caller's task pops and acts on from the reactor's thread + (`P8-91`) — the push total over the queue's close, because the source and the completer both run + their hooks after stealing them and an exchange that finished in between has closed the queue, + and the reason wrapped in `CancelledError` for the task's own `Async::Cancel` alone (`cause:` + drops anything but an `Exception`; the adapter reads none back, the pivot being settled before + the watcher acts); the owning adapter's clients are keyed by `(Fiber.scheduler, origin)` because one + `Async::HTTP::Client` cannot serve two reactors on two threads (`P8-92`); `Adapter#close` retires every + pooled resource **before** `pool.close`, which alone would drain and wait exactly as `Client#close` + does (`P8-100`); and a native HTTP/2 body is closed through `close_quietly`, because `async-http` + writes `RST_STREAM` before it transitions the stream and an `END_STREAM` in that window releases the + pooled connection twice (`P8-101`). `Kernel#Async` inside a task IS that task's child on async 2.46 + (the design's fact 10 is stale), and on Ruby 4.0 alone the first `IO::Buffer` under a scheduler prints + a once-per-process experimental warning that `test/support/async_http_warmup.rb` spends before the + fatal hook can see it (`P8-98`). - **A conformance assertion sends through `kase.settle(transport, request, options, cancellation)` and never `transport.call`, names no adapter, and waits with a bound** — what `TransportCase#transport` returns is the `SettleOnly` guard, whose `#call` raises, because the default `settle` IS @@ -1310,27 +1348,30 @@ probe compares each against the live tree, and a count written anywhere else in place that floor is stated. `dexpace-transport-net_http`'s `lib/` holds the phase-8a synchronous transport — nine files, the entry file and eight under `net_http/`, seven of them `private_constant`s, every one mirrored in `sig/` and in `test/` — and its gemspec declares `net-http >= 0.4`, a default gem - declared as the `NFR-2` third-party half; `dexpace-conformance`'s holds the phase-8a conformance suite — - twenty-three files beside phase 0's `version.rb`, seven of them `private_constant`s, every one mirrored - in `sig/` and every one but `transport_suite/checks.rb`, `wire_server/recorded_request.rb` and + declared as the `NFR-2` third-party half; `dexpace-transport-async_http`'s `lib/` holds the phase-8c + asynchronous transport — eleven files, the entry file and ten under `async_http/` beside phase 0's + `version.rb`, eight of them `private_constant`s, every one mirrored in `sig/` and every one but + `async_http/exchange.rb` mirrored in `test/` (the per-call exchange is proven through the three + behavioural suites that drive it) — and its gemspec declares `async-http ~> 0.104` as the `NFR-2` + third-party half and a Ruby floor of 3.3 read from `VERSIONS`; `dexpace-async-thread`'s `lib/` holds the + phase-8b async-runtime adapter — the entry file and three files under `thread/`, `rejected_error.rb`, + `pool.rb` and `timer.rb`, the last a `private_constant`, every one mirrored in `sig/` and the two public + ones in `test/` (`timer.rb` is proven through `pool_delay_test.rb` and `pool_test.rb`'s source scans) — + and its gemspec declares `dexpace-core` alone, by design: the gem spends none of its `NFR-2` budget; + `dexpace-conformance`'s holds the phase-8a conformance suite with phase 8c's two groups — twenty-five + files beside phase 0's `version.rb`, nine of them `private_constant`s, every one mirrored in `sig/` and + every one but `transport_suite/checks.rb`, `wire_server/recorded_request.rb` and `wire_server/request_reader.rb` mirrored in `test/` — and its gemspec declares `dexpace-core` alone, its - `socket` and `tempfile` requires carried by the allowlist's exceptions. - `dexpace-async-thread`'s `lib/` holds the phase-8b async-runtime adapter — the entry file and three - files under `thread/`, `rejected_error.rb`, `pool.rb` and `timer.rb`, the last a `private_constant`, - every one mirrored in `sig/` and the two public ones in `test/` (`timer.rb` is proven through - `pool_delay_test.rb` and `pool_test.rb`'s source scans) — and its gemspec declares `dexpace-core` - alone, by design: the gem spends none of its `NFR-2` budget. - The other gem — `dexpace-transport-async_http` — is a phase-0 skeleton - whose `lib/` holds the namespace module and a `VERSION` constant and nothing else, and its gemspec - declares `dexpace-core` and no third-party gem yet (design P0-9); the third-party half of its `NFR-2` - budget arrives with the phase that writes the code needing it, as 7a's and 8a's did. + `socket` and `tempfile` requires carried by the allowlist's exceptions. Every gem under `gems/` is real: + no phase-0 skeleton remains, and the third-party half of each `NFR-2` budget arrived with the phase that + wrote the code needing it — 7a's, 8a's and 8c's (8b's spends none). - There are eleven phase directories under `docs/work/*/`; `mvp/` is the only delivery, and it holds the v1 roadmap, `docs/work/mvp/2026-09-05-ruby-sdk-v1-roadmap-design.md`, plus `phase0/`, `phase1/`, `phase2/`, `phase3/`, `phase4/`, `phase5/`, `phase6/`, `phase7/`, `phase8/`, `phase9/` and `phase10/`. `phase0/`, `phase1/` and `phase2/` each carry that phase's design, plan and checklist; `phase3/` carries its segmentation design, `docs/work/mvp/phase3/2026-09-08-phase3-segmentation-design.md`, and two sub-phase directories — `phase3/phase3a/` and `phase3/phase3b/`, each holding that sub-phase's design, plan and checklist — - nineteen checklists written so far, each at implementation; `phase4/` + twenty checklists written so far, each at implementation; `phase4/` carries its segmentation design, `docs/work/mvp/phase4/2026-09-08-phase4-segmentation-design.md`, and three sub-phase directories — `phase4/phase4a/`, `phase4/phase4b/` and `phase4/phase4c/`; each holds that sub-phase's @@ -1362,8 +1403,8 @@ probe compares each against the live tree, and a count written anywhere else in `docs/work/mvp/phase8/2026-09-11-phase8-segmentation-design.md`, and three sub-phase directories — `phase8/phase8a/` (synchronous transport and the conformance gem), `phase8/phase8b/` (async-runtime adapter) and `phase8/phase8c/` (asynchronous transport); - each holds a design and a plan, and `phase8/phase8a/` and `phase8/phase8b/` a checklist too, written at - implementation on 2026-09-20 and 2026-09-21. Phase 8 is 52 IDs (`TRANSPORT-1`–`30`, `ASYNC-1`–`22`) and is + each holds a design and a plan, and `phase8/phase8a/`, `phase8/phase8b/` and `phase8/phase8c/` checklists + too, written at implementation on 2026-09-20, 2026-09-21 and 2026-09-21. Phase 8 is 52 IDs (`TRANSPORT-1`–`30`, `ASYNC-1`–`22`) and is the phase that ships the most gems in the roadmap — `dexpace-transport-net_http`, `dexpace-async-thread`, `dexpace-transport-async_http` and `dexpace-conformance`, whose gemspec, version and first release phase 8 owns. Its three sub-phases are independent, so @@ -1406,5 +1447,5 @@ probe compares each against the live tree, and a count written anywhere else in at `0.0.0`. Every checklist but phase 0's, phase 1's, phase 2's, phase 3a's, phase 3b's, phase 4a's, phase 4b's, phase 4c's, phase 5a's, phase 5b's, phase 5c's, phase 6a's, phase 6b's, phase 6c's, phase 7b's, - phase 7c's, phase 7a's, phase 8a's and phase 8b's is still to be written at execution time. + phase 7c's, phase 7a's, phase 8a's, phase 8b's and phase 8c's is still to be written at execution time. - There are 40 harvested topics under `docs/knowledge/harvested/`; the harvest ran here on 2026-09-05. diff --git a/Gemfile b/Gemfile index 421d1cd..1494e75 100644 --- a/Gemfile +++ b/Gemfile @@ -12,7 +12,12 @@ group :development, :test do end # The workspace's own gems, by path, so `bundle exec` resolves them without an install: the six -# MVP skeletons under gems/, and any gem a later phase adds there. +# MVP gems under gems/, and any gem a later phase adds there. A gem whose own VERSIONS floor this +# interpreter does not meet is left out rather than handed to Bundler, which refuses a path gem's +# required_ruby_version at install time for the whole workspace (phase 8c's P8-36: only +# dexpace-transport-async_http, on the 3.2 row). Dir.glob("gems/*", base: __dir__).sort.each do |dir| + next unless DexpaceVersions.gem_supported?(File.basename(dir)) + gem File.basename(dir), path: dir end diff --git a/README.md b/README.md index f9cdd45..a3f69f2 100644 --- a/README.md +++ b/README.md @@ -15,7 +15,7 @@ not compete with `faraday` or `httpx` on the easiest way to fetch a JSON endpoin ## Status -**Phases 0, 1, 2, 3a, 3b, 4a, 4b, 4c, 5a, 5b, 5c, 6a, 6b, 6c, 7b, 7c, 7a, 8a and 8b are built.** Nothing is published. The repository holds six gems under +**Phases 0, 1, 2, 3a, 3b, 4a, 4b, 4c, 5a, 5b, 5c, 6a, 6b, 6c, 7b, 7c, 7a, 8a, 8b and 8c are built.** Nothing is published. The repository holds six gems under `gems/`, every one at `0.0.0`. `dexpace-core` carries the HTTP domain model — the frozen, validated wire types every later phase stands on (`docs/sdk-documentation/http.md`) — the seam layer: the provider registry, the transport and codec seams, the core-owned async pivot, @@ -90,23 +90,31 @@ to a socket: two constructions over a fresh-per-call or a borrowed `Net::HTTP`, the wire carries, a per-response producer thread behind every streamed body, one total budget across three native knobs, a classifier that asks the cancellation token first and wraps every other failure retryable, TLS settings and the proxy over the configuration chain -(`docs/sdk-documentation/transport-net_http.md`) — and `dexpace-conformance` carries the -conformance suite: the assertion protocol, the twenty-eight-assertion transport suite with its +(`docs/sdk-documentation/transport-net_http.md`) — `dexpace-transport-async_http` carries the +asynchronous transport, the first thing on the async path that talks to a socket: two +constructions over a reactor-keyed client map or a borrowed `Async::HTTP::Client`, one exchange task +per call under the caller's own reactor task with one total budget, the queue-marshalled +cancellation bridge that lets a token cancelled from any thread reach the exchange, the RFC 7230 +token predicate applied on both protocols with once-per-name drop reporting, the lazy pull-shaped +response body, HTTP/2 by ALPN over TLS, and a Ruby floor of 3.3 that this gem alone declares +(`docs/sdk-documentation/transport-async_http.md`) — and `dexpace-conformance` carries the +conformance suite: the assertion protocol, the thirty-four-assertion transport suite with its vacuous and waived rows, the plaintext wire fixture, the Minitest and RSpec drivers and the two -observability doubles (`docs/sdk-documentation/conformance.md`). `dexpace-async-thread` carries the +observability doubles, proven by both transports as its two drivers +(`docs/sdk-documentation/conformance.md`). `dexpace-async-thread` carries the async-runtime adapter — the first executor on the async path: a fixed-size thread pool over a bounded queue whose `#post` never blocks, the bridge that makes a blocking transport asynchronous on a worker and closes an orphaned result exactly once, the diagnostic context carried across the hop with the two clears that keep it the caller's, a scheduled delay on one timer thread, and the idempotent bounded close that emits the lifecycle event phase 2 postponed -(`docs/sdk-documentation/async-thread.md`). The other one is still a -skeleton — a namespace, a `VERSION`, a gemspec, a signature mirror and a smoke suite: +(`docs/sdk-documentation/async-thread.md`) — and with that no skeleton remains: all six gems under +`gems/` carry their phase's code. | Gem | Namespace | Runtime dependencies today | |---|---|---| | `dexpace-core` | `Dexpace` | none | | `dexpace-transport-net_http` | `Dexpace::Transport::NetHTTP` | `dexpace-core`; `net-http >= 0.4` | -| `dexpace-transport-async_http` | `Dexpace::Transport::AsyncHTTP` | `dexpace-core` | +| `dexpace-transport-async_http` | `Dexpace::Transport::AsyncHTTP` | `dexpace-core`; `async-http ~> 0.104` (Ruby >= 3.3) | | `dexpace-serde-json` | `Dexpace::Serde::JSON` | `dexpace-core`; `json >= 2.19.9` | | `dexpace-async-thread` | `Dexpace::Async::Thread` | `dexpace-core` | | `dexpace-conformance` | `Dexpace::Conformance` | `dexpace-core` | diff --git a/Steepfile b/Steepfile index 219c3b0..3d5bd1f 100644 --- a/Steepfile +++ b/Steepfile @@ -32,7 +32,21 @@ end target :async_http do check "gems/dexpace-transport-async_http/lib" signature "gems/dexpace-transport-async_http/sig", "gems/dexpace-core/sig" - configure_code_diagnostics(D::Ruby.default) + # rbs's own stdlib signature sets for the two features this gem's lib/ names beyond core's + # allowlist: `openssl` for the default TLS context and `uri` for the request URL. + library "openssl", "uri" + # The second relaxation, on this target alone (phase 8c), for the same reason as + # :serde_json's: none of async, async-http, protocol-http or async-pool ships a sig/, and the + # one entry the collection carries for the closure -- `async/2.12`, whose Task declares `#stop` + # and neither `#cancel` nor `.current?` -- is ignored by name in rbs_collection.yaml because it + # would type the primitives this gem calls as missing, so every `::Async::HTTP::Client.new`, + # `::Protocol::HTTP::Request.new` and the `< ::Protocol::HTTP::Body::Readable` superclass is a + # Ruby::UnknownConstant that steep's default warning severity turns into a red gate. Downgraded + # to :information here, never a line-level ignore and never on core's strict target; every + # handle on the runtime is typed `untyped` in the gem's sig (NFR-11 admits no async-family type + # there either). Re-tighten to D::Ruby.default at the first release of those gems, or of the + # collection, that declares what this gem calls. + configure_code_diagnostics(D::Ruby.default.merge({ D::Ruby::UnknownConstant => :information })) end target :serde_json do diff --git a/VERSIONS b/VERSIONS index ad1199e..d01b751 100644 --- a/VERSIONS +++ b/VERSIONS @@ -3,6 +3,7 @@ # gem read by that gem's gemspec and by tools/versions.rb # tool read by the root Gemfile # ruby floor required_ruby_version in every gemspec (NFR-10) +# ruby floor: one gem's own floor, when narrower than the global one # ruby matrix the CI matrix; asserted against .github/workflows/ci.yml # ruby dev the development pin; asserted against .ruby-version # @@ -36,5 +37,9 @@ tool bundler-audit ~> 0.9 tool prism ~> 1.9 ruby floor 3.2 +# P8-36 (phase 8c): async-http 0.95.0 and async 2.38.0 raised their floors to 3.3, so this one +# gem declares 3.3 and is absent from the bundle, test:gems and gates:clean_bundle on the 3.2 row. +# Colon-joined into the three-token `name` column so the grammar above needs no change. +ruby floor:dexpace-transport-async_http 3.3 ruby matrix 3.2 3.3 3.4 4.0 ruby dev 4.0.6 diff --git a/docs/README.md b/docs/README.md index 65503db..c51daf2 100644 --- a/docs/README.md +++ b/docs/README.md @@ -108,12 +108,13 @@ layer, phase 6a's retry layer, phase 6c's authentication layer, phase 6b's redir 7b's server-sent-events layer, phase 7c's pagination layer, phase 7a's serialization layer and phase 8a's transport error; `dexpace-serde-json` holds phase 7a's JSON codec and declares `json >= 2.19.9`; `dexpace-transport-net_http` holds phase 8a's synchronous transport and declares -`net-http >= 0.4`, and `dexpace-conformance` phase 8a's assertion protocol, transport suite, wire -fixture, two drivers and two doubles; `dexpace-async-thread` holds phase 8b's thread pool, its -rejection error and the version-skew guard, and declares `dexpace-core` alone; the other one is -phase 0's skeleton, a namespace and a -`VERSION`. The as-built documentation lands in -[`sdk-documentation/`](./sdk-documentation/) as each gem gains code; the twenty pages written so far are [`sdk-documentation/quality-gates.md`](./sdk-documentation/quality-gates.md), because the +`net-http >= 0.4`; `dexpace-transport-async_http` holds phase 8c's asynchronous transport, declares +`async-http ~> 0.104` and a Ruby floor of 3.3 of its own, and `dexpace-conformance` phase 8a's +assertion protocol, transport suite, wire fixture, two drivers and two doubles, with phase 8c's two +assertion groups appended; `dexpace-async-thread` holds phase 8b's thread pool, its rejection error +and the version-skew guard, and declares `dexpace-core` alone. Every one of the six gems is real; no +phase-0 skeleton remains. The as-built documentation lands in +[`sdk-documentation/`](./sdk-documentation/) as each gem gains code; the twenty-one pages written so far are [`sdk-documentation/quality-gates.md`](./sdk-documentation/quality-gates.md), because the gate set is the thing phase 0 built, [`sdk-documentation/http.md`](./sdk-documentation/http.md), because the domain model is the thing phase 1 built, [`sdk-documentation/seams.md`](./sdk-documentation/seams.md), because the seam layer is the thing @@ -150,7 +151,9 @@ JSON codec are the things phase 7a built, and transport and the conformance suite are what phase 8a built — the first code outside `dexpace-core` after 7a's codec — and [`sdk-documentation/async-thread.md`](./sdk-documentation/async-thread.md), because the thread pool -that is the async path's first executor is what phase 8b built. +that is the async path's first executor is what phase 8b built, and +[`sdk-documentation/transport-async_http.md`](./sdk-documentation/transport-async_http.md), because +the asynchronous transport, the suite's second driver, is what phase 8c built. ## Keeping this file true diff --git a/docs/first-release.md b/docs/first-release.md index 1a6e348..ee40537 100644 --- a/docs/first-release.md +++ b/docs/first-release.md @@ -36,7 +36,14 @@ them at `0.0.0` until the first release: repository floor of 3.2 (phase 8c's deviation `P8-36`, whose per-gem Ruby floor gate edit is 8c's plan, Task 3): `async-http` 0.104.0 and its whole dependency closure require 3.3, and the last release allowing 3.2 is eleven minor versions behind the one -phase 8c verified against. **A consumer on Ruby 3.2 composes `dexpace-core`, +phase 8c verified against (built and proven on 0.105.0, 2026-09-21, with the floor read from one +`VERSIONS` row — `ruby floor:dexpace-transport-async_http 3.3` — by the gemspec, the gates, the +`Gemfile` and the two rake tasks that load the gem, so the 3.2 row installs, tests and clean-bundles +five gems and is green; on the 3.3 row the bundle compiles `openssl` 4.0.2, because the +interpreter's own 3.2.4 is older than `io-stream` requires — and a *networked* resolve, the +clean-bundle gate's scratch install included, picks the newest `openssl` gem on the 3.4 and 4.0 rows +too, as it picked `net-http` 0.9.1 over the default for 8a (`P8-60`), so the second extension's +toolchain need is confined to 3.3 only for an install that resolves the interpreter's own). **A consumer on Ruby 3.2 composes `dexpace-core`, `dexpace-transport-net_http`, `dexpace-serde-json`, `dexpace-async-thread` and `dexpace-conformance`, and loses only the reactor transport** — which is `NFR-2`'s separability paying for itself, and it should be stated in the release notes rather than discovered at `bundle install`. @@ -61,8 +68,14 @@ stated in the release notes rather than discovered at `bundle install`. the suite exists — twenty-eight assertions over twenty-two `TRANSPORT` IDs, `HTTP-17`/`HTTP-18` and `PAGE-36` — and 8a ran it against `dexpace-transport-net_http` on 3.2.11 (net-http 0.4.1 and 0.9.1), 3.3.12, 3.4.10 and 4.0.6, every assertion green and `TRANSPORT-18` vacuous by measurement; the - `async-http` half waits for 8c -- [ ] **Before release, `docs/sdk-documentation/` must state what a green `dexpace-conformance` run does + `async-http` half waits for 8c. **Status 2026-09-21**: 8c ran it against + `dexpace-transport-async_http` on 3.3.12, 3.4.10 and 4.0.6 — thirty-four assertions now, phase + 8c's two groups appended — with every assertion green and four skips accounted for by name: the + two assertions under `TRANSPORT-14` and the one under `TRANSPORT-27` waived by id (`P8-38`, + `protocol-http1` refuses both heads out of the read) and `TRANSPORT-18` vacuous by measurement + as on 8a's adapter; on the 3.2 row the async half is not installable and the sync half is what + runs, as this line already says +- [x] **Before release, `docs/sdk-documentation/` must state what a green `dexpace-conformance` run does and does not prove, and the run's own report preamble must name the same omissions.** Filed 2026-09-12 by phase 8a's design (`P8-9`). The wire fixture speaks plaintext only and exercises no connect timeout, so `TRANSPORT-4`'s open-timeout half and every TLS property are asserted in @@ -75,7 +88,10 @@ stated in the release notes rather than discovered at `bundle install`. badge. Cites `TRANSPORT-4`, `TRANSPORT-14`, `TRANSPORT-20`, `NFR-2`; the suite itself is phase 8a's Tasks 4–8 and 20 and phase 9's Tasks 2–12a. **Status 2026-09-20**: `TransportSuite::PREAMBLE` names the two omissions in every report, and `docs/sdk-documentation/conformance.md` states them - beside what a green run proves; the box ticks when 8c's waiver is stated beside them + beside what a green run proves; the box ticks when 8c's waiver is stated beside them. **Ticked + 2026-09-21**: the preamble names a third omission, `TRANSPORT-8`, whose antecedent only an + adapter's own suite can originate, and `conformance.md` states 8c's two waivers (`TRANSPORT-14` + and `TRANSPORT-27`, both by id, both this adapter's alone) beside the three omissions - [ ] **Before release, `docs/sdk-documentation/` must document the `include Dexpace` constant-shadow hazard.** Filed 2026-09-08 by phase 3a's design; recorded here 2026-09-13. Phase 1 measured `Dexpace::Method` shadowing `::Method` and concluded "**verified inert outside core**"; the observation @@ -130,7 +146,13 @@ stated in the release notes rather than discovered at `bundle install`. time** — `Clients::MAX_ORIGINS`, drained back to the cap in a loop after each insert, with each evicted client's pool closed — so the map is expected to be bounded the moment `8c` lands and this blocker to close then, without a phase-10 repair. Phase 9's routing assumed `8c` had - already run; it had not. This line stays open until a green `gates:bounded_map` run confirms it + already run; it had not. This line stays open until a green `gates:bounded_map` run confirms it. + **Status 2026-09-21**: 8c landed the bound — `Clients::MAX_ORIGINS` (32) over a key of + (reactor, origin), drained back to the cap in a loop after every insert with closed reactors + evicted first, every evicted client's pool retired and closed — proven by the three `XCUT-14` + cases in `gems/dexpace-transport-async_http/test/dexpace/transport/async_http/clients_test.rb`; + the gate itself does not exist yet, so the line stays open for phase 9's `gates:bounded_map` run + and nothing else - [ ] **Before release, `docs/sdk-documentation/` carries one worked end-to-end example** — a generated-style client over `dexpace-core` + `dexpace-transport-net_http` + `dexpace-serde-json`: operation descriptor, request assembly, pipeline with an AUTH step, decode, typed error, one diff --git a/docs/knowledge/notes/concurrency-and-async.md b/docs/knowledge/notes/concurrency-and-async.md index b1107c4..914860d 100644 --- a/docs/knowledge/notes/concurrency-and-async.md +++ b/docs/knowledge/notes/concurrency-and-async.md @@ -58,3 +58,32 @@ entry's stable key. SDK two answers to one question, and `close_quietly`'s contract is phase 2's. Cites `SEAM-30`, `ASYNC-5`, `ASYNC-6`, `TRANSPORT-7`, `TRANSPORT-9`, `TRANSPORT-22`, `CFG-21`, `XCUT-13`. review · `docs/work/mvp/phase8/2026-09-11-phase8-segmentation-design.md` · high · sha:manual-phase8-async-cancel-not-standarderror +- **`Async::Task#cancel` cannot be called from another OS thread, keeps only an `Exception` as its + `cause:`, and on `async` 2.46 `Kernel#Async` inside a task is that task's child.** Beside this file's + `## Superseded` entry on `#cancel`, which stands, and beside `concurrency-and-async/f4beb429` + (`ASYNC-22`'s "safe for concurrent calls from multiple threads"); three facts execution measured on + `async` 2.46.0 under Ruby 3.3.12, 3.4.10 and 4.0.6 that phase 8c's design stated otherwise or not at all. + **One.** `Task#cancel` on a task whose fiber is not the current one calls `Fiber.scheduler.raise`, and + `Fiber.scheduler` is nil on every OS thread but the reactor's, so a cancellation hook that reaches the + task directly from the canceller's thread raises `NoMethodError: private method 'raise' called for nil` + on that thread and leaves the task running (`:running` afterwards, measured). A cancellation that may + originate on any thread — a `Dexpace::Cancellation::Source#cancel`, which the conformance suite fires + from a `Thread.new` — therefore has to be marshalled into the reactor: phase 8c pushes the reason onto a + `Thread::Queue` that a transient watcher task inside the reactor pops (a scheduler-aware wait, wakeable + from any thread) and the watcher cancels the exchange on the reactor's own thread. **Two.** + `Task#cancel(cause:)` keeps the cause only when it is an `Exception`; anything else — the design's + fact 8 passed a Symbol — is replaced by the runtime's own `Async::Cancel::Cause` ("Cancelling task!"), + so a reason travels as an exception (`Dexpace::CancelledError.new(reason)`) if the cancelled task's + own `$!.cause` is to name it for whoever reads the task; phase 8c's adapter reads no cause back — + its pivot is settled with the reason before the watcher cancels the task — so the wrap is for the + task tree and a debugger, not a channel the SDK relies on. And both of the adapter's hooks may run + AFTER the exchange has ended: `Cancellation::Source#cancel` and `Async::Completer#settle` each + steal their hook list under their mutex and run it outside, so a push onto the exchange's queue + has to be total over the queue's close (`ClosedQueueError` rescued at the push), or the raise + travels back through `Hooks.notify` into the caller's `Source#cancel` on the cancelling thread — + reproduced deterministically with an ordinary caller hook registered first. **Three.** `Kernel#Async` + inside a running task delegates to `Task.current.async`, so the spawned task **is** the current + task's child (`inner.parent.equal?(task)` measured true); the design's fact 10 ("`Async { }` inside a + reactor is not a child of the caller") does not hold on 2.46.0, and the adapter's `caller_task.async` + spelling is the honest one rather than a distinction the runtime still draws. Cites `ASYNC-6`, `ASYNC-22`, `TRANSPORT-7`, `TRANSPORT-8`. + review · `docs/work/mvp/phase8/phase8c/2026-09-11-phase8c-asynchronous-transport-checklist.md` · high · sha:manual-phase8c-cancel-across-threads diff --git a/docs/knowledge/notes/transport-adapter.md b/docs/knowledge/notes/transport-adapter.md index 0c81699..d24ba90 100644 --- a/docs/knowledge/notes/transport-adapter.md +++ b/docs/knowledge/notes/transport-adapter.md @@ -146,3 +146,36 @@ stable key. the first user to benchmark this SDK against `faraday` will find the handshake and should find it already written down. Cites `TRANSPORT-5`, `TRANSPORT-29`, `SEAM-12`, `NFR-2`, `XCUT-11`. review · `docs/work/mvp/phase8/phase8a/2026-09-11-phase8a-synchronous-transport-and-conformance-design.md` · high · sha:manual-phase8a-connection-per-request +- **Under `async-http`, a response does not outlive the reactor that produced it, and the adapter's two + as-built closes are shaped by what the library does at a reactor's exit and at an HTTP/2 body's release.** + Beside `transport-adapter/82a365d4` (`TRANSPORT-15`'s ownership-aware close) and this file's `cb7901ef` + entry, which stand; what is added are three facts execution measured on `async` 2.46.0, `async-http` + 0.105.0 and `async-pool` 0.12.0 under Ruby 3.3.12, 3.4.10 and 4.0.6 that the design's thirteen facts do + not carry. **One.** `Sync { }` returns only when the reactor has no non-transient work left, and on its + way out it cancels the transient tasks — among them the connection pool's gardener, whose `ensure` calls + `Async::Pool::Controller#close`, which is `drain` and `drain` is "acquire every existing resource with + zero usage, waiting on the condition while any is busy". A connection whose response body is unread is + busy, so a `Sync` that hands a streaming response out past its own end **never returns**: found by + `TRANSPORT-29`'s conformance assertion, whose eight OS threads each opened a reactor per settle and read + the body afterwards. The adapter cannot change it and should not — it is the library's ownership model — + so the rule is a caller's: read or close the response inside the block that made the call, and a driver + that settles on a thread of its own materialises the body inside its reactor before handing the response + out (the 8c conformance driver does, through `Response#body_bytes` into a `BufferBody`). **Two.** + `Adapter#close` retires every pooled connection through `Async::Pool::Controller#retire` **before** + `pool.close` (`P8-37` as built): `retire` deletes the resource and closes it without waiting, so a close + under an open response returns at once and the open read fails, whereas `pool.close` alone would sit in + the same drain as fact one, and `Client#close` would wait too and write a Console warning to stderr. + **Three.** Closing an HTTP/2 response body that has not been read — `Protocol::HTTP2::Input#close` — + releases the pooled connection twice inside the library on every row (`RuntimeError: Trying to reuse + unacquired resource` out of `async-pool`'s `decrement_usage`): `async-http` 0.105.0 writes the + `RST_STREAM` frame before it transitions the stream's state, so a peer's `END_STREAM` landing during + that write closes the stream a second time — five of five through a response obtained in a child task + and closed unread by its parent — so the adapter's `ResponseBody#release` closes the native body through + `Dexpace.close_quietly` with the factory's `logger:` rather than letting the library's raise escape a + caller's `#close`: the connection is retired either way and the client stays sound, measured by a + sixteen-request round trip through the same client afterwards. And a row-bound fact beside them: on + Ruby 4.0.6 alone, the first `IO::Buffer` the scheduler allocates under a fiber scheduler prints Ruby's + once-per-process "IO::Buffer is experimental" warning to stderr, which a warnings-fatal suite must spend + before its first test (`test/support/async_http_warmup.rb`), as 8a parks net-http's Timeout thread. + Cites `TRANSPORT-15`, `TRANSPORT-16`, `TRANSPORT-19`, `TRANSPORT-25`, `TRANSPORT-29`, `XCUT-13`. + review · `docs/work/mvp/phase8/phase8c/2026-09-11-phase8c-asynchronous-transport-checklist.md` · high · sha:manual-phase8c-reactor-exit-and-h2-release diff --git a/docs/sdk-documentation/architecture.md b/docs/sdk-documentation/architecture.md index e4cf776..f2f9ceb 100644 --- a/docs/sdk-documentation/architecture.md +++ b/docs/sdk-documentation/architecture.md @@ -12,10 +12,11 @@ constructors, phase 7b the server-sent-events layer with the serde-boundary gate pagination layer, phase 7a the serialization layer with the JSON codec, phase 8a the first two adapter gems beside 7a's — the synchronous transport in `dexpace-transport-net_http` and the conformance suite in `dexpace-conformance` — and phase 8b the async-runtime adapter in -`dexpace-async-thread`; the asynchronous transport is still to come, so this page -is still a stub: it names the +`dexpace-async-thread` and phase 8c the asynchronous transport in +`dexpace-transport-async_http`, the suite's second driver; the remaining adapters are still to come, +so this page is still a stub: it names the pages this tree will eventually hold and where each one's content will come from, so the plan for -the documentation exists before the documentation does. Twenty pages are real already, because +the documentation exists before the documentation does. Twenty-one pages are real already, because their subjects are: [`quality-gates.md`](./quality-gates.md), [`http.md`](./http.md), [`seams.md`](./seams.md), [`io.md`](./io.md), [`body.md`](./body.md), [`execution-context.md`](./execution-context.md), [`recovery.md`](./recovery.md), @@ -24,8 +25,8 @@ their subjects are: [`quality-gates.md`](./quality-gates.md), [`http.md`](./http [`logging-and-redaction.md`](./logging-and-redaction.md), [`retry.md`](./retry.md), [`auth.md`](./auth.md), [`redirect.md`](./redirect.md), [`sse.md`](./sse.md), [`pagination.md`](./pagination.md), [`serde.md`](./serde.md), -[`transport-net_http.md`](./transport-net_http.md), [`conformance.md`](./conformance.md) and -[`async-thread.md`](./async-thread.md). +[`transport-net_http.md`](./transport-net_http.md), [`conformance.md`](./conformance.md), +[`async-thread.md`](./async-thread.md) and [`transport-async_http.md`](./transport-async_http.md). Once `dexpace-core` and the first adapters ship, this page becomes the same kind of front door the sibling Node SDK's `docs/sdk-documentation/architecture.md` is — package by package, seam by seam — and the entries below turn from plain text into real links, one at a time, as each page is @@ -194,13 +195,24 @@ does not do. Written against the adapter phase 8a shipped; derives from `docs/sdk-design-ruby/03-seam-by-seam-idiomatic-mapping.md` §3.2, read together with entries 10 and 15 of `docs/sdk-design-ruby/10-deliberate-deviations-from-the-reference-contract.md`. -[conformance.md](./conformance.md) — the conformance suite: what a green run proves and the eight +[transport-async_http.md](./transport-async_http.md) — the asynchronous transport: the reactor +every call needs and the failed future a call outside one gets, the two constructions and the +reactor-keyed client map behind the owning one, what the wire carries on HTTP/1.1 and HTTP/2 and +the one token predicate applied to both, the exchange task and its one budget, the cancellation +bridge that works from any thread and what a cancellation interrupts, the pull-shaped response +body and the reactor it must not outlive, the classifier, the lenient inbound mapping and the two +clauses it waives, TLS and HTTP/2 by ALPN, and what the gem deliberately does not do. Written +against the adapter phase 8c shipped; derives from +`docs/sdk-design-ruby/03-seam-by-seam-idiomatic-mapping.md` §3.3, read together with entries 10 +and 15 of `docs/sdk-design-ruby/10-deliberate-deviations-from-the-reference-contract.md`. + +[conformance.md](./conformance.md) — the conformance suite: what a green run proves and the two `TRANSPORT` IDs it deliberately does not carry, the assertion protocol and its five statuses, the runner and its report with vacuous and waived rows that stay visible, the two thin drivers and the three keywords that are the asynchronous adapter's contract, the case an assertion receives and the guard on its transport, the plaintext wire fixture with its fifteen scripts and bounded waits, -and the two observability doubles. Written against the gem phase 8a shipped; derives from -`docs/sdk-design-ruby/09-toolchain-and-quality-gates.md` §9.3. +and the two observability doubles. Written against the gem phase 8a shipped and the two groups +phase 8c appended; derives from `docs/sdk-design-ruby/09-toolchain-and-quality-gates.md` §9.3. [async-thread.md](./async-thread.md) — the async-runtime adapter: the fixed-size thread pool over a bounded queue and the one required keyword, `#post` as the executor duck type that never @@ -220,8 +232,8 @@ and how to run it locally. Written against the build phase 0 shipped; derives fr write-a-transport.md — implementing the `Transport` seam, and proving an implementation against `dexpace-conformance`. Derives from `docs/sdk-design-ruby/03-seam-by-seam-idiomatic-mapping.md`; the seam's contract itself is already on [seams.md](./seams.md), the proving half on -[conformance.md](./conformance.md), and the reference implementation on -[transport-net_http.md](./transport-net_http.md). +[conformance.md](./conformance.md), and the two reference implementations on +[transport-net_http.md](./transport-net_http.md) and [transport-async_http.md](./transport-async_http.md). write-a-serde.md — implementing the `Serde` seam: the serializer/deserializer pair, the `Tristate` PATCH convention, and the four encode profiles. Derives from diff --git a/docs/sdk-documentation/conformance.md b/docs/sdk-documentation/conformance.md index d276e46..ad147ff 100644 --- a/docs/sdk-documentation/conformance.md +++ b/docs/sdk-documentation/conformance.md @@ -1,8 +1,9 @@ # The conformance suite: `dexpace-conformance` -**As built by phase 8a, written against source on 2026-09-20.** This page says what the conformance gem +**As built by phase 8a, written against source on 2026-09-20; the counts, the preamble and the two +groups phase 8c appended re-run against source on 2026-09-21.** This page says what the conformance gem gives a transport-adapter author today: the assertion protocol phase 0 postponed — `Failure`, `Vacuous`, -`Assertion`, `Result` and `Report` — the twenty-eight-assertion `TransportSuite` every adapter is proven +`Assertion`, `Result` and `Report` — the thirty-four-assertion `TransportSuite` every adapter is proven against, the `TransportCase` an assertion receives and the eleven-clause contract that keeps the suite free of any adapter's name, the plaintext `WireServer` fixture and its fifteen `Scripts`, the two thin drivers for Minitest and RSpec, and the two observability doubles phases 5b and 5c assigned here, @@ -12,7 +13,8 @@ maps it to Ruby is `docs/sdk-design-ruby/09-toolchain-and-quality-gates.md` §9. proof is `docs/work/mvp/phase8/phase8a/2026-09-11-phase8a-synchronous-transport-and-conformance-checklist.md`. Signatures live in `gems/dexpace-conformance/sig/`, and this page does not restate them. Every example below was run against the built code on 4.0.6 and 3.2.11 and printed the same on both (the one -difference is Ruby 3.4's `Hash#inspect` spelling, so the examples print arrays and strings). The +difference is Ruby 3.4's `Hash#inspect` spelling, so the examples print arrays and strings), and the +ones whose printed values phase 8c's groups changed were re-run on 4.0.6 and 3.3.12. The examples drive the suite against the reference adapter, `Dexpace::Transport::NetHTTP`, because a conformance page's examples should run the real thing; `C` is `Dexpace::Conformance` and `NetHTTP` is `Dexpace::Transport::NetHTTP` throughout. @@ -26,48 +28,69 @@ gem alone. ## What a green run proves, and what it does not -Read this before the numbers. The suite is **twenty-eight assertions over twenty-six requirement IDs**: -twenty-two `TRANSPORT` IDs, `HTTP-17` and `HTTP-18` with `XCUT-18` (the wire-boundary re-validation), and +Read this before the numbers. The suite is **thirty-four assertions over thirty-two requirement IDs**: +twenty-eight `TRANSPORT` IDs, `HTTP-17` and `HTTP-18` with `XCUT-18` (the wire-boundary re-validation), and `PAGE-36` (a closed transport is a `ClosedError`, never a hang). It is deliberately **not** all thirty -`TRANSPORT` IDs, and the eight it does not carry are stated rather than left to be discovered: - -- **`TRANSPORT-7`, `-8`, `-9`, `-21` and `-23`** are the async path's — a cancelled future, a native - cancellation, the adaptation race, a pre-dispatch failure and a nil-response success — and there is - no future on the synchronous seam. They are 8c's, in the async transport's own rows. -- **`TRANSPORT-12` and `TRANSPORT-13`** have no assertion, because the per-header drop they describe — a - header the SDK model admits and the native client's stricter wire grammar rejects — has no instance on - `net-http`, which was measured to accept every byte the outbound grammars admit, and a portable - assertion cannot say "vacuous here, real there" (P8-55). They are 8c's rows too. +`TRANSPORT` IDs, and the two it does not carry — and the three things it cannot prove — are stated +rather than left to be discovered: + +- **`TRANSPORT-8`** has no assertion: its antecedent is a cancellation the *native client* originates + while the SDK future is live, which only an adapter's own suite can raise by naming its runtime — a + parent `Async` task cancelled, for `dexpace-transport-async_http` — so it is that gem's row, proven + in its own suite, and `PREAMBLE` names it as the third thing a green run does not prove. - **`TRANSPORT-30`**, the proxy limitation, is the reference adapter's own and lives in `gems/dexpace-transport-net_http/test/`, because the fixture reads no configuration and is no proxy. -- **`TRANSPORT-4`'s connect-timeout half and TLS verification** are the two things `PREAMBLE` names at - the head of every report (P8-9): the fixture accepts every connection at once, so the open timeout is - never exercised, and it has no certificate. An adapter that passes this suite has proven neither, and a - third-party author who reads the report is told so. +- **`TRANSPORT-4`'s connect-timeout half and TLS verification** are the two things `PREAMBLE` has + named at the head of every report since 8a (P8-9): the fixture accepts every connection at once, so the + open timeout is never exercised, and it has no certificate. An adapter that passes this suite has + proven neither, and a third-party author who reads the report is told so. +- **`TRANSPORT-12` and `TRANSPORT-13`** resolve **vacuous by measurement** on an adapter whose native + client accepts every model-valid header name — the bad name is sent and the wire is read, which is + what `net-http` does — and are real on one whose wire grammar is stricter than the SDK model's, which + is `async-http`'s (P8-55 is why they could not be a shared assertion declaring itself vacuous). +- **`TRANSPORT-14`'s malformed-inbound-name clause and `TRANSPORT-27`'s invalid-`Content-Length` + clause** are **waived by id** in `dexpace-transport-async_http`'s driver and by nothing in + `dexpace-transport-net_http`'s: `protocol-http1` refuses both heads out of the read before a response + exists to adapt (P8-38), `Net::HTTP` delivers both. A waiver is reported on every run, and the two + assertions under `TRANSPORT-14` are both skipped by the one id. ```ruby require "dexpace/conformance" -C::TransportSuite.assertions.size # => 28 -C::TransportSuite.assertions.flat_map(&:ids).uniq.size # => 26 -C::TransportSuite.assertions.flat_map(&:ids).grep(/TRANSPORT/).uniq.size # => 22 +C::TransportSuite.assertions.size # => 34 +C::TransportSuite.assertions.flat_map(&:ids).uniq.size # => 32 +C::TransportSuite.assertions.flat_map(&:ids).grep(/TRANSPORT/).uniq.size # => 28 C::TransportSuite.assertions.first(3).map(&:name) # => ["the caller's explicit Content-Type wins over the body's", # "a body-derived Content-Type is used only when the caller set none", # "no explicit header and no body media type is never a form type"] +C::TransportSuite.assertions.last(6).map(&:ids) +# => [["TRANSPORT-7"], ["TRANSPORT-9"], ["TRANSPORT-21"], ["TRANSPORT-23"], ["TRANSPORT-12"], ["TRANSPORT-13"]] puts C::TransportSuite::PREAMBLE # dexpace-conformance transport suite: the wire fixture speaks plaintext only and exercises no # connect timeout, so TLS verification and TRANSPORT-4's open-timeout classification are NOT among -# the things a green run proves (P8-9); assert both in the adapter's own suite. +# the things a green run proves (P8-9); assert both in the adapter's own suite. Nor is TRANSPORT-8: +# a cancellation the native client originates while the SDK future is live can only be raised by +# naming the adapter's own runtime, so that pair -- terminal on the cancellation, retryable on a +# timeout of the same path -- is the adapter's own suite's too. ``` -The twenty-eight are five groups in the order a reader meets chapter 17: **outbound** (seven — +The thirty-four are seven groups in the order a reader meets chapter 17: **outbound** (seven — `TRANSPORT-10`'s `Content-Type` precedence, `TRANSPORT-26`'s body-less `POST`, `TRANSPORT-11`'s managed headers, the two forged-header refusals), **inbound** (four — `TRANSPORT-24`'s vendor status, `TRANSPORT-14`'s malformed header drop, `TRANSPORT-27`'s malformed `Content-Length`), **streaming** (three — `TRANSPORT-25`'s large body, `TRANSPORT-19`'s prompt release, `TRANSPORT-28`'s file window), -**resilience** (eight — `TRANSPORT-1` through `-4`, `-17`, `-18`, `-20`, `-22`) and **lifecycle** (six — -`TRANSPORT-5`'s budget, `TRANSPORT-6`'s clamp, `TRANSPORT-15`/`-16`'s close, `TRANSPORT-29`, `PAGE-36`). +**resilience** (eight — `TRANSPORT-1` through `-4`, `-17`, `-18`, `-20`, `-22`), **lifecycle** (six — +`TRANSPORT-5`'s budget, `TRANSPORT-6`'s clamp, `TRANSPORT-15`/`-16`'s close, `TRANSPORT-29`, `PAGE-36`), +and phase 8c's two — **asynchronous** (four — `TRANSPORT-7`'s mid-body cancel through the token with +the connection released, `TRANSPORT-9`'s response arriving after the cancel and never delivered, +`TRANSPORT-21`'s adaptation failure classified through the send primitive's failure channel, +`TRANSPORT-23`'s success always a `Dexpace::Response`) and **header drops** (two — `TRANSPORT-12`'s +model-valid non-token name dropped with the rest dispatched, `TRANSPORT-13`'s once-per-name, bounded +drop reporting read off the transport's own `logger:`). Every one of the six is written against the +suite contract's primitives — `kase.settle`, `kase.wire`, `kase.transport(logger:)` and +`Dexpace::Cancellation` — and names no reactor, no task and no native class, which is what lets 8a's +synchronous driver run them unchanged. ## Running it, and reading a report @@ -91,16 +114,23 @@ borrow = lambda do |port| end report = C::TransportSuite.run(build: ->(**settings) { NetHTTP.build(**settings) }, borrow: borrow) report.passed? # => true -report.results.map(&:status).tally.to_a # => [[:passed, 27], [:vacuous, 1]] -report.vacuous.map { |r| r.assertion.ids } # => [["TRANSPORT-18"]] +report.results.map(&:status).tally.to_a # => [[:passed, 31], [:vacuous, 3]] +report.vacuous.map { |r| r.assertion.ids } # => [["TRANSPORT-18"], ["TRANSPORT-12"], ["TRANSPORT-13"]] puts report # dexpace-conformance transport suite: the wire fixture speaks plaintext only and exercises no # connect timeout, so TLS verification and TRANSPORT-4's open-timeout classification are NOT among -# the things a green run proves (P8-9); assert both in the adapter's own suite. -# 27 passed, 0 failed, 1 vacuous, 0 waived, 0 errored +# the things a green run proves (P8-9); assert both in the adapter's own suite. Nor is TRANSPORT-8: +# a cancellation the native client originates while the SDK future is live can only be raised by +# naming the adapter's own runtime, so that pair -- terminal on the cancellation, retryable on a +# timeout of the same path -- is the adapter's own suite's too. +# 31 passed, 0 failed, 3 vacuous, 0 waived, 0 errored # vacuous: TRANSPORT-18: the native client opened one connection and pulled the single-use body # once across a dropped first attempt, so no re-subscribable producer is in play on this adapter # (TRANSPORT-18's antecedent is absent) +# vacuous: TRANSPORT-12: the native client accepted the model-valid name X-Bad:Name and sent it, so +# it rejects no header the SDK model admits (TRANSPORT-12's antecedent is absent) +# vacuous: TRANSPORT-13: the native client accepted the model-valid names and sent them, so there +# is no drop to log (TRANSPORT-13's antecedent is absent) ``` **Vacuous is a status, not a pass.** `TRANSPORT-18` says a native retry must not re-subscribe a @@ -116,7 +146,7 @@ every run — design §9.3's "the gap stays visible", for a port that has consci waived = C::TransportSuite.run(build: ->(**s) { NetHTTP.build(**s) }, borrow: borrow, waive: ["TRANSPORT-28"]) waived.waived.map { |r| r.assertion.ids } # => [["TRANSPORT-28"]] waived.to_s.lines.grep(/waived/) -# => ["26 passed, 0 failed, 1 vacuous, 1 waived, 0 errored\n", +# => ["30 passed, 0 failed, 3 vacuous, 1 waived, 0 errored\n", # " waived: TRANSPORT-28 (a file body with a non-zero position and partial count sends exactly that range)\n"] ``` @@ -175,13 +205,22 @@ calls `Dexpace::Conformance::RSpecDriver.conformance(...)`, so a Minitest-only c file that names the other framework. Neither driver names its framework in a signature (`NFR-11`). The three keywords a synchronous adapter never passes are the **suite contract** for an asynchronous -one, and the reason the same twenty-eight assertions will run unchanged against 8c's adapter: `settle:` +one, and the reason the same thirty-four assertions run unchanged against 8c's adapter: `settle:` (clause 8) is the one send primitive, `(transport, request, options, cancellation) -> Response`, which an async driver replaces with "call, then await the future"; `around:` (clause 9) wraps each assertion's whole invocation, which is where an async driver opens the reactor the body reads a streamed response inside; and `wire:` (clause 11) is the fixture factory, whatever answers `_Wire` — `#port`, `#requests`, `#connections`, `#closed_connections`, `#await_closed_connection`, `#close` — so an HTTP/2 server can -stand in for `WireServer` without this gem naming an async constant. +stand in for `WireServer` without this gem naming an async constant. `dexpace-transport-async_http`'s +driver is the second one built to it (`gems/dexpace-transport-async_http/test/dexpace/transport/async_http/conformance_test.rb`): +`settle:` awaits the future inside `around:`'s reactor — which runs the assertion as a child task and +bounds the parent's wait at thirty seconds, cancelling the child and flunking the row on expiry, so an +adapter that never releases what it holds fails the run instead of hanging it — and on a thread of +an assertion's own — +`TRANSPORT-5`'s pair and `TRANSPORT-29`'s eight — opens a reactor per settle and reads the body inside +it before handing the response out, because under `async-http` a response cannot outlive the reactor +that produced it (`docs/sdk-documentation/transport-async_http.md`). Its run is four skips — the three +waived assertions and `TRANSPORT-18` — and every other assertion green. ## The case an assertion receives diff --git a/docs/sdk-documentation/transport-async_http.md b/docs/sdk-documentation/transport-async_http.md new file mode 100644 index 0000000..3b9670d --- /dev/null +++ b/docs/sdk-documentation/transport-async_http.md @@ -0,0 +1,456 @@ +# The asynchronous transport: `dexpace-transport-async_http` + +**As built by phase 8c, written against source on 2026-09-21.** This page says what the reference +asynchronous transport gives an SDK author today: `Dexpace::Transport::AsyncHTTP`, a module with two +constructions over `async-http`, the reactor every call needs and the failed future a call outside +one gets, a client map keyed by reactor and origin, the header policy the wire carries on HTTP/1.1 +and HTTP/2 alike, one exchange task per call under one budget, a cancellation bridge that works from +any thread, a response body that streams on the fiber that reads it, and the failure classification +that keeps phase 6a's retry layer honest. What each is *required* to do is +`docs/product-spec/17-transport-adapter-conformance-contract.md` (`TRANSPORT-1`–`TRANSPORT-30`, of +which this adapter owns seven — `TRANSPORT-7`, `-8`, `-9`, `-12`, `-13`, `-21`, `-23` — and re-proves +8a's as the suite's second driver) with `ASYNC-6`, `ASYNC-21` and `ASYNC-22` from +`docs/product-spec/18-asynchronous-runtime-adapter-contract.md`; how the design maps it to Ruby is +`docs/sdk-design-ruby/03-seam-by-seam-idiomatic-mapping.md` §3.3 read with entries 10 and 15 of +`docs/sdk-design-ruby/10-deliberate-deviations-from-the-reference-contract.md`; the per-requirement +proof is `docs/work/mvp/phase8/phase8c/2026-09-11-phase8c-asynchronous-transport-checklist.md`. +Signatures live in `gems/dexpace-transport-async_http/sig/`, and this page does not restate them. +Every example below was run, in the order printed and as one script, against the built code on +4.0.6 and 3.3.12 (async-http 0.105.0, async 2.46.0 on both) and printed the same on both, except +for the ephemeral port a `Host` line names, which is the run's own. The examples use +`dexpace-conformance`'s `WireServer` and `Scripts` as the server — a real `TCPServer` on `127.0.0.1` +answering a scripted reply and recording what it was sent — because a transport page's examples +should touch a socket and nothing outside the machine. Eight names are the page's shorthand and +nothing else is assumed: `AsyncHTTP` is `Dexpace::Transport::AsyncHTTP`; `WireServer` and +`Scripts` are `Dexpace::Conformance::WireServer` and `Dexpace::Conformance::Scripts`, after +`require "dexpace/conformance"`; `req(url, method: "GET", headers: Dexpace::Headers::EMPTY, +body: nil)` is `Dexpace::Request.build` over those four; `headers(pairs)` is +`Dexpace::Headers.builder` with each pair added; `EMPTY` is `Dexpace::RequestOptions::EMPTY`; +`sink` is a recording sink — any object answering the facade's `#debug`/`#info`/`#warn`/`#error` +and their four predicates and keeping every `(severity, payload)` it is handed; and `drops` is the +pairs that sink recorded whose payload's `"event"` is `Events::TRANSPORT_HEADER_DROPPED`, in order. +`adapter` is the owning adapter the first block builds and the last block closes, and every block +that talks to a socket starts its own `server` and reads its port for `url`. + +**The gem's whole dependency budget is `dexpace-core` and `async-http ~> 0.104`** (`NFR-2`), whose +closure brings `async`, `async-pool`, `protocol-http`, `protocol-http1`, `protocol-http2`, `io-event` +(a native extension) and `openssl`; the gem declares **Ruby >= 3.3**, narrower than the SDK's 3.2 +floor, because that closure does (`P8-36`), so a consumer on 3.2 composes `dexpace-core` with +`dexpace-transport-net_http` and loses only this gem. Requiring the gem registers it with phase 2's +registry under `:async_http`, so a consumer that requires it and nothing else resolves an adapter +without naming one (`SEAM-5`). + +```ruby +require "dexpace/transport/async_http" + +adapter = AsyncHTTP.build +adapter.owned? # => true +Dexpace::AsyncTransport.conforms?(adapter) # => true +Dexpace::AsyncTransport.registered_keys # => [:async_http] +Dexpace::AsyncTransport.resolve.class # => Dexpace::Transport::AsyncHTTP::Adapter +AsyncHTTP.default.equal?(AsyncHTTP.default) # => false (a fresh adapter every call) +AsyncHTTP::FRAMING_HEADERS +# => ["host", "content-length", "transfer-encoding", "connection", "keep-alive", "proxy-connection", +# "te", "trailer", "upgrade", "expect"] +AsyncHTTP::ALPN_PROTOCOLS # => ["h2", "http/1.1"] +[AsyncHTTP::DEFAULT_TIMEOUT_SECONDS, AsyncHTTP::DEFAULT_CONNECTION_LIMIT, AsyncHTTP::MAX_ORIGINS] +# => [60.0, 8, 32] +AsyncHTTP::DropPolicy::MODES # => [:every, :once_per_name, :quiet] +AsyncHTTP::REGISTRY_KEY # => :async_http +``` + +## A reactor, and what a call needs from it + +**Every call needs a running `Async` reactor on the calling thread**, and the adapter creates none +(`P8-39`): a `Sync { }` inside the adapter would block the caller's thread and defeat the seam, and a +reactor thread the adapter owned would be a thread pool by another name — `dexpace-async-thread`'s +job, and a long-lived fiber carrying the constructor's diagnostic context rather than the caller's +(`ASYNC-10`'s exact failure). `Adapter#call` outside a reactor does not raise. It returns an +**already-failed** future carrying `Dexpace::SeamError` whose message names the fix, which is what +`TRANSPORT-21` asks for in as many words: delivered through the returned future, never thrown +synchronously. + +```ruby +future = adapter.call(req("http://127.0.0.1:1/"), EMPTY, nil) # no Sync, no Async +future.settled? # => true +future.value +# raises Dexpace::SeamError: Dexpace::Transport::AsyncHTTP requires a running Async reactor on the +# calling thread; wrap the call in `Sync { }` or `Async { }` +Async::Task.current? # => nil +``` + +Three consequences of the reactor being the caller's, and not the adapter's: + +- **A response does not outlive the reactor that produced it.** `Sync { }` returns only when the + reactor has no non-transient work left, and on its way out it stops the connection pool's own + housekeeping task, whose release drains the pool — a wait on every busy connection, and the + connection behind an unread body is busy until that body is read or closed. So read or close the + response **inside** the block that made the call; a `Sync` that hands a streaming response out to + code after it never returns. `Response#body_string`, `#body_bytes` and `#close` all release the + connection. The 8c conformance driver met exactly this in `TRANSPORT-29`'s eight-thread assertion + and materialises the body inside its per-thread reactor before handing the response out. +- **`Dexpace::AsyncTransport.sync_over(adapter)` still needs a reactor.** The bridge awaits the future + this adapter returns, and the exchange behind it cannot run without a reactor, so a caller with no + reactor gets the same `SeamError` through the bridge as through `#call`. +- **`Dexpace::Transport.async_over(adapter, executor:)` accepts this adapter silently** and yields a + future of a future; the return-type check that would refuse it is phase 2's open finding in + `dexpace-core`, not this gem's to fix. Use the adapter as the `Dexpace::AsyncTransport` it is. + +## Two constructions, and a client map keyed by reactor and origin + +`AsyncHTTP.build(timeout: nil, logger: Instrumentation::Logger::NULL, drop_policy: nil, +connection_limit: nil, ssl_context: nil, configuration: nil)` is the SDK-managed construction: the +adapter **owns** a map of `Async::HTTP::Client`s, one per **(reactor, origin)**, built on first use +and released by `#close`. The reactor is part of the key because an `async-http` client belongs to +the reactor whose tasks drive it and cannot be shared across reactors on different threads; keying +by `Fiber.scheduler` identity is what makes `ASYNC-22`'s "safe for concurrent calls from multiple +threads" true structurally — every thread running its own reactor gets its own client per origin, +and every fiber inside one reactor shares one multiplexed client. The map is bounded at +`MAX_ORIGINS` (32) and drained back to it after every insert, evicting clients whose reactor has +closed first and the oldest after that, with every evicted client's pool retired and closed rather +than dropped (`XCUT-14`). Each client is built with `retries: 0` — `async-http` would otherwise +re-send an idempotent request on a dropped connection, which is `TRANSPORT-2`'s prohibition and +`TRANSPORT-17`'s single-use body written twice — and with `limit:` from `connection_limit:`, the +configuration key `TRANSPORT_CONNECTION_LIMIT`, or `DEFAULT_CONNECTION_LIMIT` (8), the per-origin +connection bound on HTTP/1.1 (HTTP/2 multiplexes on one). + +`AsyncHTTP.using(client, logger:, drop_policy:)` is the **borrowing** construction over a caller's +own `Async::HTTP::Client`, used verbatim: the adapter never sets a knob on it, refuses one whose +`retries` is not already zero rather than setting it (`TRANSPORT-2`, `XCUT-22`), and its `#close` +releases nothing of the caller's, so the client stays usable afterwards (`TRANSPORT-15`). A borrowed +client is bound to one endpoint, so every request through it names that origin; and it is bound to +the reactor it was built inside. `Adapter.new` is private behind the two factories. `#close` on +either construction is reactor-free — it retires every pooled connection and closes each pool +without waiting for a busy one (`P8-37`), which is what `XCUT-13`'s non-blocking shutdown asks — and +it is `Dexpace::Closeable`'s idempotent latch, the only state written after construction: nothing +per call lives on the adapter (`TRANSPORT-29`, `ASYNC-22`). The residual is the caller's: a +streaming response still open when the adapter closes has had its connection retired under it, so +the next read that reaches the native body (one the source's own buffer cannot serve) fails as a +`Dexpace::StreamError` — non-retryable, a body that failed after its head (`P3-3`), the body closed +— whose `#cause` and message are the library's own artefact of a retired connection +(`the response body failed mid-stream: NoMethodError: undefined method 'read' for nil`, measured on +4.0.6 and 3.3.12), not a description of the close; read or close the response before closing the +adapter. + +## A call, and what the wire carries + +`#call(request, options, cancellation)` returns a `Dexpace::Async::Future` **before the head has +arrived**: the request is mapped and re-validated on the caller's fiber, and the exchange runs in a +child task of the caller's current task. What reaches the wire is the four-member `Request` and +nothing else (`HTTP-6`): no `User-Agent`, no `Accept-Encoding`, no `Accept` — `async-http` stamps +none, so there is nothing to delete, unlike `Net::HTTP`. A body-less `GET` goes out with +`content-length: 0`, because `async-http` writes it and suppressing it would mean reaching under the +body layer; `TRANSPORT-26` does not forbid it and `HTTP-7` is about the model, not the wire. + +```ruby +server = WireServer.start(Scripts.fixed("[]")) +Sync do + future = adapter.call(req("http://127.0.0.1:#{server.port}/pets?limit=2", + headers: headers("Accept" => "application/json", "X-Trace" => "abc")), + EMPTY, nil) + future.settled? # => false (returned before the head) + response = future.value + response.status.code # => 200 + response.protocol.wire # => "http/1.1" + response.headers["content-type"] # => ["text/plain"] + response.body.content_length # => 2 + response.body_string # => "[]" +end +sent = server.requests.last +[sent.request_line, sent.path] # => ["GET /pets?limit=2 HTTP/1.1", "/pets?limit=2"] +[sent.header("accept"), sent.header("x-trace")] # => ["application/json", "abc"] +sent.header("content-length") # => "0" +sent.header("host") # => "127.0.0.1:33599" (the fixture's port) +[sent.header("user-agent"), sent.header("accept-encoding")] # => [nil, nil] +``` + +Before dispatch the header set meets three gates, in order, and every one is applied to the +`Request` the caller handed over — the wire-boundary re-validation phase 1 postponed to the adapters +(`HTTP-17`, `HTTP-18`, `XCUT-18`) runs first, on every name and value, so a forged request that met +no builder is refused as `Dexpace::InvalidArgumentError` through the future before anything is +mapped. Then the **framing set**: the ten folded names in `FRAMING_HEADERS` are never copied and are +logged at verbose, because on this adapter a caller's `Host` or `Content-Length` would be **appended +beside** the library's own rather than replace it (`TRANSPORT-11`). Then the **token predicate**: a +name the RFC 7230 token grammar refuses — `HTTP-17` admits seventeen bytes the grammar does not, `:` +among them — is dropped and reported through the adapter's `logger:` under +`Events::TRANSPORT_HEADER_DROPPED`, **on both protocols** (`TRANSPORT-12`, `P8-40`): `protocol-http1` +would refuse the whole request after the request line is already on the socket, and +`protocol-http2` would transmit the name unvalidated, so one predicate before dispatch is what makes +one request produce one header set whichever protocol ALPN chose. The drop's reporting is +`DropPolicy`'s: the default `ONCE_PER_NAME` warns the first time a folded name is dropped and is +quiet (verbose) afterwards, over a latch bounded at `MAX_TRACKED_NAMES` (64) distinct names, beyond +which every drop is quiet (`TRANSPORT-13`, the policy phase 5b postponed to phase 8 as `OBS-19`); +`EVERY` and `QUIET` are the other two modes, and `DropPolicy.build(mode:)` refuses anything else. + +```ruby +server = WireServer.start(Scripts.fixed("ok")) +logged = AsyncHTTP.build(logger: Dexpace::Instrumentation::Logger.build(sink: sink)) # sink: any #debug/#info/#warn/#error object +Sync do + h = headers("Host" => "bogus.example", "Content-Length" => "999", "X-Bad:Name" => "v", "X-Normal" => "n") + logged.call(req("http://127.0.0.1:#{server.port}/", headers: h), EMPTY, nil).value.close + logged.call(req("http://127.0.0.1:#{server.port}/", headers: headers("X-Bad:Name" => "again")), EMPTY, nil).value.close +end +sent = server.requests.first +sent.header("host") # => "127.0.0.1:39165" (the library's own, never bogus.example) +sent.header("content-length") # => "0" +[sent.header("x-bad:name"), sent.header("x-normal")] # => [nil, "n"] +drops.map { |severity, payload| [severity, payload["header"], payload["reason"]] } # the sink's header_dropped records +# => [[:debug, "Host", "transport framing header (TRANSPORT-11)"], +# [:debug, "Content-Length", "transport framing header (TRANSPORT-11)"], +# [:warn, "X-Bad:Name", "not an RFC 7230 token (TRANSPORT-12)"], +# [:debug, "X-Bad:Name", "not an RFC 7230 token (TRANSPORT-12)"]] (once per name, then quiet) +``` + +`Content-Type` follows `TRANSPORT-10`'s precedence and **invents nothing**: the caller's explicit +header wins, then the body's own media type, and a body with neither goes out with no +`Content-Type` — `async-http` stamps none, where `Net::HTTP`'s form-urlencoded default is what +`dexpace-transport-net_http` pre-empts with `application/octet-stream`. The length is the body's own +`#content_length` when it is known, and a streaming body of unknown length goes out chunked. + +```ruby +server = WireServer.start(Scripts.sequenced("a", "b", "c")) +Sync do + json = Dexpace::Body.string("{}", media_type: Dexpace::MediaType.parse("application/json")) + adapter.call(req(url, method: "POST", body: json), EMPTY, nil).value.close + adapter.call(req(url, method: "POST", body: json, headers: headers("Content-Type" => "text/plain")), EMPTY, nil).value.close + adapter.call(req(url, method: "POST", body: Dexpace::Body.bytes("raw".b)), EMPTY, nil).value.close +end +server.requests.map { |r| r.header("content-type") } # => ["application/json", "text/plain", nil] +server.requests.map { |r| r.header("content-length") } # => ["2", "2", "3"] +``` + +## The response body streams, inside the reactor + +The head is adapted the moment `Async::HTTP::Client#call` returns, on the exchange task; the body is +the adapter's own `ResponseBody` over the native `Protocol::HTTP::Body::Readable` — lazy, +pull-shaped, one native `#read` per chunk and nothing read ahead of demand (`ASYNC-21`'s property, +though the ID is not this port's), every chunk retagged `BINARY`, closed through the body's own +`Closeable` latch on exhaustion, on an explicit close and on a mid-stream failure alike, so the +native body is closed exactly once whichever path took it. Reading suspends the reading fiber at the +scheduler and never a thread, which is the whole reason for the gem; `Response#close` releases the +connection to the pool at once, so the server observes the release promptly (`TRANSPORT-19`, +`TRANSPORT-25`). On HTTP/2 an unread body's close is routed through `Dexpace.close_quietly` with the +adapter's `logger:`, because `async-http` writes the stream reset before it transitions the +stream's state, so a peer's end-of-stream landing during that write releases the pooled connection +twice inside the library (measured five of five through a response obtained in a child task and +closed unread by its parent); the connection is retired either way, and the library's raise is +reported through the logger rather than reaching `Response#close`. + +```ruby +server = WireServer.start(Scripts.dribble("first-half", "second-half", 0.2)) # a 200 ms gap between the chunks +Sync do + response = adapter.call(req("http://127.0.0.1:#{server.port}/stream"), EMPTY, nil).value # the head, at once + response.body.content_length # => -1 (chunked: unknown length) + buffer = (+"").b + response.body.source.read_into(buffer, count: 5) # => 5 + buffer # => "first" + response.body_bytes # => "-halfsecond-half" (the rest, then closed) + response.body.closed? # => true +end +server.await_closed_connection(timeout: 2) # => 1 + +server = WireServer.start(Scripts.large(1024 * 1024, hold: true)) +Sync do + response = adapter.call(req("http://127.0.0.1:#{server.port}/big"), EMPTY, nil).value + response.body.source.read_into((+"").b, count: 16) + response.close # => nil, at once; the server sees the peer close + response.close # => nil (idempotent) +end +server.await_closed_connection(timeout: 2) # => 1 +``` + +A `204`, a `304` or a `HEAD`'s response has no body: `response.body` is `nil` and the native body, +if the library produced one, was closed by the mapper. + +## One budget, one exchange task + +The per-call budget is `RequestOptions#timeout`, then `.build(timeout:)`, then the configuration +key `REQUEST_TIMEOUT`, then `DEFAULT_TIMEOUT_SECONDS`; a bare number in `REQUEST_TIMEOUT` is +**milliseconds** (`CFG-7`), so thirty seconds is `30s` or `PT30S`. It is applied as one +`Async::Task#with_timeout` around the exchange — connect, write and the head — so two concurrent calls +with different budgets are each bounded by their own (`TRANSPORT-5`) and a near-zero budget is still a +bound and never "no timeout" (`TRANSPORT-6`). An expiry is `Async::TimeoutError`, a `StandardError`, +wrapped as a **retryable** `Dexpace::TransportError` with the original as `#cause` and the +cancellation token left exactly as it was found (`TRANSPORT-4`, `TRANSPORT-8`'s pair); the exchange +task and its watcher are gone from the reactor once the future has settled, so a long-lived reactor +accumulates nothing per failed call. + +```ruby +server = WireServer.start(Scripts.hang_before_headers) +Sync do + options = Dexpace::RequestOptions.builder.tap { |b| b.timeout = 0.2 }.build + adapter.call(req("http://127.0.0.1:#{server.port}/slow"), options, nil).value +rescue Dexpace::TransportError => error + [error.class, error.retryable?, error.phase] # => [Dexpace::TransportError, true, :connect] + error.cause.class # => Async::TimeoutError +end # in well under a second +Dexpace::Configuration::Keys::REQUEST_TIMEOUT # => "REQUEST_TIMEOUT" +Dexpace::Configuration::Keys::TRANSPORT_CONNECTION_LIMIT # => "TRANSPORT_CONNECTION_LIMIT" +Dexpace::Configuration.build(overrides: { "REQUEST_TIMEOUT" => "250" }).duration("REQUEST_TIMEOUT") +# => 0.25 (a bare number is milliseconds) +``` + +## Cancellation: from any thread, delivered at a checkpoint + +Three things cancel an exchange, and all three reach it: the caller's `Dexpace::Cancellation` +token, `Future#cancel` on the returned pivot (`ASYNC-6`'s two directions), and a cancellation the +runtime originates — a parent task cancelled, the reactor torn down — which arrives as +`Async::Cancel` inside the exchange task (`TRANSPORT-8`, satisfied on this adapter where the +specification records it vacuous). Every one of the first two is marshalled through a queue: the +token's hook settles the pivot cancelled and pushes its reason, and a transient **watcher** task +inside the reactor pops it and acts on the reactor's own thread — cancelling the exchange task while +it is in flight, or closing the delivered response afterwards, which is what wakes a consumer +blocked in a body read. The queue is the bridge because `Async::Task#cancel` cannot be called from +another OS thread (it raises there and cancels nothing), and the conformance suite cancels its token +from a `Thread.new` exactly as a host with a reactor per thread would. A cancel that arrives as the +exchange is finishing on its own is a no-op on the canceller's side: `Source#cancel` and +`Future#cancel` return normally whatever the exchange's state, because the adapter's hook is total +over the queue it pushes onto — the token still reads cancelled, and the future settles either the +cancellation or the response, never both. What surfaces is +`Dexpace::CancelledError` with the token's reason, the future reads `cancelled?`, it answers no +`retryable?` — a cancellation is terminal, and phase 6a's retry layer treats it so — and the +connection is released (`TRANSPORT-7`). A native response obtained after the pivot was cancelled is +closed, never delivered (`TRANSPORT-9`): the pivot itself closes what a settled completer is handed, +and the exchange checks the token again after every resume. + +```ruby +server = WireServer.start(Scripts.hang_before_headers) +Sync do + source = Dexpace::Cancellation.source + future = adapter.call(req("http://127.0.0.1:#{server.port}/held"), EMPTY, source.token) + Thread.new { source.cancel(:caller_gave_up) }.join # from another OS thread + begin + future.value + rescue Dexpace::CancelledError => error + [error.class, error.reason] # => [Dexpace::CancelledError, :caller_gave_up] + end + future.cancelled? # => true + error.respond_to?(:retryable?) # => false +end +server.await_closed_connection(timeout: 2) # => 1 +``` + +**`ASYNC-7`, this adapter's half of the contrast the design fixes** (§3.3): "the thread adapter +lets an in-flight blocking read finish, the reactor-backed ones abort at the next scheduler +checkpoint." The interrupt lands only where the exchange task is suspended — a socket read, a pool +wait — and `ensure` blocks run there, which is the property `Thread#raise` lacks and the reason it +is banned in this SDK. The adapter's own suite measures it: a blocked read is abandoned well under +the time the response would have taken to arrive. + +## Failures: the token first, then a retryable wrap + +Every failure with no response goes through one classifier, and the token is asked **first**: a +cancelled token turns any error into `Dexpace::CancelledError` — a bare `IOError` from a connection +the watcher retired and one of the SDK's own errors alike — because a cancel delivered by retiring +the connection and a peer reset arrive as the same `IOError` with the same message (`TRANSPORT-3`). +Then a `Dexpace::` error passes through unchanged, so a stream-contract violation or the +re-validation's `InvalidArgumentError` is never re-wrapped into something retryable. Everything +else — a refused connection, a resolution failure, a TLS failure, `protocol-http1`'s refusal of a +malformed head, a timeout — wraps as `Dexpace::TransportError` carrying the original as `#cause`, +retryable unconditionally (`TRANSPORT-4`, `TRANSPORT-20`), never a class list: of the families +`async-http` raises, not one is an `::IOError`. A failure while the request is being adapted — a +scheme the adapter cannot dispatch, a body that raises — is delivered through the future too, never +thrown past it (`TRANSPORT-21`); and a success always carries a `Dexpace::Response`, a `204` with no +body included (`TRANSPORT-23`), because the pivot settles with what the mapper built and nothing +else. + +```ruby +Sync do + adapter.call(req("http://127.0.0.1:1/"), EMPTY, nil).value # nothing listens on port 1 +rescue Dexpace::TransportError => error + [error.retryable?, error.phase] # => [true, :connect] + error.cause.class # => Errno::ECONNREFUSED +end +Sync { adapter.call(req("ftp://example.test/"), EMPTY, nil).value } +# raises Dexpace::InvalidArgumentError (the async transport dispatches http and https only) +headers("X-Inject" => "a\r\nEvil: 1") +# raises Dexpace::InvalidArgumentError (HTTP-18): the MODEL's own builder refuses the value, so a +# request built through it never reaches the adapter with one. The adapter re-validates +# regardless, for a request-shaped object that met no builder (design §10.10's admitted hole): +server = WireServer.start(Scripts.fixed("ok")) +forged = Object.new +forged.define_singleton_method(:method) { Dexpace::Method::GET } +forged.define_singleton_method(:url) { Dexpace::URL.parse!("http://127.0.0.1:#{server.port}/") } +forged.define_singleton_method(:headers) do + Object.new.tap { |h| h.define_singleton_method(:each_entry) { |&b| b.call("X-Inject", "a\r\nEvil: 1") } } +end +forged.define_singleton_method(:body) { nil } +Sync { adapter.call(forged, EMPTY, nil).value } +# raises Dexpace::InvalidArgumentError (HTTP-18: the adapter's own re-validation, before anything is +# mapped and before a byte reaches the socket) +server.requests # => [] +``` + +## Lenient inbound mapping, and the two clauses it waives + +A vendor status inside `100`–`599` maps with its body readable (`TRANSPORT-24`); an HTTP/2 head maps +to the model's `http/2`. A header whose **value** carries a control byte is dropped alone and logged +at verbose while obs-text is preserved and a repeated `Set-Cookie` survives as two values +(`TRANSPORT-14`'s value clauses); a malformed `Content-Type` downgrades to no media type +(`TRANSPORT-27`'s media-type clause); an absent native length is the `-1` sentinel. Two clauses are +**not** satisfiable on this adapter and are named waivers in its conformance run rather than silent +gaps (`P8-38`): a malformed inbound header **name** and a non-numeric `Content-Length` both make +`protocol-http1` refuse the whole response out of the read, before a response object exists to +adapt, and the failure surfaces as a retryable `TransportError` carrying the library's own error. +`Net::HTTP` delivers both heads, so 8a's driver waives nothing. + +```ruby +server = WireServer.start(Scripts.vendor_status(299, "custom")) +Sync do + response = adapter.call(req("http://127.0.0.1:#{server.port}/"), EMPTY, nil).value + [response.status.code, response.body_string] # => [299, "custom"] +end + +server = WireServer.start(Scripts.malformed_headers) # a non-ASCII header NAME among the good ones +Sync do + adapter.call(req("http://127.0.0.1:#{server.port}/"), EMPTY, nil).value +rescue Dexpace::TransportError => error + [error.class, error.cause.class] # => [Dexpace::TransportError, Protocol::HTTP1::BadHeader] +end +``` + +## TLS and HTTP/2 + +An `https` origin under the owning construction gets the adapter's own `OpenSSL::SSL::SSLContext`: +`VERIFY_PEER` with the platform's default certificate store, and `alpn_protocols` set to +`ALPN_PROTOCOLS` (`h2`, then `http/1.1`), so HTTP/2 is negotiated wherever the peer offers it and +HTTP/1.1 otherwise. A caller's `ssl_context:` is used **verbatim** — never re-armed — so a context +without `alpn_protocols` negotiates HTTP/1.1, and one with a private store trusts what it says. Over +HTTP/2 one connection multiplexes every concurrent call to an origin (the adapter's suite drives +sixteen through one and counts one connection), which is the property a thread pool cannot have; +plaintext prior-knowledge HTTP/2 is not something an adapter can know from a URL, so it is reachable +only through `.using` over a caller's own client built for it. Both protocols are proven by the +adapter's suite against an in-process `async-http` server — HTTP/1.1, plaintext HTTP/2 and TLS HTTP/2 +by real ALPN over a per-run self-signed certificate — including the wire-boundary re-validation over +HTTP/2, where nothing below the model validates and a CRLF value would otherwise reach the peer +verbatim. The portable suite's fixture speaks plaintext HTTP/1.1 only, so `TRANSPORT-4`'s +open-timeout half and every TLS property are this gem's own suite's, as its `PREAMBLE` says. + +## Close + +```ruby +adapter.close # => nil, reactor-free, never waits on a busy connection +adapter.closed? # => true +Sync do + failed = adapter.call(req("http://127.0.0.1:1/"), EMPTY, nil) + failed.settled? # => true + failed.value # raises Dexpace::ClosedError (through the future, never synchronously) +end +``` + +An owning adapter fails every later call through the future with `Dexpace::ClosedError` and never +opens a connection; a borrowing adapter stays usable after its own close, because the client is the +caller's (`TRANSPORT-15`, `TRANSPORT-16`, `SEAM-15`). + +## What is deliberately not here + +No redirect following, no retry and no authentication live in this gem: those are phase 6's pillar +steps, and `AsyncPipeline.standard(adapter, redirect: :unsupported)` is how a caller gets retry and +authentication around this transport (`docs/sdk-documentation/pipelines.md`). No reactor of the +adapter's own, for the reasons above. No proxy: `async-http` has no proxy route the adapter could +carry the configuration chain into, so `TRANSPORT-30` stays the synchronous adapter's. No +transport-milestone tracing (`OBS-28`): the adapter takes a `logger:` and no tracer, because no +route exists from a three-argument seam to a per-operation `HTTPTracer` (8a's R6, `P8-7`, on phase +10's inbound list). No `Content-Type` default, no `User-Agent`, no auto-stamp of any kind. And no +Ruby 3.2: this gem alone declares 3.3, and the composition that keeps a 3.2 consumer whole is the +synchronous transport, `dexpace-transport-net_http`. diff --git a/docs/work/mvp/2026-09-05-ruby-sdk-v1-roadmap-design.md b/docs/work/mvp/2026-09-05-ruby-sdk-v1-roadmap-design.md index c62f7f1..5c50002 100644 --- a/docs/work/mvp/2026-09-05-ruby-sdk-v1-roadmap-design.md +++ b/docs/work/mvp/2026-09-05-ruby-sdk-v1-roadmap-design.md @@ -2721,6 +2721,46 @@ design. deadline arithmetic want the same screen. Touches `CFG-15`, `CFG-17`, `CFG-18` and `XCUT-11`, and nothing normative beyond the messages. Recorded by phase 8b's review round 1 on its docs branch; referred to by date and content, never by ordinal. +- **Two of phase 8c's design facts are stale on `async` 2.46.0, and `async-http`'s server lets a + peer's mid-head `EOFError` reach Console.** Found 2026-09-21 by phase 8c's implementation against + the bundle's `async` 2.46.0 / `async-http` 0.105.0, where the design measured 0.104.0. Verified fact 8 + passes `cause: :sym` to `Task#cancel` and reads it back from the task's `$!.cause`: on 2.46.0 a + non-`Exception` cause is replaced by the runtime's own `Async::Cancel::Cause` ("Cancelling task!"), + so a reason travels only as an exception (the adapter wraps it in `Dexpace::CancelledError`; `P8-91`). + Verified fact 10 says "`Async { }` inside a reactor is not a child of the caller": `Kernel#Async` + inside a running task delegates to `Task.current.async`, `inner.parent.equal?(task)` measured true + on 3.3.12, 3.4.10 and 4.0.6, so the reviewer's mutation 14 (the exchange spawned with `Async { }`) + is an equivalent mutant and `caller_task.async` is the honest spelling rather than a distinction + the runtime draws. Both are corrected in `docs/knowledge/notes/concurrency-and-async.md`'s new entry + and the design's As-built addendum, and the design document itself — frozen to its phase — still + states them; the `docs/deviations.md` flip should read the addendum, not the fact list. Separately, + `Async::HTTP::Server#accept` rescues `Protocol::HTTP::BadRequest` and nothing else, so a peer that + closes between the request line and the end of the headers — an exchange cancelled mid-send, which + 8c's suites do on purpose — raises `EOFError` out of the per-connection task, which Console reports + to stderr as a task failure; seen once in a hundred-odd whole-suite runs on 4.0.6 under load and + worked around in the test fixture (`AsyncHTTPServerFixture::QuietServer`), and worth an upstream + report rather than an SDK change. Touches `TRANSPORT-8`, `ASYNC-6` and nothing normative in the + code. Recorded by phase 8c on its docs branch; referred to by date and content, never by ordinal, + because 8b's lane is writing to the same list. +- **The portable `TRANSPORT-7` row proves its delivered-body clause by chance against a streaming + adapter.** Found 2026-09-21 by phase 8c's review round 1 (R1-1), measured by the fix round: the row + cancels the token on a second thread the moment the server has written the head, and against a + streaming adapter that cancel lands either before the adapter has checked its token on the + delivered head (the send surfaces the cancellation — the in-flight path) or after the consumer's + body read has blocked (the read does), a race the contract's primitives cannot settle without a + bound inside the assertion, because an eager adapter's send never returns under the script and only + the server can signal it. Against a mutant of `Dexpace::Transport::AsyncHTTP` with the watcher's + close of a delivered response deleted, half the runs on 4.0.6 and two thirds on 3.3.12 passed through + the send path and the rest hung in the read until the async driver bounded `around:` from a parent + task; the adapter's own + `cancellation_test.rb` pins the body path deterministically (the consumer signals from inside the + read, and the test refutes its own bound as the wake's cause), and the row's comment in + `transport_suite/asynchronous.rb` states the race. A deterministic portable form — the script + writing head and first chunk, the consumer signalling after it, an eager adapter measured vacuous + through a short transport timeout — changes what the row asserts on 8a's driver and 8a's own + `RawWireTransport` proof, so it is conformance-gem work for the phase that next touches the suite, + not a fix round's. Touches `TRANSPORT-7` and nothing normative in the code. Recorded by phase 8c's + review round 1 on its docs branch; referred to by date and content, never by ordinal. **2026-09-13** — **Execution order amended by the roadmap-level generator-fitness review, which read the plan end to end against one question: will a generated OpenAPI client be able to use this?** No cell of @@ -4783,3 +4823,224 @@ stays a sentence here and in the checklist, never a row in `docs/deviations.md`. `P8-20`–`P8-25` and `P8-71`–`P8-78` into design §10 is a human's: `docs/sdk-design-ruby/` is frozen. `main` is `a7cfeb6` before and after; the stack is not on it, 8c is being built beside it, and umbrella #29 stays open for both. + +**2026-09-21** — **Phase 8c implemented**, as three stacked branches against issue #32: code, tests, +documentation, cut from `main` at `a7cfeb6`, which holds every phase through 7 and phase 8a — +concurrently with 8b off the same base, so this is the phase-8 lane that lands **second of the two**, +and the one that makes the wire-boundary re-validation phase 1 postponed complete in both adapters +(8a's Task 16 and this lane's Task 9; phase 9's Task 7 adds the portable assertion), and the one that +lands the header-drop policy phase 5b postponed to phase 8 as `OBS-19` (`DropPolicy`, the predicate at +dispatch, the both-protocols test, the antecedent confirmed on `protocol-http1` 0.41.0). The one core +widening is `Configuration::Keys::TRANSPORT_CONNECTION_LIMIT`; `Dexpace::TransportError`, +`Events::TRANSPORT_HEADER_DROPPED` and `Keys::REQUEST_TIMEOUT` were 8a's and on the base. +**`dexpace-transport-async_http`** carries `Dexpace::Transport::AsyncHTTP`: the entry file's six +constants, `.build(timeout:, logger:, drop_policy:, connection_limit:, ssl_context:, configuration:)` +over a client map the adapter owns — one `Async::HTTP::Client` per **(reactor, origin)**, bounded at +`MAX_ORIGINS` and drained after every insert — `.using(client, logger:, drop_policy:)` over a caller's +own client with `retries` already zero, `.default` and the require-time `AsyncTransport.register(:async_http, …)`, +and ten files under `async_http/`: `adapter.rb` and `drop_policy.rb` public and the eight +`private_constant`s `clients.rb`, `endpoints.rb`, `errors.rb`, `exchange.rb`, `request_body.rb`, +`request_mapper.rb`, `response_body.rb` and `response_mapper.rb`, every one mirrored in `sig/` and +every one but `exchange.rb` in `test/`; its gemspec declares `async-http ~> 0.104` and a Ruby floor of +3.3 read from `VERSIONS`' per-gem row (`P8-36`). **`dexpace-conformance`** gains two private groups, +`Asynchronous` (`TRANSPORT-7`, `-9`, `-21`, `-23`) and `HeaderDrops` (`TRANSPORT-12`, `-13`), so the +suite is thirty-four assertions in seven groups with `PREAMBLE` naming `TRANSPORT-8` as the third thing +a green run does not prove; `Scripts.write_response` takes `close:` and every head a script writes +carries `Connection: close`, with every 8a count unchanged. **The repository** gains the per-gem floor: +`VERSIONS`' `floor:` row, `DexpaceVersions.ruby_floor(gem)` and `.gem_supported?`, the two gates +and the three loaders (`Gemfile`, `test:gems`, `gates:clean_bundle`) reading it, and two gate fixtures; +the `Steepfile`'s `:async_http` target relaxed exactly as `:serde_json`'s; `test/support/async_http_warmup.rb`; +and two surface manifests regenerated once — core 1 335 → 1 336, `async_http` 2 → 25, `conformance` +unchanged. Four earlier-phase test files changed on the code branch as pins the code moved +(`keys_test.rb`, `downstream_wirings_test.rb`, 8a's driver's size pin, `lifecycle_test.rb`'s) plus +8a's `keep_alive_twice` fixture passing `close: false`, and core's `async_transport_test.rb` moved its +four in-process bare-require pins to `async_transport_bare_require_test.rb` (8a's shape on the other +seam). **The design's five rows stand; `R13`–`R16` were built as written**, with the As-built addendum +adding `P8-91`–`P8-102`: the cancellation bridge is queue-marshalled in both directions, because +`Async::Task#cancel` from a foreign OS thread raises and cancels nothing and `cause:` drops a Symbol +(P8-91); the client map is keyed by reactor, the manager's decision on the cross-check's open question +1 (P8-92); `TRANSPORT-8` stays an adapter's own row and the portable groups carry six assertions plus +the preamble sentence (P8-93); a response does not outlive the reactor that produced it — `Sync`'s exit +drains the pool and waits on every busy connection — so the driver's foreign-thread settle materialises +the body inside its reactor, found by `TRANSPORT-29`'s eight threads hanging (P8-94); the per-gem floor +is one `VERSIONS` row read everywhere (P8-95); `Connection: close` in `Scripts`, because a keep-alive +client re-used a connection the fixture had closed one time in two (P8-96); no `Content-Type` is +invented, since `async-http` stamps none and neither of 8a's `P8-4` reasons exists here (P8-97); the +4.0-only `IO::Buffer` warning parked at test-helper load (P8-98); the Steep relaxation beside an +`rbs_collection.yaml` row ignoring the collection's stale `async/2.12` by name — the first cut said the +collection carried none, and review round 0 measured it installing the moment the workspace gem's own +ignore was lifted (P8-99); `P8-37` as built +retires every pooled resource before `pool.close`, which alone would drain and wait exactly as +`Client#close` does (P8-100); the native HTTP/2 body closed through `close_quietly`, because +`async-http` writes `RST_STREAM` before it transitions the stream and an `END_STREAM` in that window +releases the pooled connection twice, five of five (P8-101); and the public surface as built against +the design's object model — `.build`/`.using` over `.owning`/`.borrowing`, `DropPolicy` public, no +`_Client` interface (P8-102). Two of the design's thirteen facts do not hold on `async` 2.46.0 — +`cause: :sym` and "`Async { }` inside a reactor is not a child of the caller" — corrected in a new +`docs/knowledge/notes/concurrency-and-async.md` entry beside a new `transport-adapter.md` entry for the +reactor-exit drain, `P8-37` as built, the h2 double release and the 4.0 warning; the design is frozen to +this phase and phase 10's inbound list carries the pointer, by date and content. The checklist is at +`docs/work/mvp/phase8/phase8c/2026-09-11-phase8c-asynchronous-transport-checklist.md`: ten own rows — +nine ✅ and `ASYNC-21` N/A with its one honourable property asserted — plus eleven cross-reference rows; +the reviewer's thirty mutations run as thirty-six rows on 4.0.6 and 3.3.12 (the gem's floor row; +3.2.11 has no bundle for it), thirty-four red on both and two equivalent mutants recorded with their +measurement (`Kernel#Async` is the current task's child; a raise inside `#dispatch`'s fence is still a +settlement), while check-after-resume — recorded equivalent when only its close count was measured — is +caught by the round-2 hook-race case and counted among the red rows below — thirty-nine rows after review rounds 0 and 1, whose three surviving extra mutants (the +`BINARY` retag unobserved by a BINARY-only fixture; `Exchange#net`'s settle with no fixture reaching it; +the watcher's close of a delivered response indistinguishable from the body-path test's own bound) were +each given the guard that runs them red on 2026-09-21, and forty-three rows after review round 2, whose +three surviving extras (a cancel hook's push racing the exchange's own end; the body-forbidden guard +no test reached; the watcher's transience provable only by a hang) were each given theirs the same day +and whose fourth extra is the third equivalent mutant (the `cause:` wrap the adapter never reads) — +forty red in all — after five guards the +first pass found missing were added — the mutex scan reaching +`build_client`, `assert_exchange_released` over the watcher's annotation, `reactor_over` closing a +holding fixture inside the reactor so a blocked exchange fails instead of hanging, every wait bounded, +and `TRANSPORT-3`'s list carrying the SDK's own errors; the design's facts re-run on 3.3.12, 3.4.10 and +4.0.6, eight of them as `matrix_facts_test.rb` printing the row's versions; forty-three departures from +the plan's text itemised, among them the nine suites wrapped into modules of nested classes and the +conformance groups split in two under the metric cops, and the two found only by the whole-repository +`test:gems` process. The driver reports **four** skips, not the plan's three — two assertions carry +`TRANSPORT-14` and a waiver is by id — beside `TRANSPORT-27`'s waiver and `TRANSPORT-18`'s measured +vacuity; on the 3.2 row the gem is absent, five gems install, test and clean-bundle, and every gate is +green. Re-proven at every tip: the code tip green on every one of the +eighteen gates run individually on 4.0.6 (`test:gems` 3,713 runs, 72,711 assertions, 0 failures, +0 errors, 3 skips — 8a's — and 97.52 % line coverage, above the floor, so no tip in the stack is red; the +honest RuboCop run over 683 files clean; `gates:clean_bundle` loading all six gems) and on the 3.2.11 +matrix row (3,704 runs, 72,677 assertions, 3 skips, `gates:clean_bundle` five gems — this gem absent by +its floor); the tests tip green on the whole default task on 4.0.6 (3,900 runs, 73,502 assertions, +0 failures, 0 errors, **7 skips** — 8a's three and this driver's four — 99.88 % line coverage; the honest +RuboCop run over 708 files clean; `steep check` over six targets clean), on the matrix set on 3.2.11 +(3,712 runs, 72,726 assertions, 3 skips, five gems), 3.3.12 (3,900 runs, 73,502 assertions, 7 skips, six +gems, `openssl` 4.0.2 the bundle's on that row and on 3.4.10) and 3.4.10 (3,900 runs, 7 skips), with +`matrix_facts_test.rb` printing `async-http 0.105.0, async 2.46.0, protocol-http 0.72.0` on the two rows +that carry the gem; the docs tip green on the default task, the honest RuboCop run, the probe, the +knowledge-structure verifier, the housekeeping and knowledge test suites, and every `ruby` fence of +`transport-async_http.md` (as one script, on 4.0.6 and 3.3.12) and the changed fences of +`conformance.md`. `docs/sdk-documentation/transport-async_http.md` is the +twentieth as-built page, every example run on 4.0.6 and 3.3.12 and identical on both but for the +ephemeral port one `Host` line names; `conformance.md`'s counts, preamble and report examples re-run +on both; `architecture.md`, the gem README (with its `ASYNC-7` section, the reactor rule, the +`sync_over` caveat, the `async_over` hazard, `content-length: 0`, the timeout unit and the 4.0 warning +line), `README.md` and `docs/README.md` point at the page; `CLAUDE.md`'s built-phases paragraph gains +the gem and core's key, its floor sentence names this gem's 3.3, its counts move to nineteen checklists +and the two adapter gems' file counts, and its constraints-that-bite list gains one line; +`docs/first-release.md` changes in existing entries only — the conformance-suite line's second-driver +status, the `P8-9` documentation box ticked, the `gates:bounded_map` blocker's status and the +supported-Ruby note's 0.105.0; `docs/deviations.md` is untouched, for phase 10 to flip. One dated +bullet joins phase 10's inbound list above — the design's facts 8 and 10 stale on `async` 2.46.0, and +`async-http`'s server letting a mid-head `EOFError` reach Console — by date and content, never by +ordinal, because 8b's lane is writing to the same list. The consolidation of `P8-36`–`P8-40` and +`P8-91`–`P8-102` into design §10 is a human's, as for every phase before: `docs/sdk-design-ruby/` is +frozen. + +**2026-09-21, review rounds 0 and 1 of the phase-8c stack.** Round 0 returned `changes_requested` with +one blocking finding and six others, every one repaired in place in the note above on the branch that +owns the file — the hermetic configuration double behind every default pin, the collection's +`async/2.12` row with its measured reason, guards 34 and 35, the page's forged-request and post-close +examples — and one deferred to the manager (the merge-order sentences). Round 1 returned +`changes_requested` with one should-fix and two nits. Should-fix, on the tests branch: the body-path +cancellation test passed with the watcher's close of a delivered response deleted, because its own +five-second `with_timeout` fired inside the native read and the adapter's token-first classifier +turned that `Async::TimeoutError` into the `CancelledError` the test expected, body closed — the test +now refutes `Async::TimeoutError` as the cancellation's cause (guard 37, red on 4.0.6 and 3.3.12); and +the same mutant hung the portable `TRANSPORT-7` row under the async driver whenever its race fell +on the body path (half the runs on 4.0.6, a third on 3.3.12), so the driver's `around:` now runs +each assertion as a child task and bounds the parent's wait at thirty seconds, cancelling the child +and flunking by name on expiry — a bound raised into the assertion's own fiber meets the same +classifier and was measured PASSING the row thirty seconds late, which is why the child-task shape and +not the reviewer's one-liner (deviation 42). The row's comment on the code branch now states that its +delivered-body clause is proven by chance against a streaming adapter, and the deterministic portable +form is a dated bullet on phase 10's inbound list above. The nits: the counts in this note were round +0's (the tests tip was 3,898 runs and 73,490 assertions on every six-gem row after guards 35 and 37, +and the `async_http` manifest is 25 rows, a gain of twenty-three), and the gemspec's comment +understated `~> 0.104`, which admits every 0.x release from 0.104 on. Nothing in `lib/` changed but +two comments; every gate re-run green at every tip. + +**2026-09-21, review round 2 of the phase-8c stack.** `changes_requested` with two should-fixes and +two nits, every round-0 and round-1 finding verified fixed. The one change to `lib/`: a token cancel +in flight while the exchange finished on its own raised `ClosedQueueError` out of +`Cancellation::Source#cancel` on the canceller's thread — the source steals its hooks under its +mutex and runs them outside, so the adapter's hook could run after check-after-resume had settled +the pivot cancelled and closed the exchange's queue, and `Hooks.notify` handed the push's raise back +to the caller (reproduced deterministically on 4.0.6 and 3.3.12 with an ordinary caller hook +registered first). Both hooks now push through `Exchange#signal`, which rescues that one error +(deviation 43, guard 51, `P8-91` amended). On the tests branch: the body-forbidden clause of dispatch +step 8 is asserted over a forged GET and HEAD carrying a body, the only shape that reaches the guard +(guard 47v); and the delivered response's watcher is asserted present and transient before the body +is released, so a watcher spawned without `transient: true` fails by name in milliseconds instead of +holding a reactor open until the run is killed (guard 44). The other nit is the record's: the +`CancelledError` wrapped into `Task#cancel(cause:)` was described as load-bearing and is read back +by nothing — the pivot is settled with the reason before the watcher acts — so the checklist, +`P8-91`, the knowledge note and `CLAUDE.md` now say it names the reason on the task's own +`Async::Cancel` for whoever reads the task (guard 46, equivalent). The tests tip is 3,900 runs and +73,502 assertions on every six-gem row; every gate re-run green at every tip. + +**2026-09-22** — **Phase 8c reconciled onto `main` after phase 8b (the async-runtime adapter)** — 8b is the +lane that lands first, and `main` is still `a7cfeb6` while its three squashes are in flight, so the base +this pass rebased onto is 8b's reconciled docs tip `5755267`, whose tree they land byte for byte. 8c's +three branches, built off `a7cfeb6` concurrently with 8b and reviewed at `c830725` → `2449b6c` → +`13873e5`, were rebased onto that tree with `git rebase --onto` (rerere disabled), every 8c commit +preserved and none reordered or reworded: the stack is `c05ada2` (code, seven commits) → `a56336e` +(tests, four) → `6cc9d52` (docs, four) plus this paragraph's own commit, the pass's one commit of its own +on the docs branch, carrying what no 8c commit could: this paragraph, the dated "Reconciled" note at the +head of 8c's checklist, the checklist count in `CLAUDE.md` re-derived to twenty (the replay had kept one +lane's nineteen), and the three documentation nits review round 3 left — guard row 18 rewritten as +red-by-name through the round-2 hook-race case with its close-count measurement kept as its own sentence +and the guards arithmetic in the checklist and this note moved from thirty-nine of forty-three / four +equivalent to forty of forty-three / three, guard rows 8 and 17's stale second citations dropped or +replaced (`dispatch_conformance_test.rb`'s tls variant builds its adapter from a caller `ssl_context` the +default context's ALPN line never reaches; row 17 now names `adapter_test.rb`'s already-cancelled-token +case), and the code tip's coverage written as the measured 97.52 %. +**Six files both lanes rewrote were reconciled inside the replayed 8c commits and nowhere else.** +`CLAUDE.md`: the built-phases sentence names 8a, 8b and 8c and now says the whole of phase 8 is built — +8a first, then 8b and 8c concurrently off the tree that holds 8a, 8c landing second; the opening +paragraph carries the fifth real gem (`dexpace-async-thread`) and the sixth +(`dexpace-transport-async_http`) in merge order, the two skeleton clauses become one statement that no +phase-0 skeleton remains, the socket sentence carries both the pool and the asynchronous transport, the +gem table and the claim paragraph carry both lanes' clauses, the floor paragraph is 8c's, the constraints +list carries 8c's async-http line at its own anchor before the conformance bullet and 8b's four lines +after it, and every count is re-derived: "twenty checklists written so +far", `phase8/` checklists for 8a, 8b and 8c, and "Every checklist but …" naming all three. `README.md`, +`docs/README.md` and `docs/sdk-documentation/architecture.md`: both gems' paragraphs and both new pages +linked in merge order, no skeleton sentence left, "the twenty-one pages written so far" and +architecture's opening count twenty-one. `docs/first-release.md` auto-merged and verified hunk by hunk: +8b's one dated sentence in the unsatisfied-MUST entry beside every 8c hunk — the 3.2 supported-Ruby +lines, the openssl-on-3.3 clause, the conformance line's async-http status, the `P8-9` box ticked and the +`gates:bounded_map` blocker's status. And this roadmap: 8b's status note then 8c's with its +review-round paragraphs, and the phase-10 inbound list's four new bullets, 8b's two before 8c's two, +each by date and content. **Counted from the rebased tree, never copied from either side's prose**: 220 +`lib/dexpace/` files beside `version.rb` with 220 `sig/` mirrors and the same nineteen +`private_constant` test-mirror exceptions, each verified to have no `test/` mirror while every other core +lib file has one; twenty `*-checklist.md`; twenty-two pages under `docs/sdk-documentation/` with +twenty-one written beside the front-door `architecture.md`; eighteen gates; this gem's eleven `lib/` +files beside `version.rb` (eight private), `dexpace-conformance`'s twenty-five beside `version.rb` (nine +private), `dexpace-async-thread`'s four (one private), `dexpace-transport-net_http`'s nine (seven +private) and `dexpace-serde-json`'s two; and the six manifests — core 1 335 → 1 336, this gem's 2 → 25, +`dexpace-async-thread`'s 14 (8b's rows, the base's), `conformance` 100, `net_http` 17, `serde-json` 15 — +with `surface:regenerate` on the rebased tests tip a no-op. Every file only one lane touched is +byte-identical to that lane's tip — each 8b-only file to `5755267`, and each 8c-only file to `13873e5`, +its checklist excepted for this pass's note and three nit fixes — and the only files differing from both +are the six above. **Re-proven at every rebased tip.** The code tip is green on every one of the eighteen +gates run individually on 4.0.6 (`test:gems` 3,833 runs, 73,334 assertions, 0 failures, 0 errors, 3 skips +— the `net_http` driver's `TRANSPORT-18` vacuity and the two new groups' rows measured vacuous there — +and 97.57 % line coverage, above the floor, so no tip in the stack is red; the honest RuboCop run over +700 files clean; `gates:clean_bundle` loading all six gems) and on the 3.2.11 matrix row (3,824 runs, +73,300 assertions, 3 skips, 95.62 %, five gems, the lock naming neither this gem nor `async-http`). The +tests tip is green on the whole default task on 4.0.6 (4,020 runs, 74,125 assertions, 0 failures, +0 errors, **7 skips** — the `net_http` driver's three and the async driver's four, each named by its +driver — and 99.88 % line coverage; the honest RuboCop run over 725 files clean), on the matrix set on +3.2.11 (3,832 runs, 73,349 assertions, 3 skips, five gems), 3.3.12 (4,020 runs, 74,125 assertions, +7 skips, six gems, `openssl` 4.0.2 the bundle's on that row and on 3.4.10) and 3.4.10 (4,020 runs, +74,125 assertions, 7 skips), with the whole-process 3.2.11 error the known interleaving-dependent +`RETRY-42` / `RECOV-28` eight-thread case took under this pass's first seed (54433) rerunning green under +the same seed and twice more (3,832 runs and 3 skips every time; the case is on phase 10's inbound list +above), with 8b's composed suite run by name — ten cases, 45 assertions, green against the +`Connection: close` fixture 8c gave `WireServer` — and with `surface:regenerate` on the tests tip +changing nothing. The docs tip is green on the default task, the honest RuboCop run, the probe, the +knowledge-structure verifier and both process-tooling suites, with every `ruby` fence of +`transport-async_http.md` run as one script on 4.0.6 and 3.3.12 (the same printed values on both but the +ephemeral port), `conformance.md`'s changed fences on both and `async-thread.md`'s blocks once more on +4.0.6. `main` is `a7cfeb6` before and after this pass; nothing is pushed, and umbrella #29 stays open for +both. diff --git a/docs/work/mvp/phase8/phase8c/2026-09-11-phase8c-asynchronous-transport-checklist.md b/docs/work/mvp/phase8/phase8c/2026-09-11-phase8c-asynchronous-transport-checklist.md new file mode 100644 index 0000000..f7efbd5 --- /dev/null +++ b/docs/work/mvp/phase8/phase8c/2026-09-11-phase8c-asynchronous-transport-checklist.md @@ -0,0 +1,459 @@ +# Phase 8c — Asynchronous Transport: Checklist + +**Written at execution time, 2026-09-21, from what was built** — not from the plan. A row whose task +did not do what the plan said is a row that says so, and the "Deviations from the plan" section +below is where each departure is stated with its reason. The design and the plan were written on +2026-09-11 (reviewed 2026-09-12 and 2026-09-13) against phases 0–7's documents and `async-http` +0.104.0, concurrently with 8a's and 8b's documents, on a machine that then had only Ruby 3.4.10. +Since then every phase through 7 and phase 8a were built and merged; this phase was cut from `main` +at `a7cfeb6`, which holds all of them, and it executed **concurrently with phase 8b** off the same +base — so nothing here names an 8b constant, the `Transport.async_over`-over-a-socket half of the +seam stays 8b's, and this document describes only what this lane built. It is the phase-8 lane that +lands **second of the two off this base**, and the one that makes the wire-boundary re-validation +complete in both adapters. Where the plan's text and the built tree disagree the tree wins and this +document records it. The bundle the whole run used resolved `async-http` **0.105.0**, `async` 2.46.0, +`async-pool` 0.12.0, `protocol-http` 0.72.0, `protocol-http1` 0.41.0, `protocol-http2` 0.28.0, +`io-event` 1.22.0 and, on every row this gem builds on, `openssl` 4.0.2 — the design's facts were +measured on 0.104.0 and re-run on these (the *Matrix facts* section). + +**Reconciled 2026-09-22.** 8b is the sibling that lands first — its reconciled docs tip +`5755267` holds, byte for byte, the tree its three squashes put on `main`, which is still `a7cfeb6` +during this pass — so this phase's three branches were rebased onto that tree by `git rebase +--onto` with rerere disabled, every 8c commit preserved and none reordered or reworded: code +`c830725` → `c05ada2` (seven commits), tests `2449b6c` → `a56336e` (four) and docs `13873e5` → +`6cc9d52` (four) plus the pass's one commit of its own, which carries this note, the roadmap's +reconciliation paragraph, the `CLAUDE.md` checklist count re-derived to twenty, and the three +documentation nits review round 3 left (guard +row 18's status, guard rows 8 and 17's citations, and the roadmap note's 97.55 %). The sentences here +that count the tree or say what landed first describe **this phase's own base**, `a7cfeb6`, and are +left as written; on the combined tree the figures are: every one of the six gems real — no phase-0 +skeleton remains — **220** `lib/dexpace/` files beside `version.rb` with 220 `sig/` mirrors and the +same **nineteen** `private_constant` test-mirror exceptions (this phase adds none in core: its one +core edit, `configuration/keys.rb`, is an existing public file), **twenty** checklists, +**twenty-one** as-built pages beside `architecture.md`, eighteen gates, the core manifest +1 335 → 1 336, this gem's 2 → 25 and `dexpace-async-thread`'s 14 (8b's rows, already on the base); +a `surface:regenerate` on the rebased tests tip changed nothing. The six files both lanes rewrote — +`CLAUDE.md`, `README.md`, `docs/README.md`, `docs/sdk-documentation/architecture.md`, +`docs/first-release.md` and the roadmap — were reconciled inside the replayed 8c commits. Deviation +40 below is the one sentence here that 8b's presence makes false: it counts nineteen checklists, and +the combined tree carries twenty. + +Legend, verbatim from the roadmap's cross-cutting constraint 3: ✅ implemented and tested · +🚫 not built (permanent simplification, named reason) · ⏳ deferred (naming the plan task — phase, +task number and path — that will do it, or the `docs/first-release.md` entry that owns it) · +N/A not applicable in this port. + +Plan: `docs/work/mvp/phase8/phase8c/2026-09-11-phase8c-asynchronous-transport.md`. Task numbers are +that plan's (nineteen numbered tasks). Design: +`docs/work/mvp/phase8/phase8c/2026-09-11-phase8c-asynchronous-transport-design.md`, whose Deviation +Ledger carries `P8-36`–`P8-40` and whose As-built addendum, written with this checklist, adds +**`P8-91`–`P8-102`** (the charter fixes the bands: `P8-36`–`P8-50` for 8c's design rows, as-built rows +from `P8-91`; nothing is renumbered). The charter is +`docs/work/mvp/phase8/2026-09-11-phase8-segmentation-design.md`. Test files are named with their gem: +`async_http/…` is `gems/dexpace-transport-async_http/test/dexpace/transport/async_http/`, +`conformance/…` is `gems/dexpace-conformance/test/dexpace/conformance/`, and `core/…` is +`gems/dexpace-core/test/dexpace/`. Every `lib/` file in this gem has a `sig/` mirror (the eight +`private_constant`s with `hooks.rbs`'s comment) and every one but `exchange.rb` a `test/` mirror — +`Exchange` is the per-call object and is proven through `async_http/cancellation_test.rb`, +`async_http/parent_cancellation_test.rb` and `async_http/adapter_test.rb`, the three suites that +drive it, and nowhere else; the files with no `lib/` mirror say so in their headers. + +## Requirement rows + +Ten own rows — `TRANSPORT-7`, `-8`, `-9`, `-12`, `-13`, `-21`, `-23`, `ASYNC-6`, `-21`, `-22` — plus the +cross-reference rows for the IDs this phase exercises and does not own. **Nine ✅ and one N/A** +(`ASYNC-21`, §11.21's reactive SSE bridge, whose one property this adapter can honour is asserted on +`ResponseBody` beside the row), nothing 🚫, nothing ⏳; `TRANSPORT-8`'s row states that it is +**satisfied on this adapter** where §12 records it vacuous, `TRANSPORT-12`'s that the drop is applied +on both protocols by construction and its antecedent is the seventeen delimiter bytes `HTTP-17` admits, +`TRANSPORT-13`'s that the bound is sixty-four names, and `ASYNC-22`'s that cross-thread safety is +structural through a client map keyed by reactor. + +| ID | Level | Status | Task(s) | What was built, and where it is proven | +|---|---|---|---|---| +| `TRANSPORT-7` | MUST | ✅ | 11, 12, 16, 19 | Cancelling the token or the future reaches the in-flight exchange through the queue-marshalled watcher (`Exchange#watch` cancels the exchange task while it is in flight — with the reason wrapped in a `CancelledError` as the `cause:`, which names it on the task's own `Async::Cancel` for whoever reads the task and which nothing in the adapter reads back, the pivot being settled with the reason before the watcher acts — and closes the delivered response afterwards), the connection is released and the future settles a terminal, non-retryable `Dexpace::CancelledError` with the token's reason (`async_http/cancellation_test.rb`, "TRANSPORT-7/ASYNC-6: cancelling the token aborts a blocked native call…", "a token cancelled from a foreign OS thread still reaches the exchange, promptly", "TRANSPORT-7 on the body path: a token cancelled under a blocked body read wakes the reader…", which since review round 1 refutes `Async::TimeoutError` as the cancellation's cause — the wake must be the watcher's close, a bare `IOError` under the read, and never the test's own bound, because the token-first classifier turns the bound's expiry into the same `CancelledError` with the body closed (guard 37); the portable assertion `conformance/transport_suite/asynchronous_test.rb` "TRANSPORT-7", passing against `RawWireTransport` and failing against its `misclassify_cancel` defect, and green against this adapter through the driver, `async_http/conformance_test.rb` — proving the in-flight clause on every adapter and the delivered-body clause only when its cancel lands after the consumer's read has blocked, a race between the server's signal and the client's head parse that the contract's primitives cannot settle (measured against the mutant of guard 37: half the runs on 4.0.6 and a third on 3.3.12 took the body path; the body path's deterministic proof is the adapter's own test above, and the driver's bounded `around:` is what turns that mutant's hang into a flunk). A cancel in flight while the exchange finishes on its own never raises back into the canceller: `Source#cancel` steals its hooks and runs them outside its mutex, so the adapter's hook can run after check-after-resume has settled the pivot and closed the queue, and `Exchange#signal` swallows the `ClosedQueueError` that push would otherwise hand back through `Hooks.notify` to the caller's own `Source#cancel` ("a token cancel in flight while the exchange finishes never raises out of Source#cancel on the canceller's thread, and the future is cancelled", deterministic through an ordinary caller hook registered first; review round 2's R2-1, guard 51). Guards 15, 16, 17, 37 and 51. | +| `TRANSPORT-8` | MUST | ✅ | 13, 19 | **Satisfied on this adapter, where §12 records it vacuous** (`R14`): a cancellation the host runtime originates — a parent task cancelled while the exchange is its live child — arrives as `Async::Cancel`, leaves `Exchange#run` through its `rescue ::Exception` arm after settling the pivot with `request_cancel(:async_cancelled)`, and surfaces as a terminal, non-retryable `CancelledError` with the future reading `cancelled?`; a `with_timeout` expiry on the **same** withheld-head path settles a retryable `TransportError` carrying `Async::TimeoutError`, discriminated by class and never by message (`async_http/parent_cancellation_test.rb`, "TRANSPORT-8: cancelling a PARENT task…" and "TRANSPORT-8's pair…"; `async_http/adapter_test.rb` `TimeoutTest` "TRANSPORT-4/TRANSPORT-8's pair"). Not a portable assertion: its antecedent is a cancellation only an adapter's own suite can originate, and `TransportSuite::PREAMBLE` says so in every report (`conformance/transport_suite/lifecycle_test.rb` and `asynchronous_test.rb` assert the absence and the sentence). Guards 14 (equivalent, measured), 19 and 35 (a runtime cancellation landing inside an exit arm — the native close of an undelivered body suspending, the parent cancelled there — is settled by `#run`'s `ensure` net, `parent_cancellation_test.rb` "a runtime cancellation landing inside an exit arm's native close still settles the pivot cancelled, through the ensure's net", review round 0's R0-4). | +| `TRANSPORT-9` | MUST | ✅ | 11, 12 | A native response obtained after the token was cancelled mid-flight is closed exactly once and never delivered: the exchange re-checks the token after `Client#call` returns (`Exchange#perform`'s `check!`), and independently core's `Completer#fulfil` closes a response handed to an already-settled pivot — so the close holds even without the check, which guard 18 measures (`async_http/cancellation_test.rb`, "TRANSPORT-9: a native response obtained after the token was cancelled mid-flight is closed exactly once"; the portable assertion `asynchronous_test.rb` "TRANSPORT-9", failing against the `ignore_cancel` defect; `async_http/parent_cancellation_test.rb` "R13" for the two paths that reach the undelivered close). | +| `TRANSPORT-12` | MUST | ✅ | 9, 15, 19 | The RFC 7230 token predicate (`HeaderSyntax.token?`) is applied **before dispatch on both protocols** (`P8-40`): a name the SDK model admits and the grammar refuses is dropped, reported through `DropPolicy`, and the rest of the headers and the body dispatch — measured over HTTP/1.1, plaintext HTTP/2 and TLS HTTP/2 against the in-process `async-http` server (`async_http/wire_grammar_test.rb` `TokenPredicateTest`, one test per protocol shape), with the antecedent measured on `protocol-http1` 0.41.0 (a `RefusedError` wrapping `BadHeader`, **after** the request line is on the wire) and on `protocol-http2` 0.28.0 (the name transmitted lowercased and unvalidated). The seventeen bytes `HTTP-17` admits and the grammar refuses — `"(),/:;<=>?@[\]{}` — are asserted one by one (`async_http/request_mapper_test.rb` `HeaderGatesTest` "TRANSPORT-12: the seventeen bytes…"). The portable assertion `conformance/transport_suite/header_drops_test.rb` "TRANSPORT-12" passes against `RawWireTransport`'s drop mode, fails against its `refuse_non_token` defect, resolves **vacuous by measurement** against the plain double (and against `Net::HTTP` through 8a's driver, which now carries the row as a skip), and is real here. Guards 1, 2, 6. | +| `TRANSPORT-13` | SHOULD | ✅ | 7, 9, 15, 19 | `DropPolicy` with the closed three-mode set `EVERY`, `ONCE_PER_NAME` (the default) and `QUIET`, `DropPolicy.build(mode:)` refusing anything else; the per-name latch is keyed on the **folded** name (`HTTP-13`), warns once and is quiet (verbose) afterwards, and is bounded at `MAX_TRACKED_NAMES` (64) distinct names, beyond which every drop is quiet — one frozen `Snapshot` replaced under the policy's own mutex, never a growing map (`async_http/drop_policy_test.rb`, nine cases); a real dispatch through the default policy warns once per distinct bad name across three requests (`async_http/wire_grammar_test.rb` "TRANSPORT-13: a real dispatch through the default policy…"); the framing set is reported at verbose on a **separate** path and never through the policy (`request_mapper_test.rb` "TRANSPORT-11"). This is the header-drop policy phase 5b postponed to phase 8 as `OBS-19`, landed. The portable assertion `header_drops_test.rb` "TRANSPORT-13" is proven in both directions and vacuous by measurement against a client that sends the name. Guards 3, 4a, 4b, 5. | +| `TRANSPORT-21` | MUST | ✅ | 11, 16, 19 | Every failure before dispatch is delivered through the returned future and never thrown: a call outside a reactor settles `Dexpace::SeamError` carrying `REACTOR_MESSAGE` (`P8-39`), a send after close on an owning adapter settles `ClosedError`, a header the re-validation refuses settles `InvalidArgumentError`, an `ftp://` URL settles `InvalidArgumentError` from `Endpoints.screen!` before anything is dialled, an already-cancelled token settles a cancellation before anything is mapped, and a native failure raised inline settles a retryable `TransportError` — all through `Adapter#dispatch`'s one `rescue ::StandardError` fence into `Errors.settle` (`async_http/adapter_test.rb` `PreDispatchTest`, eight tests; the portable assertion `asynchronous_test.rb` "TRANSPORT-21", failing against the `bare_adaptation` defect). Guards 28, 29 (equivalent by construction), 29b, 30. | +| `TRANSPORT-23` | MUST | ✅ | 11, 16, 19 | The pivot settles with the `Dexpace::Response` `ResponseMapper.call` built and nothing else, for a 200 with a body and a 204 with none alike; the mapper hands on the native **body** (`Protocol::HTTP::Body::Readable`), never the response, whose `#read` would be the whole body joined (`async_http/adapter_test.rb` "TRANSPORT-23: a successful dispatch never settles with a nil response"; `async_http/dispatch_conformance_test.rb` `DeliveryTest` "TRANSPORT-23… through a real reactor"; `async_http/response_mapper_test.rb` `BodyTest` "the body handed on is a ResponseBody over the native BODY"; the portable assertion `asynchronous_test.rb` "TRANSPORT-23", failing against the `null_success` defect). Guard 24b. | +| `ASYNC-6` | MUST | ✅ | 11, 12, 13 | Both directions through the queue-marshalled bridge (`P8-91`): the token's hook settles the pivot cancelled and pushes its reason, `Future#cancel` settles the pivot whose own `on_cancel` hook pushes, and the watcher task acts on the reactor's thread (`async_http/cancellation_test.rb` "TRANSPORT-7/ASYNC-6" and "ASYNC-6: cancelling the future reaches the native exchange"); the runtime's own cancellation of the exchange settles the pivot (`parent_cancellation_test.rb` "TRANSPORT-8"); and a cancellation is settled through `#request_cancel`, never `#fail`, so `Future#cancelled?` reads true (`async_http/errors_test.rb` "settle routes a cancelled token to #request_cancel"; guard 17). Both hooks push through `Exchange#signal`, which is total over the exchange's end: the source and the completer each steal their hook list under their mutex and run it outside, so a hook can run after the exchange has settled the pivot itself and closed the queue, and the `ClosedQueueError` that push raises is swallowed there rather than handed back to the caller's `Source#cancel` or `Future#cancel` ("a token cancel in flight while the exchange finishes…", guard 51; review round 2's R2-1). The `CancelledError` the watcher wraps into `Task#cancel(cause:)` is for whoever reads the cancelled task — a non-Exception cause is replaced by the runtime's own — and is read back by nothing here: the pivot's reason travels through the token and the completer, both settled before the watcher acts, which is why dropping the wrap is an equivalent mutant (guard 46; R2-4). The design's direct `#cancel` from the hook is superseded: `Async::Task#cancel` from a foreign OS thread raises `NoMethodError` and cancels nothing (matrix fact, `async_http/matrix_facts_test.rb`). | +| `ASYNC-21` | MUST | N/A | 10 | §11.21's reactive SSE bridge is `dexpace-async-thread`'s and the reactive form post-v1 (`docs/first-release.md`); the one property of it this adapter can honour — the source is polled at most once per unit of demand, never eagerly — is asserted on `ResponseBody` on 7b's precedent: one native `#read` per yield and nothing read ahead (`async_http/response_body_test.rb` `PullAndCloseTest` "ASYNC-21's property"). | +| `ASYNC-22` | MUST | ✅ | 8, 11, 16 | Structural: nothing per-call lives on the adapter — the ivar set is pinned (`async_http/adapter_test.rb` "ASYNC-22: nothing per-call lives on the adapter"), every per-call value is on the `Exchange` and the `Completer`; and the client map is keyed by **(reactor, origin)** (`P8-92`), so every OS thread running its own reactor gets its own `Async::HTTP::Client` per origin and fibers inside one reactor share one — sixteen concurrent calls through one owning adapter over HTTP/1.1 (pool ≤ 8) and over TLS HTTP/2 (one multiplexed connection) each resolve to their own response, and two threads in two reactors through one adapter mismatch nothing (`async_http/dispatch_conformance_test.rb` `DeliveryTest`, "ASYNC-22 over http1", "ASYNC-22 over tls", "ASYNC-22 across threads"; `async_http/clients_test.rb` `FetchTest` "the same origin under a different reactor is a different client"). Guards 11, 12a, 12b. | + +**Cross-reference rows** — IDs this phase exercises and does not own, one line each, none counted +above: + +| ID | Owner | What this phase adds | +|---|---|---| +| `HTTP-17`, `HTTP-18`, `XCUT-18` | phase 1 (postponed to the adapters) | The wire-boundary re-validation's **second** call site: `RequestMapper#revalidate!` runs `HeaderSyntax.validate_name!` and `.validate_outbound_value!` before anything is mapped, on every name and value, so a forged request that met no builder is refused through the future (`request_mapper_test.rb` "HTTP-17", "HTTP-18"; `adapter_test.rb` "TRANSPORT-21 / HTTP-17"), and over HTTP/2 — where nothing below the model validates and a CRLF value reaches the peer verbatim without it — the h2 half is measured against the in-process server (`wire_grammar_test.rb` `RevalidationTest`, with its antecedent). **With 8a's Task 16 already on the tree, the re-validation phase 1 postponed has landed in both adapters** (the roadmap's status note says so); phase 9's Task 7 adds the portable assertion beyond 8a's two. Guard 1. | +| `XCUT-14` | phase 9 | The second bounded map: `Clients` at `MAX_ORIGINS` (32), drained back to the cap in a loop after every insert — closed reactors first, then the oldest — with every evicted client's pool retired and closed; the `gates:bounded_map` blocker in `docs/first-release.md` gains its dated status and stays open only for the gate itself, which phase 9 builds (`clients_test.rb` `ReleaseTest`, three "XCUT-14" cases). | +| `TRANSPORT-1`, `-2`, `-10`, `-11`, `-14`, `-15`, `-16`, `-17`, `-18`, `-19`, `-20`, `-22`, `-24`, `-25`, `-26`, `-27`, `-28`, `-29` | phase 8a | The **second driver**: `async_http/conformance_test.rb` runs the whole shared suite — thirty-four assertions — against the real adapter through `MinitestDriver` with `settle:` (call, then await the future; from a thread of the assertion's own, a reactor per settle with the body materialised inside it, `P8-94`), `around:` (a `Sync` running the assertion as a child task under a thirty-second bound kept on the PARENT's wait, `AROUND_BOUND`, which cancels the child and flunks the row by name when it expires — a bound raised into the assertion's own fiber would meet the token-first classifier and pass the cancellation rows late; review round 1's R1-1) and `borrow:` (a caller's own `Async::HTTP::Client` with `retries: 0`), waiving `TRANSPORT-14` and `TRANSPORT-27` by id (`P8-38`; both heads refused out of the read by `protocol-http1`, measured as retryable `TransportError`s carrying `Protocol::HTTP1::Error` in the driver's own second test) — **four skips**: three waived (two assertions carry `TRANSPORT-14`) and `TRANSPORT-18` vacuous by measurement, exactly as on 8a's adapter. `TRANSPORT-2`: every owned client is built with `retries: 0` and a borrowed one is refused unless it already is (`clients_test.rb`, `adapter_test.rb`). `TRANSPORT-11`: the ten folded `FRAMING_HEADERS` are never copied, because `async-http` appends a caller's `Host` beside its own (`request_mapper_test.rb`, two "TRANSPORT-11" cases). `TRANSPORT-10`/`-26`: the explicit header, then the body's media type, then **nothing** — no octet-stream default, `P8-97`; and dispatch step 8's "a body-forbidden method gets no body attached" is proven over a forged GET and HEAD carrying a body, the only shape that reaches the guard since `HTTP-7` makes a built GET's body nil (`request_mapper_test.rb` "a body-forbidden method attaches none, even on a forged request carrying one"; guard 47v, review round 2's R2-2). `TRANSPORT-14`'s value clauses and `TRANSPORT-24`/`-27` on `ResponseMapper` (`response_mapper_test.rb`, fourteen cases). `TRANSPORT-15`/`-16`: `P8-37` as built — retire every pooled resource, then `pool.close`, never `Client#close`; a close under an open response returns at once (`clients_test.rb` `ReleaseTest`; `adapter_test.rb` `ConstructionTest`). `TRANSPORT-20`/`-4`/`-3`: `Errors.wrap` asks the token first and wraps every native family retryable with `#cause` (`errors_test.rb`). `TRANSPORT-22`: an adaptation failure after the head closes the native body exactly once (`parent_cancellation_test.rb` `CloseDisciplineTest`). `TRANSPORT-30` stays 8a's and ⏳ there (no proxy route exists in `async-http` for this adapter to carry). | +| `ASYNC-7` | phase 8b | This gem's half of §3.3's contrast: the README's `ASYNC-7` section states the sentence verbatim, and `dispatch_conformance_test.rb` `CompositionTest` "ASYNC-7" measures a blocked read abandoned at the next scheduler checkpoint, well under the time the response would have taken. | +| `OBS-19` | phase 5b (postponed to phase 8) | **Landed**: `DropPolicy` (Task 7), the predicate at dispatch (Task 9), the both-protocols dispatch test (Task 15), the antecedent confirmed on `protocol-http1` 0.41.0's grammar. | +| `OBS-29`, `OBS-28` | phase 5c / phase 10 | Not wired, and no route exists: the adapter takes a `logger:` and no tracer (8a's `R6`, `P8-7`, on phase 10's inbound list); stated in the as-built page. | +| `SEAM-5`, `SEAM-6`, `SEAM-16` | phase 2 | The require-time `AsyncTransport.register(:async_http, …, core: "~> 0.0")` — the version-skew guard's third real registration — and its consequence for core's suite: the in-process "starts empty" and "nothing resolved after a swap" pins on the async seam moved to a child process (`core/async_transport_bare_require_test.rb`, 8a's shape on the sync seam). | +| `SEAM-24` | phase 2 / post-v1 | First sentence by construction (the adapter is a `Dexpace::AsyncTransport`); the second sentence's cancellation bridge to a third-party runtime stays post-v1 per the phase-8 charter. | +| `CFG-7` | phase 5a | `REQUEST_TIMEOUT` is read through `Configuration#duration`, so a bare number is milliseconds; `TRANSPORT_CONNECTION_LIMIT` — the one core widening this phase adds, `Configuration::Keys`' tenth key — through `#integer` (`adapter_test.rb` `TimeoutTest`; `clients_test.rb` `FetchTest`, two configuration cases). | +| `NFR-2`, `NFR-3`, `NFR-11`, `NFR-13` | phase 9 | The gemspec declares `dexpace-core` and `async-http ~> 0.104` and nothing else, and `gates:gemspec_audit` reads it; every file mirrored in `sig/` with no `Async::`, `Protocol::` or `OpenSSL::` type in any signature (`gates:rbs_surface`), the `:async_http` Steep target relaxed exactly as `:serde_json` is (`P8-99`); every file opens with the SPDX header. | +| `NFR-10`, `NFR-14` | phase 0 / phase 9 | The per-gem Ruby floor: `VERSIONS` gains `ruby floor:dexpace-transport-async_http 3.3`, `DexpaceVersions.ruby_floor(gem)` and `.gem_supported?` read it, `gates:versions` and `gates:gemspec_audit` assert it per gem, and the `Gemfile`, `test:gems` and `gates:clean_bundle` skip the gem on a row below its floor (`test/gates/versions_gate_test.rb` and `gemspec_audit_test.rb`'s `per_gem_floor_ahead` fixtures; `P8-36`, and `P8-95` for the shape). | + +## What was built + +**`dexpace-core`**: one constant, `Configuration::Keys::TRANSPORT_CONNECTION_LIMIT`, with its `sig/` +line; three pins moved on the code branch (`core/configuration/keys_test.rb`, ten keys, and +`core/instrumentation/downstream_wirings_test.rb`, ten), and the four in-process registry pins on the +async seam that the require-time registration invalidated converted to a child-process suite, +`core/async_transport_bare_require_test.rb` over the shared `BareRequire` helper, with +`core/async_transport_test.rb` keeping the in-process halves — 8a's conversion of the sync seam, +applied to the other one. + +**`dexpace-transport-async_http`**: `lib/dexpace/transport/async_http.rb` gains its six constants +(`DEFAULT_TIMEOUT_SECONDS`, `DEFAULT_CONNECTION_LIMIT`, `MAX_ORIGINS`, `REGISTRY_KEY`, +`FRAMING_HEADERS`, `ALPN_PROTOCOLS`), `.build(timeout:, logger:, drop_policy:, connection_limit:, +ssl_context:, configuration:)`, `.using(client, logger:, drop_policy:)`, `.default` and the +registration; ten new files under `async_http/` — `adapter.rb` and `drop_policy.rb` public, and the +eight `private_constant`s `clients.rb`, `endpoints.rb`, `errors.rb`, `exchange.rb`, `request_body.rb`, +`request_mapper.rb`, `response_body.rb`, `response_mapper.rb` — every one mirrored in `sig/` (with the +`_Release` interface for the body's release hook) and every one but `exchange.rb` in `test/`; the +gemspec declares `async-http ~> 0.104` and `required_ruby_version >= 3.3` read from `VERSIONS`' +per-gem row. Its `test/support/` holds eight doubles and fixtures, every top-level name prefixed +`AsyncHTTP` because `test:gems` loads every gem's suite into one process: `AsyncHTTPRecordingSink`, +`AsyncHTTPRecordingBody`, `AsyncHTTPHoldingServer`, `AsyncHTTPSilentServer`, `AsyncHTTPServerFixture` +(an in-process `async-http` server over HTTP/1.1, plaintext prior-knowledge HTTP/2 and TLS HTTP/2 by +real ALPN over a per-run self-signed certificate, wrapping the library's server in a `QuietServer` +that swallows a peer's mid-head EOF), `AsyncHTTPReactor` (`reactor_over(server)` and +`assert_exchange_released(task)`), `AsyncHTTPHermeticConfiguration` (`hermetic_configuration(overrides)`, +a chain whose environment tier answers nothing — review round 0's R0-1) and the repository-level +`test/support/async_http_warmup.rb`. +Sixteen suites: the smoke suite, `matrix_facts_test.rb`, the ten unit suites, and the four behavioural +ones — `cancellation_test.rb`, `parent_cancellation_test.rb`, `dispatch_conformance_test.rb`, +`wire_grammar_test.rb` — plus the second driver, `conformance_test.rb`. + +**`dexpace-conformance`**: two new `private_constant` groups and their suites — `Asynchronous` +(`TRANSPORT-7`, `-9`, `-21`, `-23`) and `HeaderDrops` (`TRANSPORT-12`, `-13`) — so the suite is +thirty-four assertions in seven groups with `TransportSuite::PREAMBLE` naming `TRANSPORT-8` as the +third thing a green run does not prove; `Scripts.write_response` gains `close:` and every head a script +writes carries `Connection: close` (`P8-96`); `RawWireTransport` gains six defects and a drop mode; +`lifecycle_test.rb`'s size pin moves to thirty-four in seven. + +**`dexpace-transport-net_http`**: two test-side changes only — `adapter_fixtures.rb`'s +`keep_alive_twice` passes `close: false` for its first response, and `conformance_test.rb`'s +generated-test pin moves to thirty-four with `TRANSPORT-12` and `TRANSPORT-13` now present (and +vacuous by measurement there). + +**The repository**: `VERSIONS`' `floor:` grammar and row; `tools/versions.rb`'s `ruby_floor(gem)` +and `gem_supported?`; `tools/versions_gate.rb` and `tools/gemspec_audit.rb` reading the per-gem floor; +the `Gemfile`, `tasks/quality.rake`'s `supported_gem_dirs` and `tasks/gates.rake`'s `clean_bundle` +skipping an unsupported gem; the two gate fixtures under `test/fixtures/gates/{versions,gemspec_audit}/ +per_gem_floor_ahead/`; the `Steepfile`'s `:async_http` target with `library "openssl", "uri"` and the +`UnknownConstant` relaxation; three surface manifests regenerated once — core 1 335 → 1 336 +(`TRANSPORT_CONNECTION_LIMIT`), `async_http` 2 → 25, `conformance` unchanged (both new groups are +private) — with every added row read against the object model. + +## Matrix facts, re-run on every interpreter + +The design's thirteen facts and the plan's were run on 2026-09-21 on 3.3.12, 3.4.10 and 4.0.6 against +the bundle's `async-http` 0.105.0 / `async` 2.46.0 (the design measured 0.104.0), first as a scratch +script and then, for the eight the adapter's shape rests on, as `async_http/matrix_facts_test.rb` on +every row, which prints the row's versions. The 3.2.11 row has no bundle for this gem (`P8-36`; the +whole closure declares `>= 3.3`, and `bundle install` there refuses it), which the per-gem floor +machinery turns into a skipped gem rather than a red row. What holds identically on every row: +`Async::Cancel < Exception` outside `StandardError` and `Async::TimeoutError < StandardError`, with +`Async::Stop` the same class; outside a reactor `Task.current?` and `Fiber.scheduler` are nil; +`Task#cancel` from a foreign OS thread raises `NoMethodError: private method 'raise' called for nil` +and cancels nothing; `Task#cancel(cause:)` keeps an `Exception` cause and replaces a Symbol with the +runtime's own `Cancel::Cause`; `Task#async` runs the child eagerly to its first suspension and hands +control back to the caller's fiber; `Fiber.scheduler` is one object across a reactor's tasks and a +different one on another thread, closed once its `Sync` returns; `Client.new(endpoint, retries: 0, +limit:)` opens no socket and a caller's `ssl_context` reaches the endpoint verbatim; +`Async::HTTP::Endpoint` builds for an `ftp` URL, so the scheme screen is the adapter's; the response +body is lazy, pull-shaped, BINARY and unfrozen; `Protocol::HTTP::Headers#add` keeps a duplicate name. +**What the design stated that 2.46.0 does not bear out**, each a ledger row or a note: `Kernel#Async` +inside a running task **is** that task's child (the design's fact 10 said otherwise; guard 14 is +therefore equivalent), and `cause:` does not carry a Symbol (the design's fact 8). **What differs by +row**: on 4.0.6 alone the first `IO::Buffer` under a scheduler prints Ruby's once-per-process +experimental warning, parked by `test/support/async_http_warmup.rb` (`P8-98`); on 3.3.12 the bundle +must carry a compiled `openssl` 4.0.2 because the interpreter's own 3.2.4 is older than `io-stream` +requires, on 3.4.10 the interpreter's 3.3.3 would satisfy it and the bundle used here resolved the +installed 4.0.2 anyway (`matrix_facts_test.rb` prints "installed gem" on both rows), and 4.0.6's own is +4.0.2 (its row prints "installed gem" too, for the reason 8a's checklist gives about this machine's +gem directories). + +## Guards run red + +The reviewer's thirty mutations were run one at a time through a harness that applies the edit, runs +the owning suites under `ruby -w`, captures the first failure and restores the file — on **4.0.6 and +3.3.12** (the gem's floor row; 3.2.11 has no bundle for it). **Thirty-four of the thirty-six rows +they make (the a/b splits counted) are red on both interpreters, and the two equivalent mutants +are recorded with their measurement.** Five guards the +first pass found missing were added before the second pass and are what make rows 11, 13a, 13b, 15, +16, 20 and 27b red rather than surviving or hanging: the mutex scan reaches `build_client`; +`assert_exchange_released` proves the watcher is gone after an undelivered settlement; `reactor_over` +closes a holding fixture *inside* the reactor so an exchange a defect left blocked is released and the +failed assertion surfaces instead of the reactor waiting forever; every wait in a cancellation or +timeout test is bounded (`value_within`, `with_timeout`) so a missing timeout or a lost cancel is a +failed assertion, not a hang; and `TRANSPORT-3`'s test feeds the classifier the SDK's own errors under +a cancelled token. Review round 0 ran six mutations of its own beyond the thirty; two survived and +are rows 34 and 35 below, each made red on 2026-09-21 by the guard its row names. Review round 1 re-ran +every row and seven extras of its own: five caught by existing tests (its 36, 39, 41, 42 and 43), one +equivalent by measurement (its 40 — `RequestBody#read`'s `String#b` is redundant behind 3a's ingress +retag) and one surviving, row 37 below, made red the same day by the cause the body-path test now +refutes. Review round 2 re-ran every row and seven extras of its own: three caught by existing +tests (its 45, 48v and 49), one equivalent by measurement (its 46, row 46 below — the +`CancelledError` wrapped into `Task#cancel(cause:)` is read back by nothing, and its +documentation was the fix), and three surviving, rows 44, 47v and 51 below, each made red on +2026-09-21 by the guard its row names — **forty of forty-three rows red**, three equivalent. + +| # | Mutation | Caught by (first failure) | Rows | +|---|---|---|---| +| 1 | `revalidate!` skipped | `request_mapper_test.rb` `HeaderGatesTest` "HTTP-17: a header name HeaderSyntax rejects raises before anything is mapped" (no raise), "HTTP-18"; `wire_grammar_test.rb` `RevalidationTest` (the CRLF value reaches the h2 peer) | 4.0.6, 3.3.12 | +| 2 | the token predicate replaced by `HeaderSyntax.valid_name?` | `wire_grammar_test.rb` `TokenPredicateTest` over http1 (`RefusedError` through the future, no 200), over plaintext and tls (`x-bad:name` in `received`); `request_mapper_test.rb` "TRANSPORT-12/13, P8-40" | 4.0.6, 3.3.12 | +| 3 | the latch keyed on the raw name | `drop_policy_test.rb` "the per-name latch is case-insensitive on the folded name (HTTP-13)" (`[:warn, :warn]`) | 4.0.6, 3.3.12 | +| 4a | `MAX_TRACKED_NAMES` bound removed | "bounded at MAX_TRACKED_NAMES distinct names; the next degrades to quiet" (`:warn` for the 65th) | 4.0.6, 3.3.12 | +| 4b | `DropPolicy.build(mode: :bogus)` accepted | "a rejected mode raises Dexpace::InvalidArgumentError rather than degrading silently" | 4.0.6, 3.3.12 | +| 5 | a framing drop routed through the policy | `request_mapper_test.rb` "TRANSPORT-11: the ten framing headers are dropped and logged verbose" (`:warn` where `:debug` was expected) | 4.0.6, 3.3.12 | +| 6 | `FRAMING_HEADERS` loses `host` | "TRANSPORT-11: the drop set is exactly the ten folded names", "a body maps to a RequestBody…; framing is never copied" (a second `host:`) | 4.0.6, 3.3.12 | +| 7 | `Endpoints.for` through `Endpoint.parse(url.to_s)` | `endpoints_test.rb` "never calls Endpoint.parse or URI.parse anywhere under lib/" (the source scan) | 4.0.6, 3.3.12 | +| 8 | the default context drops `alpn_protocols` | `endpoints_test.rb` "the adapter-supplied ssl_context offers h2 and http/1.1 by ALPN" — the default context's ALPN is asserted at unit level only, because `dispatch_conformance_test.rb`'s tls variant builds its adapter with the fixture's own caller `ssl_context`, a context that never reaches the default line | 4.0.6, 3.3.12 | +| 9 | the default context at `VERIFY_NONE` | `endpoints_test.rb` "an https URL always gets an adapter-supplied ssl_context that verifies the peer" | 4.0.6, 3.3.12 | +| 10 | `retries: 0` dropped from `build_client` | `clients_test.rb` `FetchTest` "every client disables the native retry loop (TRANSPORT-2, 17, 18)" | 4.0.6, 3.3.12 | +| 11 | the client built inside the mutex | `clients_test.rb` "the client is built outside the mutex: no Endpoints call inside a synchronize block" (the scan now reaches `build_client`; the first pass survived) | 4.0.6, 3.3.12 | +| 12a | the `MAX_ORIGINS` drain removed | "XCUT-14: the map is bounded at MAX_ORIGINS…" (33), "…a client whose reactor has closed is evicted before a live one", "…an evicted client's pool is retired and closed" | 4.0.6, 3.3.12 | +| 12b | an evicted client dropped, never retired | "XCUT-14: an evicted client's pool is retired and closed, never merely dropped" | 4.0.6, 3.3.12 | +| 13a | `Clients.release` through `Client#close` | `clients_test.rb` `ReleaseTest` "close returns at once with a response still open" (the bounded close flunks: "close waited on the open response instead of retiring it (P8-37)"), "close retires every pooled resource… never through Client#close" — the first pass **hung** until the test released the open response inside the reactor | 4.0.6, 3.3.12 | +| 13b | `pool.close` without retiring busy resources first | the same two, the drain waiting on the busy connection | 4.0.6, 3.3.12 | +| 14 | the exchange spawned with `Async { }` instead of `caller_task.async` | **Equivalent on async 2.46.0**: `Kernel#Async` inside a running task delegates to `Task.current.async`, so the exchange is the supervisor's child either way (`inner.parent.equal?(task)` measured true; `matrix_facts_test.rb` "P8-39 fact: Task#async runs the child eagerly…" and the design's fact 10 corrected in the knowledge note); `parent_cancellation_test.rb` stays green, honestly | measured on 4.0.6, 3.3.12 | +| 15 | the token hook cancels the exchange directly | `cancellation_test.rb` "a token cancelled from a foreign OS thread still reaches the exchange, promptly" (`NoMethodError: private method 'raise' called for nil` out of the hook on the canceller's thread) — the first pass hung after the error until the body-path read was bounded | 4.0.6, 3.3.12 | +| 16 | `queue.close` missing from `release_watch` | `parent_cancellation_test.rb` "TRANSPORT-8's pair", `adapter_test.rb` `TimeoutTest` — "the exchange task or its watcher outlived the settlement by 10 turns" (the first pass survived: nothing asserted the watcher's release) | 4.0.6, 3.3.12 | +| 17 | a cancellation settled through `#fail` | `errors_test.rb` "settle routes a cancelled token to #request_cancel and everything else to #fail" (`Future#cancelled?` false); `adapter_test.rb` "an already-cancelled token settles a CANCELLATION before anything is mapped or sent" (the `check!` raise goes through `#dispatch`'s fence into `Errors.settle`) | 4.0.6, 3.3.12 | +| 18 | check-after-resume removed | `cancellation_test.rb` "a token cancel in flight while the exchange finishes never raises out of Source#cancel on the canceller's thread, and the future is cancelled" — with the check gone the fake 204 is delivered once the flag is up and the `Dexpace::CancelledError` the test expects is never raised (the second 4.0.6 failure is the knock-on leaked canceller thread); the close count measured separately stands: core's `Completer#fulfil` closes a response handed to an already-settled pivot (`close_quietly` inside `fulfil`, phase 2), so `TRANSPORT-9`'s native body is closed exactly once either way and the check is a shortcut past the mapping, not the guarantee | 4.0.6, 3.3.12 | +| 19 | the undelivered native body not closed on `finish` | `parent_cancellation_test.rb` `CloseDisciplineTest` "TRANSPORT-22: an adaptation failure after the head closes the native body exactly once" (0), "R13" | 4.0.6, 3.3.12 | +| 20 | the per-call `with_timeout` removed | `adapter_test.rb` `TimeoutTest` "TRANSPORT-4/TRANSPORT-8's pair" (a `:deadline_expired` cancellation where a `TransportError` was expected), `parent_cancellation_test.rb` "TRANSPORT-8's pair" — the first pass hung until the waits were bounded | 4.0.6, 3.3.12 | +| 21 | `ResponseBody#each` delegating to the native `#each` | `response_body_test.rb` `PullAndCloseTest`, six failures: the double close, the read-after-close, the release hook's count | 4.0.6, 3.3.12 | +| 22 | `#source` not memoised | "#source is built with BufferedSource.over and is the same handle every call (BODY-14)", "reading through #source twice continues where the first read stopped" | 4.0.6, 3.3.12 | +| 23 | the mid-stream `StreamError` classification removed | `ReadSurfaceTest` "P3-3: a native failure mid-stream surfaces as StreamError with the cause, and closes" (a bare `EOFError` escapes) | 4.0.6, 3.3.12 | +| 24a | the mapper reads the response's length | `response_mapper_test.rb`, ten errors (`NoMethodError`) and the length test | 4.0.6, 3.3.12 | +| 24b | a nil native body (204) unhandled | `BodyTest` "a response the library delivers with no body — a 204 — gets body nil" (`NoMethodError` on nil) | 4.0.6, 3.3.12 | +| 25 | `media_type_for` loses its rescue | "TRANSPORT-27: a malformed Content-Type downgrades to no media type" (`InvalidArgumentError` escapes) | 4.0.6, 3.3.12 | +| 26 | `inbound_headers` loses the per-value guard | `HeadTest` "TRANSPORT-14: a control byte in an inbound value is dropped, that header only" (`Headers::Builder#add` raises `HTTP-19`) | 4.0.6, 3.3.12 | +| 27a | `Errors.wrap` re-wraps a `Dexpace::` error | `errors_test.rb` "a Dexpace:: error is passed through unwrapped, unchanged" (`assert_same`) | 4.0.6, 3.3.12 | +| 27b | `Errors.wrap` consults the class before the token | "TRANSPORT-3: asks the cancellation token first… an IOError and a Dexpace:: error included" (a `ClosedError` under a cancelled token passed through) — the first pass survived because the list held native families only | 4.0.6, 3.3.12 | +| 28 | the reactor check removed | `adapter_test.rb` `PreDispatchTest` "TRANSPORT-21: calling outside a reactor settles a SeamError through the future" (`RuntimeError: No async task available!` wrapped retryable instead) | 4.0.6, 3.3.12 | +| 29 | the post-close guard raises inside `#dispatch` | **Equivalent by construction**: `Adapter#dispatch`'s one `rescue ::StandardError` fence settles the raise through the future, so the guard's channel is structural (`TRANSPORT-21`); measured green | measured on 4.0.6, 3.3.12 | +| 29b | the post-close guard raises from `#call`, outside the fence | "a send after close settles ClosedError through the future on an owning adapter" (a synchronous `ClosedError`) | 4.0.6, 3.3.12 | +| 30 | `Endpoints.screen!` admits every scheme | `endpoints_test.rb` "a scheme other than http or https is refused as InvalidArgumentError, not dialled"; `adapter_test.rb` "TRANSPORT-21: a URL the endpoint cannot dispatch settles InvalidArgumentError" (a `TransportError` from dialling port 21 instead) | 4.0.6, 3.3.12 | +| 34 | `ResponseBody#each` yields the chunk without the `String#b` retag (review round 0's R0-3) | `response_body_test.rb` "#each yields one native #read per chunk, retagged BINARY, and stops at nil" — the first chunk the double hands over is now a UTF-8 literal, so the yielded encodings `[UTF-8, BINARY]` fail the `[BINARY, BINARY]` pin; the round-0 fixture fed BINARY chunks alone and the mutant survived it | 4.0.6, 3.3.12 | +| 35 | `Exchange#net`'s `request_cancel(:async_cancelled) unless settled?` removed (review round 0's R0-4) | `parent_cancellation_test.rb` `CloseDisciplineTest` "a runtime cancellation landing inside an exit arm's native close still settles the pivot cancelled, through the ensure's net" — the adaptation-failure arm is parked inside a native `#close` that waits on a queue, the parent is cancelled there, and with the net gone the future stays pending until the bounded wait expires (`:deadline_expired` where `:async_cancelled` is pinned, 5.2 s); no fixture reached the path before this case, and no sleep is involved | 4.0.6, 3.3.12 | +| 37 | `Exchange#watch` no longer closes the DELIVERED response on a cancel (review round 1's R1-1) | `cancellation_test.rb` "TRANSPORT-7 on the body path: a token cancelled under a blocked body read wakes the reader with CancelledError and releases the body" — `refute_kind_of(::Async::TimeoutError, error.cause)`: with the close gone the read is woken by the test's own five-second bound, the token-first classifier still answers `CancelledError(:reader_cancelled)` with the body closed, and only the cause (`Async::TimeoutError` where the watcher's close leaves a bare `IOError`) tells the two wakes apart — the round-1 fixture passed in 5.0 s where the real code takes 16 ms. The same mutant hung the portable `TRANSPORT-7` row under the driver whenever the race fell on the body path (half the runs on 4.0.6, a third on 3.3.12; the rest take the in-flight path and pass) until `around:` bounded the assertion from a parent task: a flunk at thirty seconds now on both rows, no hang, nothing on stderr | 4.0.6, 3.3.12 | +| 44 | the watcher spawned without `transient: true` (review round 2's R2-3) | `cancellation_test.rb` "ASYNC-20: cancelling the future after delivery does not close the delivered response; its watcher stays, transient, until the body is released" — the watcher's `transient?` is read while the body is open and asserted after its release (`[false]` where `[true]` is pinned, 18 ms); before this guard the mutant surfaced only as `adapter_test.rb`'s borrowing case holding its reactor open until the run was killed, because a non-transient watcher under a never-closed body keeps `Sync` from returning | 4.0.6, 3.3.12 | +| 46 | `Task#cancel(cause: reason)` instead of `cause: CancelledError.new(reason)` (review round 2's R2-4) | **Equivalent, measured**: `cancellation_test.rb` and `parent_cancellation_test.rb` stay green (12 runs) because the pivot is settled with the reason before the watcher acts — the token's hook settles it before it pushes, `Future#cancel` settled it to run its hook at all — so `#run`'s exit arm never reads the `Cancel`'s cause; the wrap stays for whoever reads the cancelled task (a Symbol is replaced by `Async::Cancel::Cause`) and the record says so instead of calling it load-bearing | measured on 4.0.6, 3.3.12 | +| 47v | `RequestMapper#body_for`'s `body_forbidden?` guard removed (review round 2's R2-2) | `request_mapper_test.rb` "a body-forbidden method attaches none, even on a forged request carrying one" (a `RequestBody` where nil is pinned, for the forged GET) — the former assertion built its GET through the model, whose `HTTP-7` had already made the body nil, so the guard was reachable by no test | 4.0.6, 3.3.12 | +| 51 | `Exchange#signal`'s `rescue ::ClosedQueueError` removed — the bare push of the pre-fix tree (review round 2's R2-1) | `cancellation_test.rb` "a token cancel in flight while the exchange finishes never raises out of Source#cancel on the canceller's thread, and the future is cancelled" — `Source#cancel` returns `ClosedQueueError: queue closed` where `true` is pinned: a caller hook registered first parks the canceller between the flag flip and the adapter's hook, a fake client answers a 204 once the flag is up, check-after-resume settles the pivot and closes the queue, and the adapter's hook then pushes onto it | 4.0.6, 3.3.12 | + +Beside the thirty: `async_http_test.rb`'s two source scans (every `Async`, `Protocol`, `OpenSSL` and +`Console` reference under `lib/` `::`-qualified, and the require set exactly `dexpace`, `async/http` +and `openssl`), the ivar pin behind `ASYNC-22`, and `conformance/transport_suite/asynchronous_test.rb` +and `header_drops_test.rb`, which run every portable assertion against `RawWireTransport`'s +correct behaviour and against a named defect for each direction the assertion has. + +## Audit groups run + +The phase-start pair first, at implementation on 2026-09-21: `--origin note --brief` (the notes on +`transport-adapter`, `concurrency-and-async` and the phase-8 facts among them) and +`--section conflicts --brief` with every one of the six harvested conflicts +`[overridden by notes/…]` and none open. The thirteenth audit group's two halves, +`--prefix TRANSPORT --section rules --brief` and `--prefix ASYNC --section rules --brief`, and +`--req` per task against the IDs each task names; the corpus's `ASYNC-21`/`-22` hits are +appendix-B roll-ups, so both were read from chapter 18 itself. + +| Group | Result at implementation | +|---|---| +| Public API surface | One public constant per file; `async_http.rb` carries the module's constants and functions on the entry-file precedent; the eight helpers are `private_constant`s, `Adapter.new` is private behind the two factories, `DropPolicy.new` behind `.build`; the manifest lists `Adapter`, `DropPolicy`, the entry-file constants and nothing private | +| Gem layout, zero-dependency core | `gates:gemspec_audit` (`dexpace-core` + `async-http`), `gates:require_allowlist` (the adapter's `openssl` is on the allowlist; `async/http` is the gem's own declaration) and `gates:clean_bundle` (six gems on 4.0.6, five on 3.2.11) green; nothing in core names the gem beyond the one configuration key | +| RBS / Steep typing | Every new file mirrored; the `:async_http` target relaxed for `UnknownConstant` exactly as `:serde_json` is, with `library "openssl", "uri"`; `_Release` is the one new interface; three typing facts the build met are under Deviations (`Dexpace::Error` is a module, so the classifier's pass-through predicate lives in `own?`; `URI::Generic#request_uri` exists only on `URI::HTTP`, so the mapper asserts the class the screen guarantees; a `Method` does not satisfy a proc type, so the release hook is a lambda over an interface) | +| Minitest conventions | Every suite subclasses `DexpaceTestCase`; nine suites split into nested classes under `Metrics/ClassLength` (8a's shape); no `.stub`; every wait bounded — a queue pop, a `with_timeout`, a `value(deadline:)` — and no `sleep` outside `matrix_facts_test.rb`'s scheduler facts; every thread a test starts is joined before it returns | +| Fiber scheduler, thread safety | The adapter is frozen in effect; the one mutable per-call object is the `Exchange`, whose cross-thread state is one `Thread::Queue`; `Clients`' mutex guards a `Hash` read and insert and nothing else (the client is built outside it, asserted by scan); the pool's own gardener is the one transient task that outlives an exchange, by the library's design | +| Transport and async-runtime adapters | Every `TRANSPORT` rule in the group restates a clause proven above; `transport-adapter/cb7901ef`'s per-protocol grammar entry is the note the design filed and stands; the two new note entries record what 2.46.0 and 0.105.0 measured that the design did not (the cancel-across-threads facts; the reactor-exit drain and the h2 double release) | +| Observability, configuration and redaction | Every emission inside `Instrumentation.contain`; the drop record carries the header name and the reason under `TRANSPORT_HEADER_DROPPED`; `REQUEST_TIMEOUT` through `#duration`, `TRANSPORT_CONNECTION_LIMIT` through `#integer`, both read at construction — and every test that pins a value the chain resolves builds its chain through `AsyncHTTPHermeticConfiguration` (`Sources::NONE` on the environment tier), because `Configuration.build`'s default reads the real process environment and the two default pins and the `ASYNC-22` pool bound moved under an exported `TRANSPORT_CONNECTION_LIMIT` / `REQUEST_TIMEOUT` until review round 0's R0-1 (measured: 3 where 8 was pinned, 5.0 where 60.0, 16 connections where at most 8); a `.build` with no `configuration:` still reads `Dexpace.configuration`, as `dexpace-transport-net_http`'s does, and no test pins a default through it | + +## Deviations from the plan + +Departures from the plan's text, each with its reason. None lowers, disables or narrows a gate. Items +1–13 are where the built tree or the manager's binding decisions overrode the plan's assumptions, +in the order the brief's as-built list gives them; 14–42 are this build's. The ones that touch public +behaviour or a statement the design makes are also the as-built ledger rows `P8-91`–`P8-102`. + +1. **`Dexpace::TransportError`, `Events::TRANSPORT_HEADER_DROPPED` and `Keys::REQUEST_TIMEOUT` were + already on the base** (8a's Task 2), so Task 4 verified and added nothing; the one core widening + is `Keys::TRANSPORT_CONNECTION_LIMIT`, and its two pins were flipped on the code branch. +2. **The client map is keyed by (reactor, origin), not by origin alone** — the manager's decision on + the cross-check's open question 1 (`P8-92`); `MAX_ORIGINS` caps pairs. +3. **The adapter opens no reactor of its own, and the conformance driver opens one per settle on a + foreign thread** (`P8-94`). +4. **8a's `Scripts` gained `Connection: close` on every head** with identical counts, and a `close:` + keyword for the one keep-alive fixture (`P8-96`). +5. **The require set is `dexpace`, `async/http` and `openssl` only**, asserted by scan; the design's + two extra `require` lines are not written. +6. **`TRANSPORT-8` is in this gem's own suite, and the portable groups carry six assertions plus the + `PREAMBLE` sentence**, not the plan's seven (`P8-93`). +7. **The cancellation bridge is queue-marshalled** (`P8-91`), not the design's direct `#cancel`. +8. **`TRANSPORT_CONNECTION_LIMIT` is the one core widening.** +9. **The `:async_http` Steep target relaxes `UnknownConstant` to `:information`** with `library + "openssl", "uri"`, and `rbs_collection.yaml` carries the plan's `- name: async / ignore: true` + row (`P8-99`). The first cut of this record said the row was unnecessary because "the collection + carries none"; review round 0's R0-2 measured otherwise: `ruby/gem_rbs_collection` carries + `gems/async/2.12` — a `Task` with `#stop` and neither `#cancel` nor `.current?` — and `rbs + collection install` installed it the moment the workspace gem's own `ignore: true` was lifted, + turning `steep` red; rbs 4.2.0 cuts its dependency walk at an ignored gem, which is the only + reason the walk never reached `async` on the committed tree. The row keeps the stale signatures + out however the walk is reached (measured on 2026-09-21: with the workspace gem un-ignored the + walk reaches 38 gems and installs no `async`; with it ignored the lock is unchanged and `steep` + green), and nothing else in the closure has a collection entry or ships a `sig/`. +10. **Every top-level test double is prefixed `AsyncHTTP`** (`test:gems` loads six gems' `test/support/` + into one process). +11. **`ResponseMapper` hands the native body, never the response.** +12. **The bare-require child process carries `GEM_PATH`** so the scratch bundle resolves there. +13. **`Adapter.new` is private**, `openssl` 4.0.2 on 3.3 is stated in `docs/first-release.md`, and + 8a's `R3-1` chunked-length case is reported on 8a's row (unreachable here: `protocol-http1` + refuses the head). +14. **The gemspec reads its floor from `VERSIONS`** — `DexpaceVersions.ruby_floor("dexpace-transport-async_http")` + — rather than the literal `">= 3.3"` the plan wrote, so the file has one source of truth (`P8-95`); + `gates:versions` and `gates:gemspec_audit` read the same row. +15. **`P8-37` as built retires every pooled resource before `pool.close`**, because `pool.close` + alone drains and waits on a busy connection (`P8-100`). +16. **The native body's close goes through `Dexpace.close_quietly`**, because closing an unread HTTP/2 + body double-releases the pooled connection inside the library (`P8-101`). +17. **`test/support/async_http_warmup.rb`** spends Ruby 4.0's `IO::Buffer` warning before the fatal + hook can see it (`P8-98`). +18. **No `application/octet-stream` default** (`P8-97`): `async-http` stamps no `Content-Type`, so + 8a's reason does not exist here and inventing a type would be a claim about the bytes. +19. **`Endpoints.screen!` is called first in `RequestMapper.call`**, so an undispatchable scheme is + `InvalidArgumentError` and never a `NoMethodError` wrapped retryable. +20. **`AsyncHTTPServerFixture` needs no readiness probe** (`Task#async` runs the server to its first + suspension, the listener bound), closes through `#cancel`, exposes `closed?` over the task tree, + and wraps the library's server in `QuietServer` so a peer that closes mid-head — an exchange + cancelled during its send — is not a Console line on stderr. +21. **HTTP/2 through the owning adapter is reached over TLS by ALPN** (`build(ssl_context:)` trusting + the fixture's certificate); plaintext prior-knowledge h2 is a borrowed client's. +22. **The parent-cancellation supervisor parks on a queue**, never a sleep, and hands its future + back through a `ready` queue, because `Task#async` returns to the caller before the child's + block has assigned it. +23. **Nine suites are wrapped in a module holding nested classes** under `Metrics/ClassLength` (8a's + shape), `Style/OneClassPerFile` and `Style/Documentation`. +24. **The conformance groups are two files**, `asynchronous.rb` and `header_drops.rb`, under + `Metrics/ModuleLength`; the suite is seven groups, and the counting pins say so. +25. **`Scripts` gained a private `head(*fields)`** so the module stays under its length cap with the + `Connection: close` lines. +26. **`RawWireTransport`'s `header_lines` folds the drop into `copied?(name, folded)`**, its `attempt` + into `subscribe`, its `build_response` into `reason_for`, and the fixture's `certificate_for` + applies its attributes from a hash — all `Metrics` cops, no behaviour. +27. **`Adapter#exchange_for`, `Exchange#net`, `Clients#drain`, `Errors#own?`** are extractions the + `AbcSize` cop asked for; `own?` also keeps `Errors.wrap`'s parameter typed `Exception` for Steep, + because `is_a?(Dexpace::Error)` narrows to a module type. +28. **`RequestMapper` asserts `URI::HTTP`** (`url = request.url #: URI::HTTP`) after the scheme + screen, because rbs declares `#request_uri` on `URI::HTTP` alone. +29. **The body's release hook is a lambda over the `_Release` interface**, never a `Method`, which + Steep does not accept for a proc type. +30. **`Exchange::WATCHER_ANNOTATION`** names the watcher in the task tree, for the suite's release + assertion and for anyone reading a reactor's hierarchy. +31. **`AsyncHTTPReactor`** — `reactor_over(server)` and `assert_exchange_released(task)` — was added + after the first mutation pass, for the reasons the *Guards* section gives; and + **`AsyncHTTPHermeticConfiguration`** after review round 0, because the plan's tests built every + chain through `Dexpace::Configuration.build`'s defaults and the two default pins and the + `ASYNC-22` pool bound read the host's environment (R0-1; the *Audit groups* row has the + measurement). +32. **The three cancellation tests and both timeout pairs assert the exchange and its watcher are + gone** within ten reactor turns (measured: two on a cancellation, one on a timeout). +33. **The driver's foreign-thread settle materialises the body inside its reactor** through + `Response#body_bytes` into a `BufferBody` (`P8-94`); the plan's plain `Sync { value }` hangs. +34. **The driver reports four skips, not three**: the suite carries two assertions under + `TRANSPORT-14`, and a waiver is by id. +35. **`matrix_facts_test.rb`** re-runs eight facts per row on 8a's precedent; the plan ran them once + in a scratch script. +36. **`errors_test.rb`'s `TRANSPORT-3` list includes the SDK's own errors** under a cancelled token, + because the native-only list let mutation 27b survive. +37. **The clients suite's open-response close runs over `reactor_over` and releases the response in + its ensure**, because a close that waited leaves the reactor unable to exit. +38. **`docs/first-release.md` changes in existing entries only**: the conformance-suite blocker and + the `P8-9` documentation blocker each gain a dated status sentence and the latter's box ticks, + the `gates:bounded_map` blocker gains its status, and the supported-Ruby note names 0.105.0. +39. **Two knowledge-note entries were added and none edited**: `transport-adapter.md` (the reactor + exit drain, `P8-37` as built, the h2 double release, the 4.0 warning) and + `concurrency-and-async.md` (the three `Task#cancel`/`Kernel#Async` facts). +40. **`CLAUDE.md`'s built-phases paragraph, gem sentences, floor sentence and counts were re-derived + from the tree**: 220 `lib/dexpace/` files unchanged, nineteen checklists, twelve files in this + gem's `lib/` and twenty-five in `dexpace-conformance`'s. +41. **The `TRANSPORT-8` and `ASYNC-21` rows state their disposition rather than tick**, as the + cross-check's summary directed. +42. **The driver's `around:` runs each assertion as a child task under `finished: false` and bounds + the PARENT's wait** (`AROUND_BOUND`, thirty seconds; review round 1's R1-1). The plan's shape + with a bound added — `with_timeout` around the assertion's body — raises into the assertion's + own fiber, where the adapter's token-first classifier turns it into the `CancelledError` the + cancellation rows expect: measured passing the mid-body row thirty seconds late against the + mutant of guard 37. On expiry the driver cancels the child instead (`Async::Cancel`, which no + classifier converts, and `Response#body_string`'s ensure releases the connection so the reactor + drains) and raises its own `Failure`, a flunk naming the row's ids. +43. **Both cancellation hooks push through `Exchange#signal`, which rescues `ClosedQueueError`** + (review round 2's R2-1). The plan's hooks pushed onto the queue bare; `Cancellation::Source#cancel` + and `Completer#settle` each steal their hook list under their own mutex and run it outside, so + an exchange that finished between the steal and the run — check-after-resume saw the flag the + cancel had already flipped, settled the pivot cancelled and closed the queue — met a push onto + a closed queue, and `Hooks.notify` handed that `ClosedQueueError` back to the caller's own + `Source#cancel` on the cancelling thread while the future read cancelled. Reproduced + deterministically on 4.0.6 and 3.3.12 with an ordinary caller hook registered first; a + `closed?` check first would be the same race one instruction later, so the push is total + instead. The `CancelledError` the watcher wraps into `Task#cancel(cause:)` is kept, and the + record now says what it is for: the task's own `Async::Cancel` names the SDK's reason for + whoever reads the task, and nothing in the adapter reads it back (R2-4, guard 46 equivalent). + +## Findings routed + +Anything execution found, routed to its owner when found; a finding the design already routed is +**verified still owned** and not re-recorded: + +- **Verified still owned, on phase 10's inbound list**: §12's `TRANSPORT-14` scoping and its + `TRANSPORT-8` vacuity claim (the design's two entries); 8a's `R6` no-route-to-a-tracer finding + (`P8-7`); phase 2's `Transport.async_over` return-type check (the plan's README item 5, verified + open and stated in the README rather than fixed here). +- **New, to phase 10's inbound list, by date and content (2026-09-21)**: the design's verified facts 8 + and 10 are stale on `async` 2.46.0 (`cause:` drops a Symbol; `Kernel#Async` inside a task is the + task's child) — the design is frozen to this phase and the correction is a knowledge-note entry, so + the roadmap bullet is the pointer for whoever consolidates the ledger; and `async-http`'s server + letting a mid-head `EOFError` escape to Console, worked around in the test fixture and worth an + upstream report. +- **New, to phase 10's inbound list, by date and content (2026-09-21, review round 1)**: the portable + `TRANSPORT-7` row's path against a streaming adapter is a race — the cancel lands before or after + the adapter has checked its token on the delivered head — so the row proves the in-flight clause + everywhere and the delivered-body clause by chance; the contract's primitives cannot settle it for + an eager and a streaming adapter alike without a bound inside the assertion, the row's own + comment states the race (the code branch), and a deterministic portable form — the consumer + signalling from inside the read, an eager adapter measured vacuous — is conformance-gem work + routed rather than done in a fix round. +- **To `docs/first-release.md`**, in existing entries only: the conformance-suite line's status (the + `async-http` half ran, four skips accounted for), the `P8-9` box ticked with 8c's waivers stated in + `conformance.md` and the `PREAMBLE`, the `gates:bounded_map` line's status. +- **To the knowledge notes**: the two Reference entries named above. +- **Fixed in material this phase may write, not findings**: the reactor-exit hang in the driver, the + h2 double release, the stale keep-alive race in `Scripts`, the surviving and hanging mutants, the + fixture's Console line; and, from review round 1, the body-path test's bound indistinguishable + from the watcher's close and the driver's unbounded `around:` (guard 37, deviation 42); and, from + review round 2, the hook's push racing the exchange's own end (guard 51, deviation 43), the + body-forbidden guard no test reached (guard 47v), the watcher's transience provable only by a + hang (guard 44), and the record calling the `cause:` wrap load-bearing (guard 46). + +## Postponed work + +Task 19 Step 5a's two items, as the design's *Work phase 8c postponed* section directs: + +- **`OBS-19`'s header-drop policy (phase 5b postponed it to phase 8) — landed.** `DropPolicy` (Task 7), + the predicate at dispatch (Task 9), the both-protocols test (Task 15), the antecedent confirmed on + `protocol-http1` 0.41.0; the roadmap's status note says so. +- **The wire-boundary re-validation (phase 1 postponed it to the adapters) — complete in both + adapters.** 8a's Task 16 landed first on this tree, this phase's Task 9 second, and this phase says + so in the roadmap's status note; phase 9's Task 7 adds the portable assertion beyond 8a's two. + +The consolidation of `P8-36`–`P8-40` and `P8-91`–`P8-102` into design §10 is a human's: +`docs/sdk-design-ruby/` is frozen, as it was for every phase before, and `docs/deviations.md` waits +for phase 10 to flip. diff --git a/docs/work/mvp/phase8/phase8c/2026-09-11-phase8c-asynchronous-transport-design.md b/docs/work/mvp/phase8/phase8c/2026-09-11-phase8c-asynchronous-transport-design.md index 9dee25d..857cda7 100644 --- a/docs/work/mvp/phase8/phase8c/2026-09-11-phase8c-asynchronous-transport-design.md +++ b/docs/work/mvp/phase8/phase8c/2026-09-11-phase8c-asynchronous-transport-design.md @@ -2086,6 +2086,56 @@ rather than a decision about a requirement; the unbounded pool default is a knob `P8-38` narrows a MUST, and it narrows it by an amount the requirement's own per-transport scoping anticipates. +### As built, 2026-09-21 + +The five rows above stand as decided; `R13`, `R14`, `R15` and `R16` were built as written, with `R13`'s +orphan close reached through one more channel than this document named and `R14`'s `TRANSPORT-8` +measured live. This sub-phase was cut from `main` at `a7cfeb6`, which holds every phase through 7 and +phase 8a, and executed concurrently with 8b off the same base, so it lands **second** of the two: +`Dexpace::TransportError`, `Events::TRANSPORT_HEADER_DROPPED` and `Keys::REQUEST_TIMEOUT` were 8a's +and were on the base; `Keys::TRANSPORT_CONNECTION_LIMIT` is this phase's one core widening; and the +wire-boundary re-validation is complete in both adapters with this phase's call site. The bundle +resolved `async-http` 0.105.0 over `async` 2.46.0 where this document measured 0.104.0, and two of the +thirteen verified facts above do not hold on that pair (rows P8-91 and P8-102's note). Execution added +the rows below, numbered from **P8-91** as the charter fixes it — 8a's as-built rows are P8-51–P8-65 +and 8b's start at P8-71 — and nothing is renumbered. The checklist's "Deviations from the plan" is the +itemised list against the plan's text; the rows here are the ones that touch public behaviour, the +contract a later phase cites, or a statement this document makes. + +| # | Deviation | Requirement / document | Why | +|---|---|---|---| +| P8-91 | **The cancellation bridge is queue-marshalled, in both directions.** The token's hook settles the pivot cancelled and pushes its reason onto a `Thread::Queue`; `Future#cancel` settles the pivot whose own `on_cancel` hook pushes; a transient watcher task spawned beside the exchange under the caller's task pops the queue and, on the reactor's thread, cancels the exchange task with `cause: Dexpace::CancelledError.new(reason)` while it is in flight or closes the delivered response afterwards. The queue is closed when the exchange ends undelivered or when the delivered body is released, and the watcher exits; nothing in the adapter calls `Task#cancel` from outside the reactor. Both hooks push through `Exchange#signal`, which swallows the `ClosedQueueError` of a push after that close — the source and the completer each steal their hook list under their mutex and run it outside, so a hook can run after the exchange has settled the pivot itself and closed the queue, and the raise would otherwise travel back through `Hooks.notify` into the caller's own `Source#cancel` or `Future#cancel` (review round 2's R2-1, reproduced deterministically on 4.0.6 and 3.3.12) | `ASYNC-6`, `TRANSPORT-7`, `TRANSPORT-9`; the dispatch path's steps 12–13 and verified fact 8, which put `exchange.cancel` in the hook | `Async::Task#cancel` on a task whose fiber is not current is `Fiber.scheduler.raise`, and `Fiber.scheduler` is nil on every OS thread but the reactor's: a hook that reached the task directly raised `NoMethodError: private method 'raise' called for nil` on the canceller's thread and left the exchange running (measured on 3.3.12, 3.4.10 and 4.0.6), and the conformance suite cancels its token from a `Thread.new`. Closing the native body from the canceller's thread instead corrupted the reactor's selector. The `cause:` the watcher passes is the reason wrapped in a `CancelledError` so that the task's own `Async::Cancel` names it for whoever reads the task — a Symbol is replaced by the runtime's own `Cancel::Cause` (fact 8 corrected) — and nothing in the adapter reads it back: the pivot is settled with the reason before the watcher acts (the token's hook settles it before it pushes; `Future#cancel` settled it to run its hook at all), so `#run`'s exit arm has nothing left to settle, and dropping the wrap is an equivalent mutant (review round 2's R2-4, guard 46) | +| P8-92 | **The owning adapter's client map is keyed by (reactor, origin), never by origin alone**, the reactor being `Fiber.scheduler` at the call compared by identity; `MAX_ORIGINS` (32) caps pairs, the drain evicts clients whose reactor has closed before the oldest, and every evicted client is released | `ASYNC-22`, `TRANSPORT-29`, `XCUT-14`; *The object model* → `Clients` | An `Async::HTTP::Client` belongs to the reactor whose tasks drive it: one client shared by two OS threads each in their own `Sync` hung (the cross-check's point 5), so a per-origin map made `ASYNC-22`'s "safe for concurrent calls from multiple threads" false. The manager's decision on the cross-check's open question 1, option (a). Two threads in two reactors through one adapter now each get their own client and mismatch nothing; fibers inside one reactor share one multiplexed client | +| P8-93 | **`TRANSPORT-8` is asserted in this gem's own suite and not in the portable one; the portable groups carry six assertions** — `TRANSPORT-7`, `-9`, `-21`, `-23` in `Asynchronous` and `-12`, `-13` in `HeaderDrops` — and `TransportSuite::PREAMBLE` names `TRANSPORT-8` as the third thing a green run does not prove | `TRANSPORT-8`; the plan's Task 19 Step 1a ("seven portable assertions"); 8a's `P8-9` and `R16` | The row's antecedent is a cancellation the host **runtime** originates while the SDK future is live, which only an adapter's own suite can produce — a portable assertion has no reactor to cancel a parent task in. The manager's decision (the brief's point 26): six portable assertions plus the preamble sentence, which is 8a's own mechanism for what a green run does not prove | +| P8-94 | **A response does not outlive the reactor that produced it, and the conformance driver honours that**: `settle:` on a thread with no reactor opens one, awaits the future, reads the body inside it through `Response#body_bytes` and hands the assertion a `BufferBody` of the same media type; inside `around:`'s reactor the response streams. The as-built page and the README state the rule for a consumer | `TRANSPORT-29`, `TRANSPORT-19`, `SEAM-11`; the plan's Task 19 `settle:` lambda; `R16`'s "the reactor `settle:` needs" | `Sync { }` returns only when the reactor has no non-transient work left, and on its way out it cancels the transient tasks — the pool's gardener among them, whose `ensure` calls `Pool::Controller#close`, which drains, and `drain` waits on every busy connection; the connection behind an unread body is busy. `TRANSPORT-29`'s eight-thread assertion hung under the plan's `Sync { value }` on every row until the body was read inside the reactor | +| P8-95 | **The per-gem Ruby floor is one row in `VERSIONS`** (`ruby floor:dexpace-transport-async_http 3.3`), read by the gemspec (`DexpaceVersions.ruby_floor(gem)`), by `gates:versions` and `gates:gemspec_audit`, and by the `Gemfile`, `test:gems` and `gates:clean_bundle` through `DexpaceVersions.gem_supported?`, which skip a gem on a row below its floor with the count printed | `P8-36`, `NFR-10`, `NFR-14`; the plan's Task 3, which wrote the literal `">= 3.3"` into the gemspec and left the `Gemfile` to a comment | `NFR-14` makes `VERSIONS` the single source of truth for every version-shaped fact, and a floor that lived in a gemspec literal and a gate table alike would drift; the two gate fixtures `per_gem_floor_ahead` prove the gates read it. On 3.2.11 `bundle install` resolves five gems, `test:gems` loads five suites, `gates:clean_bundle` prints "5 gem(s)", and every gate is green | +| P8-96 | **Every head `dexpace-conformance`'s `Scripts` writes carries `Connection: close`**, `Scripts.write_response` takes `close: true`, and a script that deliberately serves two responses on one connection passes `close: false` for every response but its last | `TRANSPORT-5`, `TRANSPORT-29`, `PAGE-36` and every multi-settle assertion; 8a's `Scripts` (8a's Task 6) | `WireServer` closes the socket after one exchange; without the header a client that pools keep-alive connections re-used one the server had already closed and read EOF on its next request — 12 of 20 immediate sequential GETs, measured — which made every multi-settle assertion a coin flip against this adapter. `Net::HTTP` builds a client per call and reads the header as nothing; every 8a count is unchanged | +| P8-97 | **No `Content-Type` is invented**: the caller's explicit header, then the body's own media type, then nothing — a body with neither goes out with no `Content-Type` | `TRANSPORT-10`, `TRANSPORT-26`; 8a's `P8-4` (`application/octet-stream` on every body-permitted method) | `async-http` stamps no `Content-Type` and emits no warning for its absence, so neither of `P8-4`'s reasons — the warnings-fatal build and a form type as a claim about the bytes — exists here, and a default the library did not need would be the adapter's own claim about a payload it has not read. The two adapters differ on this one header, and both as-built pages say so | +| P8-98 | **`test/support/async_http_warmup.rb` allocates and frees one `IO::Buffer` at this gem's test-helper load**, with `Warning[:experimental]` off around it | `NFR-6`; the CI matrix | On 4.0.6 alone the first `IO::Buffer` the scheduler allocates prints Ruby's once-per-process "IO::Buffer is experimental" warning to stderr, which the test base makes fatal in whichever test first drives a socket under the reactor; a test-support arrangement on the precedent of 8a's `net_http_warmup.rb`, not the SDK mutating a host global. The README tells a warnings-fatal host to expect the line once | +| P8-99 | **The `Steepfile`'s `:async_http` target downgrades `Ruby::UnknownConstant` to `:information`** with `library "openssl", "uri"`, and `rbs_collection.yaml` carries the plan's `- name: async / ignore: true` row | `NFR-3`, `NFR-11`; the plan's Task 17 | None of `async`, `async-http`, `protocol-http` or `async-pool` ships a `sig/`, and the one collection entry for the closure — `gems/async/2.12`, whose `Task` declares `#stop` and neither `#cancel` nor `.current?` — is ignored by name, because installing it would type the primitives this gem calls as missing and turn `steep` red (measured by review round 0's R0-2: it installed the moment the workspace gem's own `ignore: true` was lifted; rbs 4.2.0 cuts its dependency walk at an ignored gem, which is the only reason the committed tree never reached it, and the first cut of this row wrongly said the collection carried none). So every `::Async::HTTP::Client.new`, `::Protocol::HTTP::Request.new` and the `< ::Protocol::HTTP::Body::Readable` superclass is an unknown constant; the relaxation is the one 7a's `:serde_json` target established, on this target alone, with every runtime handle typed `untyped` | +| P8-100 | **`P8-37` as built: `Clients.release` retires every pooled resource through `Pool::Controller#retire` and then closes the pool** — never `pool.close` alone, never `Client#close` | `P8-37`, `TRANSPORT-16`, `XCUT-13`; *The object model* → `Clients` ("`client.pool.close`, not `client.close`") | `Pool::Controller#close` is `drain` then clear, and `drain` is "acquire every resource with zero usage, waiting on the condition while any is busy" — so `pool.close` under an open response waits exactly as `Client#close` does, which this document's fact 9 measured for the latter and not the former. `retire` deletes the resource and closes it without waiting; a close under an open response returns in under a millisecond and the open read then fails | +| P8-101 | **`ResponseBody#release` closes the native body through `Dexpace.close_quietly` with the adapter's `logger:`**, and a failure there is one `http.instrumentation.close` WARNING that never reaches `Response#close` | `TRANSPORT-16`, `TRANSPORT-25`, `XCUT-13`; *The object model* → `ResponseBody` | Closing an HTTP/2 body before it was read to the end resets the stream, and `async-http` 0.105.0 writes the `RST_STREAM` frame before it transitions the stream's state, so a peer's `END_STREAM` landing during that write closes the stream twice and releases the pooled connection twice — `RuntimeError: Trying to reuse unacquired resource` out of `Input#close`, five of five through a response obtained in a child task and closed unread by its parent. The connection is retired either way and the client stays sound; the raise is a library artefact a caller can do nothing with | +| P8-102 | **The public surface as built**, against *The object model 8c ships*: `AsyncHTTP.build(timeout:, logger:, drop_policy:, connection_limit:, ssl_context:, configuration:)` and `AsyncHTTP.using(client, logger:, drop_policy:)` — not `Adapter.new(**settings)` and `Adapter.over(client)` — with `AsyncHTTP.default`; `Adapter.owning` / `Adapter.borrowing` behind them and `Adapter.new` private; `Adapter::REACTOR_MESSAGE`; `DropPolicy.build(mode:)` with `MAX_TRACKED_NAMES`, `EVERY`, `ONCE_PER_NAME`, `QUIET`, `MODES`, `#mode` and `#report`; the entry file's `DEFAULT_TIMEOUT_SECONDS`, `DEFAULT_CONNECTION_LIMIT`, `MAX_ORIGINS`, `REGISTRY_KEY`, `FRAMING_HEADERS` and `ALPN_PROTOCOLS`; the private `Exchange::WATCHER_ANNOTATION`; and **no `_Client` interface** — a borrowed client is checked with `respond_to?(:call)`, `respond_to?(:retries)` and `retries.zero?` | *The object model 8c ships*, *The `sig/` shape*, *The interface surface later phases may cite* | 8a fixed the construction vocabulary — `.build` / `.using` over `.owning` / `.borrowing` — and one vocabulary for two adapters is what lets `docs/sdk-documentation/architecture.md` describe them together; `configuration:` is the injection the tests need for the two configuration keys; a `_Client` interface would name a duck no core type can check and `NFR-11` could not carry. The twenty-three rows the `async_http` manifest gained (2 → 25) were read one by one against this list, and `dexpace-conformance`'s manifest gained none — both new groups are private | + +**Two facts this document states that `async` 2.46.0 does not bear out, corrected in +`docs/knowledge/notes/concurrency-and-async.md` and not deviations of this phase's**: verified fact 8's +`cause: :sym` reaching the task's `$!.cause` (it is replaced by `Async::Cancel::Cause`, and the adapter reads no cause back either way; row P8-91), +and verified fact 10's "`Async { }` inside a reactor is not a child of the caller" — `Kernel#Async` +inside a running task delegates to `Task.current.async`, measured `inner.parent.equal?(task)` true, so +the reviewer's mutation 14 is an equivalent mutant and the adapter's `caller_task.async` spelling is +the honest one rather than a distinction the runtime still draws. + +**One property of the portable `TRANSPORT-7` row, found by review round 1 (2026-09-21) and not a +deviation**: against a streaming adapter the row's cancel lands either before the adapter has checked +its token on the delivered head or after the consumer's read has blocked — a race between the +server's signal and the client's head parse — so it proves the in-flight clause on every adapter and +the delivered-body clause by chance: half the runs on 4.0.6 and a third on 3.3.12 against a mutant +with the watcher's delivered-response close deleted, which hung the rest until the driver's `around:` +was bounded from a parent task (a bound raised into the assertion's own fiber meets the token-first classifier +and passes the row late). The body path's deterministic proof is the adapter's own +`cancellation_test.rb`, which refutes its own bound as the wake's cause; the row's comment states the +race, and a deterministic portable form is on phase 10's inbound list. + --- ## Work phase 8c postponed, and who owns what it inherited diff --git a/gems/dexpace-conformance/lib/dexpace/conformance/scripts.rb b/gems/dexpace-conformance/lib/dexpace/conformance/scripts.rb index 1e2224e..691065b 100644 --- a/gems/dexpace-conformance/lib/dexpace/conformance/scripts.rb +++ b/gems/dexpace-conformance/lib/dexpace/conformance/scripts.rb @@ -13,6 +13,13 @@ module Conformance # socket -- `wait_readable` for a delay, `readpartial` for "until the peer goes away" -- and # never on `sleep`, so WireServer#close, which closes every accepted socket, wakes it at once # and no handler thread outlives the test that started it. + # + # Every head a script writes carries `Connection: close` (phase 8c, 2026-09-21): WireServer + # closes the socket after one exchange, so the header only says what the server does anyway + # -- and without it a client that pools keep-alive connections, async-http, re-used a + # connection the server had already closed and read EOF on its next request one time in two + # (measured: 12 of 20 immediate sequential GETs), which made every multi-settle assertion a + # coin flip against that adapter. Net::HTTP builds a client per call and reads it as nothing. module Scripts extend self @@ -66,8 +73,7 @@ def large_body(byte_count) # @return [Proc] the script def dribble(first, second, delay_seconds) lambda do |conn, _head| - conn.write("HTTP/1.1 200 OK\r\nContent-Type: text/plain\r\n" \ - "Transfer-Encoding: chunked\r\n\r\n") + conn.write(head("Content-Type: text/plain", "Transfer-Encoding: chunked")) conn.write(chunk(first)) conn.flush conn.wait_readable(delay_seconds) @@ -98,10 +104,8 @@ def vendor_status(code, body) # @return [Proc] the script def malformed_headers lambda do |conn, _head| - conn.write( - "HTTP/1.1 200 OK\r\nX-Ctl: a\x01b\r\nX-B\xE9d: y\r\nX-Obs: caf\xE9\r\n" \ - "Set-Cookie: a=1\r\nSet-Cookie: b=2\r\nContent-Length: 2\r\n\r\nhi".b, - ) + conn.write(head("X-Ctl: a\x01b", "X-B\xE9d: y", "X-Obs: caf\xE9", "Set-Cookie: a=1", + "Set-Cookie: b=2", "Content-Length: 2",).b, "hi",) end end @@ -111,8 +115,7 @@ def malformed_headers # @return [Proc] the script def malformed_content_length lambda do |conn, _head| - conn.write("HTTP/1.1 200 OK\r\nContent-Type: not a/;;media type\r\n" \ - "Content-Length: abc\r\n\r\nhi") + conn.write(head("Content-Type: not a/;;media type", "Content-Length: abc"), "hi") end end @@ -142,7 +145,7 @@ def hang_before_headers(on_request_read: nil) # @return [Proc] the script def hang_after_headers(on_headers_written: nil) lambda do |conn, _head| - conn.write("HTTP/1.1 200 OK\r\nTransfer-Encoding: chunked\r\n\r\n") + conn.write(head("Transfer-Encoding: chunked")) conn.flush on_headers_written&.call drain_until_closed(conn) @@ -172,7 +175,7 @@ def fail_first_connection_then_succeed(body) # @return [Proc] the script def truncated(declared_length:, actual_body:) lambda do |conn, _head| - conn.write("HTTP/1.1 200 OK\r\nContent-Length: #{declared_length}\r\n\r\n#{actual_body}") + conn.write(head("Content-Length: #{declared_length}"), actual_body) end end @@ -198,19 +201,24 @@ def echo_path ->(conn, head) { write_response(conn, body: head.first.to_s.split[1].to_s) } end - # The one write primitive: a status line, the headers plus a computed Content-Length, and - # the body. It does not close the connection -- WireServer#handle's ensure does, once, for - # every connection whatever its script did. + # The one write primitive: a status line, the headers plus a computed Content-Length and + # `Connection: close`, and the body. It does not close the connection -- WireServer#handle's + # ensure does, once, for every connection whatever its script did -- and the header is what + # HTTP says a server that will do that must send (see the module comment). A script that + # deliberately serves a SECOND response on the same connection -- a keep-alive proof -- + # passes `close: false` for every response but its last. # # @param conn [Object] the accepted socket # @param status [String] the status line's code and reason # @param headers [Hash{String => String}] response headers # @param body [String] the body + # @param close [Boolean] whether to announce that this response ends the connection # @return [nil] def write_response(conn, status: "200 OK", headers: { "Content-Type" => "text/plain" }, - body: "") - lines = headers.merge("Content-Length" => body.bytesize.to_s) - .map { |name, value| "#{name}: #{value}" }.join("\r\n") + body: "", close: true) + framing = { "Content-Length" => body.bytesize.to_s } + framing["Connection"] = "close" if close + lines = headers.merge(framing).map { |name, value| "#{name}: #{value}" }.join("\r\n") conn.write("HTTP/1.1 #{status}\r\n#{lines}\r\n\r\n".b, body.b) conn.flush nil @@ -218,6 +226,10 @@ def write_response(conn, status: "200 OK", headers: { "Content-Type" => "text/pl private + # A raw 200 head for the scripts that write their own framing: the given fields, then the + # `Connection: close` every head here carries (the module comment), then the blank line. + def head(*fields) = "HTTP/1.1 200 OK\r\n#{fields.join("\r\n")}\r\nConnection: close\r\n\r\n" + def chunk(bytes) "#{bytes.bytesize.to_s(16)}\r\n#{bytes}\r\n" end diff --git a/gems/dexpace-conformance/lib/dexpace/conformance/transport_suite.rb b/gems/dexpace-conformance/lib/dexpace/conformance/transport_suite.rb index 7420ca9..ef6a96e 100644 --- a/gems/dexpace-conformance/lib/dexpace/conformance/transport_suite.rb +++ b/gems/dexpace-conformance/lib/dexpace/conformance/transport_suite.rb @@ -13,6 +13,8 @@ require_relative "transport_suite/streaming" require_relative "transport_suite/resilience" require_relative "transport_suite/lifecycle" +require_relative "transport_suite/asynchronous" +require_relative "transport_suite/header_drops" module Dexpace module Conformance @@ -26,14 +28,21 @@ module TransportSuite extend self # P8-9, printed at the head of every report: what a green run of this suite does NOT prove, - # so a third-party adapter author whose adapter passes is not misled about TLS verification - # or connect-timeout classification, which live in the adapter's own suite. + # so a third-party adapter author whose adapter passes is not misled about TLS verification, + # connect-timeout classification, or a runtime-originated cancellation (TRANSPORT-8, whose + # antecedent only an adapter's own suite can name -- phase 8c's decision), which live in the + # adapter's own suite. PREAMBLE = "dexpace-conformance transport suite: the wire fixture speaks plaintext only " \ "and exercises no connect timeout, so TLS verification and TRANSPORT-4's " \ "open-timeout classification are NOT among the things a green run proves " \ - "(P8-9); assert both in the adapter's own suite." + "(P8-9); assert both in the adapter's own suite. Nor is TRANSPORT-8: a " \ + "cancellation the native client originates while the SDK future is live can " \ + "only be raised by naming the adapter's own runtime, so that pair -- terminal " \ + "on the cancellation, retryable on a timeout of the same path -- is the " \ + "adapter's own suite's too." - # The suite's assertions, frozen and ordered: the five groups this phase ships, concatenated. + # The suite's assertions, frozen and ordered: phase 8a's five groups and phase 8c's two, + # concatenated. # # @return [Array] def assertions @@ -93,11 +102,13 @@ def run_one(assertion, build:, borrow:, around:, settle:, wire:) end end - # The five groups, each a private module of its own file (one constant per file, and each + # The seven groups, each a private module of its own file (one constant per file, and each # under RuboCop's module-length cap), concatenated in the order a reader meets the chapter: - # outbound, inbound, streaming, resilience, lifecycle. + # outbound, inbound, streaming, resilience, lifecycle, and phase 8c's two -- the + # cancellation and delivery rows, then the header-drop rows. ASSERTIONS = (Outbound::ASSERTIONS + Inbound::ASSERTIONS + Streaming::ASSERTIONS + - Resilience::ASSERTIONS + Lifecycle::ASSERTIONS).freeze + Resilience::ASSERTIONS + Lifecycle::ASSERTIONS + Asynchronous::ASSERTIONS + + HeaderDrops::ASSERTIONS).freeze private_constant :ASSERTIONS end end diff --git a/gems/dexpace-conformance/lib/dexpace/conformance/transport_suite/asynchronous.rb b/gems/dexpace-conformance/lib/dexpace/conformance/transport_suite/asynchronous.rb new file mode 100644 index 0000000..f5d4b57 --- /dev/null +++ b/gems/dexpace-conformance/lib/dexpace/conformance/transport_suite/asynchronous.rb @@ -0,0 +1,192 @@ +# frozen_string_literal: true +# SPDX-License-Identifier: MIT + +require "dexpace" +require_relative "../scripts" +require_relative "checks" + +module Dexpace + module Conformance + module TransportSuite + # Group 6, phase 8c's cancellation and delivery rows driven through the contract's + # primitives: TRANSPORT-7, TRANSPORT-9, TRANSPORT-21 and TRANSPORT-23 (its two header-drop + # rows, TRANSPORT-12 and TRANSPORT-13, are group 7, HeaderDrops). Every one is written + # against `kase.settle`, `kase.wire` and Dexpace::Cancellation -- and nothing else, so no + # reactor, task or native class is named here and the gem stays at dexpace-core alone. A + # cancellation is fired through the token, because the contract exposes no future; on an + # async driver `settle:` awaits the future the token settles, on a sync one the token is + # what the blocked send observes, and the requirement's observable -- the native exchange + # released, the cancellation terminal -- is the same on both. TRANSPORT-8 is NOT here: its + # antecedent is a cancellation the host runtime originates, which only an adapter's own + # suite can name (PREAMBLE says so). A private_constant of TransportSuite. + module Asynchronous + extend self + + # TRANSPORT-7's clause, "cancel an in-flight future and assert the native call is + # cancelled", through the token: the head has arrived and the body is what blocks, the + # token is cancelled on a second thread once the server has written the head, and the + # cancellation surfaces terminal -- from the send when the adapter reads eagerly, from the + # body read when it streams -- with the server observing the connection released. + # + # Which of the two a STREAMING adapter takes is a race the contract cannot settle: the + # server's signal fires as the head leaves its socket, and the cancel lands either before + # the adapter has checked its token on the delivered head (the send surfaces it) or after + # the consumer's body read has blocked (the read does). Measured one in two against the + # async-http adapter with its delivered-response close deleted (2026-09-21): half the runs + # passed through the send path and half hung in the read. So this row proves the + # in-flight clause on every adapter and the delivered-body clause only when the race + # falls that way; an adapter's own suite pins the body path deterministically, with the + # consumer signalling from inside the read, and its driver bounds `around:`. + # + # @param kase [TransportCase] + # @return [nil] + def cancelling_the_token_mid_body_releases_the_exchange(kase) + source = Dexpace::Cancellation.source + written = ::Thread::Queue.new + on_headers_written = -> { written.push(true) } + kase.wire(script: Scripts.hang_after_headers(on_headers_written: on_headers_written)) + canceller = cancel_when(written, source, :conformance_mid_body) + error = Checks.error_from do + kase.settle(kase.transport, kase.request, Dexpace::RequestOptions::EMPTY, source.token) + .body_string + end + canceller.join + + expect_terminal_cancellation(kase, error, "TRANSPORT-7", + misclassified: "a mid-body cancellation did not surface " \ + "as the terminal interrupt", + unreleased: "the cancelled exchange did not release its " \ + "connection",) + end + + # TRANSPORT-9: the token is cancelled while the send is blocked on the head, and THEN the + # server answers in full -- a native response delivered after the SDK side has already + # cancelled. It must not be delivered, and the connection it arrived on must be released. + # + # @param kase [TransportCase] + # @return [nil] + def a_response_arriving_after_cancellation_is_closed_not_delivered(kase) + source = Dexpace::Cancellation.source + blocked = ::Thread::Queue.new + answer = ::Thread::Queue.new + kase.wire(script: gated_answer(blocked, answer)) + canceller = cancel_when(blocked, source, :conformance_late_answer) { answer.push(true) } + error = Checks.error_from do + kase.settle(kase.transport, kase.request, Dexpace::RequestOptions::EMPTY, source.token) + end + canceller.join + + expect_terminal_cancellation(kase, error, "TRANSPORT-9", + misclassified: "a response arriving after the " \ + "cancellation was delivered or misclassified", + unreleased: "the late response's connection was not " \ + "released",) + end + + # TRANSPORT-21: a failure raised while the request is adapted -- the body's own + # `#content_length` raising, which no adapter can know in advance -- comes back through the + # send primitive's failure channel classified as one of the SDK's own errors, never as the + # raw exception thrown past it. On an async driver the primitive awaits the future, so a + # classified failure here is one the future carried. + # + # @param kase [TransportCase] + # @return [nil] + def an_adaptation_failure_is_delivered_classified(kase) + kase.wire(script: Scripts.fixed("ok")) + error = Checks.error_from do + kase.settle(kase.transport, kase.request(method: "POST", body: RaisingBody.new)).close + end + + Checks.check(error.is_a?(Dexpace::Error), + "an adaptation failure escaped the send contract unclassified", + ids: ["TRANSPORT-21"], expected: "a Dexpace::Error", actual: error&.class,) + end + + # TRANSPORT-23: a success is always a Dexpace::Response, for a 204 with no body as for a + # 200 with one; a transport with no response completes exceptionally instead. + # + # @param kase [TransportCase] + # @return [nil] + def a_success_always_carries_a_response(kase) + kase.wire(script: Scripts.sequenced("ok", "")) + transport = kase.transport + settled = Array.new(2) { kase.settle(transport, kase.request) } + settled.each { |response| Dexpace.close_quietly(response) } + + Checks.expect(settled.map(&:class), [Dexpace::Response, Dexpace::Response], + "a success settled with something other than a Dexpace::Response", + ids: ["TRANSPORT-23"],) + end + + # The registry's rows, `[ids, name, function]`, in the group's order. + ROWS = [ + ["TRANSPORT-7", "cancelling the token mid-body releases the native exchange as the " \ + "terminal interrupt", + :cancelling_the_token_mid_body_releases_the_exchange,], + ["TRANSPORT-9", "a response arriving after the cancellation is closed, never delivered", + :a_response_arriving_after_cancellation_is_closed_not_delivered,], + ["TRANSPORT-21", "an adaptation failure is delivered classified through the send " \ + "primitive's failure channel", + :an_adaptation_failure_is_delivered_classified,], + ["TRANSPORT-23", "a success always carries a Dexpace::Response, even with no body", + :a_success_always_carries_a_response,], + ].freeze + private_constant :ROWS + + # The registry, built from ROWS. + ASSERTIONS = Checks.registry(self, ROWS) + + # A body whose adaptation fails: every adapter asks a body's length or pulls it before or + # while dispatching, and this one raises the moment either happens. + class RaisingBody + include Dexpace::Body + + # Raises: the adaptation's first question about the body. + # + # @raise [RuntimeError] always + def content_length + raise "the body's own content_length failed during adaptation" + end + + # Raises: the adaptation's other route into the body. + # + # @raise [RuntimeError] always + def write_to(_sink) + raise "the body's own write failed during adaptation" + end + end + private_constant :RaisingBody + + private + + # The second thread both cancellation rows need: waits for the fixture's signal, cancels + # the token with the given reason, then runs the block -- how TRANSPORT-9 lets the server + # answer only after the cancel has landed. + def cancel_when(signal, source, reason) + ::Thread.new do + signal.pop + source.cancel(reason) + yield if block_given? + end + end + + # Holds the connection after reading the request until `answer` says so, then answers in + # full: the shape TRANSPORT-9 needs and no named script has. + def gated_answer(blocked, answer) + lambda do |conn, _head| + blocked.push(true) + answer.pop + Scripts.write_response(conn, body: "late") + end + end + + # Both cancellation rows' outcome: the terminal interrupt, and the connection released. + def expect_terminal_cancellation(kase, error, id, misclassified:, unreleased:) + Checks.expect(error.class, Dexpace::CancelledError, misclassified, ids: [id]) + Checks.check_connection_released(kase, ids: [id], message: unreleased) + end + end + private_constant :Asynchronous + end + end +end diff --git a/gems/dexpace-conformance/lib/dexpace/conformance/transport_suite/header_drops.rb b/gems/dexpace-conformance/lib/dexpace/conformance/transport_suite/header_drops.rb new file mode 100644 index 0000000..c90d654 --- /dev/null +++ b/gems/dexpace-conformance/lib/dexpace/conformance/transport_suite/header_drops.rb @@ -0,0 +1,167 @@ +# frozen_string_literal: true +# SPDX-License-Identifier: MIT + +require "dexpace" +require_relative "../vacuous" +require_relative "../failure" +require_relative "../scripts" +require_relative "checks" + +module Dexpace + module Conformance + module TransportSuite + # Group 7, phase 8c's header-drop rows: TRANSPORT-12 and TRANSPORT-13. Written against + # `kase.settle`, `kase.wire` and the `build:` factory's `logger:` setting -- the two + # settings the suite contract's clause 4 allows -- and nothing else. Both resolve vacuous BY + # MEASUREMENT on an adapter whose native client accepts every model-valid name: the bad name + # is sent and the wire is read, which is what the first-party sync adapter's driver reports, + # and why neither row could be a shared assertion declaring itself vacuous (8a's R16). A + # private_constant of TransportSuite. + module HeaderDrops + extend self + + # A model-valid header name the RFC 7230 token grammar refuses: HTTP-17 admits `:`. + NON_TOKEN = "X-Bad:Name" + + # How many distinct non-token names TRANSPORT-13's bound is probed with; a policy that + # warns for every one of them is unbounded. + DISTINCT_NAMES = 200 + + # TRANSPORT-12's clause: "send a model-valid non-token header name plus a normal header; + # assert send does not throw, the bad header is absent, the normal header present". The + # antecedent is measured: an adapter whose native client accepted the name -- it reached + # the wire beside the normal one and the send succeeded -- has no drop to make. + # + # @param kase [TransportCase] + # @return [nil] + # @raise [Vacuous] when the native client accepted the model-valid name + def non_token_name_is_dropped_and_the_rest_dispatched(kase) + kase.wire(script: Scripts.fixed("ok")) + request = kase.request(headers: Checks.headers(NON_TOKEN => "v", "X-Normal" => "n")) + error = Checks.error_from { kase.settle(kase.transport, request).close } + + Checks.check(error.nil?, "the native exception escaped the send contract", + ids: ["TRANSPORT-12"], expected: "a normal completion", + actual: error&.class,) + sent = Checks.last_request(kase, ids: ["TRANSPORT-12"]) + Checks.expect(sent.header("x-normal"), "n", "the normal header did not dispatch", + ids: ["TRANSPORT-12"],) + vacuous_unless_absent!(sent) + end + + # TRANSPORT-13's clause: "under once-per-header assert the same name warns once then goes + # quiet, a different name warns once" -- read off the transport's own logger, which the + # `build:` factory takes -- and the bound: two hundred distinct names in one request warn + # fewer than two hundred times. + # + # @param kase [TransportCase] + # @return [nil] + # @raise [Vacuous] when the native client accepted the model-valid name + def header_drops_are_logged_once_per_name_and_bounded(kase) + kase.wire(script: Scripts.fixed("ok")) + sink = RecordingSink.new + transport = kase.transport(logger: Dexpace::Instrumentation::Logger.build(sink: sink)) + first = Checks.headers(NON_TOKEN => "v") + second = Checks.headers("Y-Bad:Name" => "v") + [first, first, second].each do |headers| + kase.settle(transport, kase.request(headers: headers)).close + end + vacuous_unless_dropped!(kase, sink) + + Checks.expect(sink.severities, %i[warn debug warn], + "the once-per-name policy did not warn once then go quiet", + ids: ["TRANSPORT-13"],) + check_bounded(kase, transport, sink) + end + + # The registry's rows, `[ids, name, function]`, in the group's order. + ROWS = [ + ["TRANSPORT-12", "a model-valid non-token header name is dropped and the rest still " \ + "dispatches", :non_token_name_is_dropped_and_the_rest_dispatched,], + ["TRANSPORT-13", "header drops are logged once per name, then quietly, and the latch " \ + "is bounded", :header_drops_are_logged_once_per_name_and_bounded,], + ].freeze + private_constant :ROWS + + # The registry, built from ROWS. + ASSERTIONS = Checks.registry(self, ROWS) + + # Dexpace::Instrumentation's duck-typed sink, recording the severity and payload of every + # drop record so an assertion can read them back; every other record is ignored. + class RecordingSink + def initialize + @records = [] #: Array[[Symbol, untyped]] + @mutex = ::Thread::Mutex.new + end + + # The sink's four writers, each recording under its own severity. + def debug(message = nil, &) = record(:debug, message, &) + # (see #debug) + def info(message = nil, &) = record(:info, message, &) + # (see #debug) + def warn(message = nil, &) = record(:warn, message, &) + # (see #debug) + def error(message = nil, &) = record(:error, message, &) + # The four predicates: every severity is enabled, so the policy's choice is what shows. + def debug? = true + def info? = true + def warn? = true + def error? = true + + # The severities of every drop record, in order. + def severities + @mutex.synchronize { @records.map(&:first) } + end + + private + + def record(severity, message) + payload = block_given? ? yield : message + event = payload.is_a?(::Hash) ? payload["event"] : nil + return nil unless event == Dexpace::Instrumentation::Events::TRANSPORT_HEADER_DROPPED + + @mutex.synchronize { @records << [severity, payload] } + nil + end + end + private_constant :RecordingSink + + private + + def vacuous_unless_absent!(sent) + return if sent.header(NON_TOKEN.downcase).nil? + + raise Vacuous, "the native client accepted the model-valid name #{NON_TOKEN} and sent " \ + "it, so it rejects no header the SDK model admits (TRANSPORT-12's " \ + "antecedent is absent)" + end + + def vacuous_unless_dropped!(kase, sink) + return unless sink.severities.empty? + + sent = Checks.last_request(kase, ids: ["TRANSPORT-13"]) + if sent.header("y-bad:name").nil? + raise Failure.new("a non-token name was dropped without a drop record", + expected: "a TRANSPORT_HEADER_DROPPED record per drop", + actual: "none", requirement_ids: ["TRANSPORT-13"],) + end + + raise Vacuous, "the native client accepted the model-valid names and sent them, so " \ + "there is no drop to log (TRANSPORT-13's antecedent is absent)" + end + + def check_bounded(kase, transport, sink) + many = Checks.headers(Array.new(DISTINCT_NAMES) { |i| ["Z-Bad:#{i}", "v"] }.to_h) + before = sink.severities.size + kase.settle(transport, kase.request(headers: many)).close + warned = sink.severities.drop(before).count(:warn) + + Checks.check(warned < DISTINCT_NAMES, "the once-per-name latch grew without bound", + ids: ["TRANSPORT-13"], expected: "fewer than #{DISTINCT_NAMES} warnings", + actual: warned,) + end + end + private_constant :HeaderDrops + end + end +end diff --git a/gems/dexpace-conformance/sig/dexpace/conformance/scripts.rbs b/gems/dexpace-conformance/sig/dexpace/conformance/scripts.rbs index 3712a5d..b7c6641 100644 --- a/gems/dexpace-conformance/sig/dexpace/conformance/scripts.rbs +++ b/gems/dexpace-conformance/sig/dexpace/conformance/scripts.rbs @@ -23,10 +23,11 @@ module Dexpace def self?.sequenced: (*String bodies) -> ^(untyped, Array[String]) -> void def self?.echo_path: () -> ^(untyped, Array[String]) -> void def self?.write_response: (untyped conn, ?status: String, ?headers: Hash[String, String], - ?body: String) -> nil + ?body: String, ?close: bool) -> nil private + def head: (*String fields) -> String def chunk: (String bytes) -> String def drain_until_closed: (untyped conn) -> nil end diff --git a/gems/dexpace-conformance/sig/dexpace/conformance/transport_suite.rbs b/gems/dexpace-conformance/sig/dexpace/conformance/transport_suite.rbs index 5742da9..6cf9181 100644 --- a/gems/dexpace-conformance/sig/dexpace/conformance/transport_suite.rbs +++ b/gems/dexpace-conformance/sig/dexpace/conformance/transport_suite.rbs @@ -5,7 +5,7 @@ module Dexpace # runtime check does not have. module TransportSuite PREAMBLE: String - # A private_constant with no visibility in RBS; the five groups, concatenated. + # A private_constant with no visibility in RBS; the seven groups, concatenated. ASSERTIONS: Array[Assertion] def self?.assertions: () -> Array[Assertion] diff --git a/gems/dexpace-conformance/sig/dexpace/conformance/transport_suite/asynchronous.rbs b/gems/dexpace-conformance/sig/dexpace/conformance/transport_suite/asynchronous.rbs new file mode 100644 index 0000000..6248e17 --- /dev/null +++ b/gems/dexpace-conformance/sig/dexpace/conformance/transport_suite/asynchronous.rbs @@ -0,0 +1,33 @@ +# Dexpace::Conformance::TransportSuite::Asynchronous is a private_constant and not public API: +# this declaration exists because the conformance Steep target checks every file under lib/. RBS +# has no visibility for a constant, so the privacy lives in asynchronous.rb alone. +module Dexpace + module Conformance + module TransportSuite + module Asynchronous + ROWS: Array[[untyped, String, Symbol]] + ASSERTIONS: Array[Assertion] + + class RaisingBody + include Dexpace::Body + + def content_length: () -> Integer + def write_to: (untyped sink) -> Integer + end + + def self?.cancelling_the_token_mid_body_releases_the_exchange: (TransportCase kase) -> nil + def self?.a_response_arriving_after_cancellation_is_closed_not_delivered: (TransportCase kase) -> nil + def self?.an_adaptation_failure_is_delivered_classified: (TransportCase kase) -> nil + def self?.a_success_always_carries_a_response: (TransportCase kase) -> nil + + private + + def cancel_when: (Thread::Queue signal, Dexpace::Cancellation::Source source, Symbol reason) + ?{ () -> untyped } -> Thread + def gated_answer: (untyped blocked, untyped answer) -> ^(untyped, Array[String]) -> void + def expect_terminal_cancellation: (TransportCase kase, Exception? error, String id, + misclassified: String, unreleased: String) -> nil + end + end + end +end diff --git a/gems/dexpace-conformance/sig/dexpace/conformance/transport_suite/header_drops.rbs b/gems/dexpace-conformance/sig/dexpace/conformance/transport_suite/header_drops.rbs new file mode 100644 index 0000000..6208712 --- /dev/null +++ b/gems/dexpace-conformance/sig/dexpace/conformance/transport_suite/header_drops.rbs @@ -0,0 +1,44 @@ +# Dexpace::Conformance::TransportSuite::HeaderDrops is a private_constant and not public API: +# this declaration exists because the conformance Steep target checks every file under lib/. RBS +# has no visibility for a constant, so the privacy lives in header_drops.rb alone. +module Dexpace + module Conformance + module TransportSuite + module HeaderDrops + NON_TOKEN: String + DISTINCT_NAMES: Integer + ROWS: Array[[untyped, String, Symbol]] + ASSERTIONS: Array[Assertion] + + class RecordingSink + @records: Array[[Symbol, untyped]] + @mutex: Thread::Mutex + + def initialize: () -> void + def debug: (?untyped message) ?{ () -> untyped } -> nil + def info: (?untyped message) ?{ () -> untyped } -> nil + def warn: (?untyped message) ?{ () -> untyped } -> nil + def error: (?untyped message) ?{ () -> untyped } -> nil + def debug?: () -> bool + def info?: () -> bool + def warn?: () -> bool + def error?: () -> bool + def severities: () -> Array[Symbol] + + private + + def record: (Symbol severity, untyped message) ?{ () -> untyped } -> nil + end + + def self?.non_token_name_is_dropped_and_the_rest_dispatched: (TransportCase kase) -> nil + def self?.header_drops_are_logged_once_per_name_and_bounded: (TransportCase kase) -> nil + + private + + def vacuous_unless_absent!: (untyped sent) -> nil + def vacuous_unless_dropped!: (TransportCase kase, RecordingSink sink) -> nil + def check_bounded: (TransportCase kase, untyped transport, RecordingSink sink) -> nil + end + end + end +end diff --git a/gems/dexpace-conformance/test/dexpace/conformance/transport_suite/asynchronous_test.rb b/gems/dexpace-conformance/test/dexpace/conformance/transport_suite/asynchronous_test.rb new file mode 100644 index 0000000..ea43ff2 --- /dev/null +++ b/gems/dexpace-conformance/test/dexpace/conformance/transport_suite/asynchronous_test.rb @@ -0,0 +1,53 @@ +# frozen_string_literal: true +# SPDX-License-Identifier: MIT + +require_relative "../../../test_helper" +require_relative "../../../support/assertion_probe" +require_relative "../../../support/raw_wire_transport" +require "dexpace/conformance" + +# Group 6 (phase 8c): TRANSPORT-7, 9, 21 and 23, each proven in both directions against +# RawWireTransport -- which reads bodies eagerly, so the mid-body cancellation surfaces from the +# send rather than from a later read, which is the shape the assertion admits. Phase 8c's other +# two rows, TRANSPORT-12 and 13, are group 7 (header_drops_test.rb). +class DexpaceConformanceAsynchronousAssertionsTest < DexpaceTestCase + include AssertionProbe + + test "the group registers four assertions after the first twenty-eight, and no TRANSPORT-8" do + assert_equal([%w[TRANSPORT-7], %w[TRANSPORT-9], %w[TRANSPORT-21], %w[TRANSPORT-23]], + Suite.assertions[28, 4].map(&:ids),) + assert_equal(34, Suite.assertions.size) + refute_includes(Suite.assertions.flat_map(&:ids), "TRANSPORT-8") + assert_match(/TRANSPORT-8/, Suite::PREAMBLE) + end + + test "TRANSPORT-7: a mid-body cancel as CancelledError with the connection released passes; " \ + "the same as a retryable failure fails" do + assertion = find("TRANSPORT-7") + + assert_passes(assertion, build: raw) + assert_fails(assertion, build: raw(:misclassify_cancel), matching: /terminal interrupt/) + end + + test "TRANSPORT-9: a response arriving after the cancel is not delivered passes; a transport " \ + "that ignores the token delivers it and fails" do + assertion = find("TRANSPORT-9") + + assert_passes(assertion, build: raw) + assert_fails(assertion, build: raw(:ignore_cancel), matching: /delivered or misclassified/) + end + + test "TRANSPORT-21: a classified adaptation failure passes; the raw exception escaping fails" do + assertion = find("TRANSPORT-21") + + assert_passes(assertion, build: raw) + assert_fails(assertion, build: raw(:bare_adaptation), matching: /unclassified/) + end + + test "TRANSPORT-23: a Dexpace::Response for a body and for none passes; a nil success fails" do + assertion = find("TRANSPORT-23") + + assert_passes(assertion, build: raw) + assert_fails(assertion, build: raw(:null_success), matching: /other than a Dexpace::Response/) + end +end diff --git a/gems/dexpace-conformance/test/dexpace/conformance/transport_suite/header_drops_test.rb b/gems/dexpace-conformance/test/dexpace/conformance/transport_suite/header_drops_test.rb new file mode 100644 index 0000000..fa99c6e --- /dev/null +++ b/gems/dexpace-conformance/test/dexpace/conformance/transport_suite/header_drops_test.rb @@ -0,0 +1,48 @@ +# frozen_string_literal: true +# SPDX-License-Identifier: MIT + +require_relative "../../../test_helper" +require_relative "../../../support/assertion_probe" +require_relative "../../../support/raw_wire_transport" +require "dexpace/conformance" + +# Group 7 (phase 8c): TRANSPORT-12 and 13, each proven in both directions against +# RawWireTransport's drop mode, and in the third direction the two share: VACUOUS by measurement +# against the plain double, which copies a model-valid non-token name to the wire exactly as +# Net::HTTP does. +class DexpaceConformanceHeaderDropsAssertionsTest < DexpaceTestCase + include AssertionProbe + + # The build factory that honours the `logger:` setting TRANSPORT-13 passes. + def raw_logged(*defects, drop_non_token: false) + lambda do |logger: Dexpace::Instrumentation::Logger::NULL, **_settings| + RawWireTransport.new(*defects, drop_non_token: drop_non_token, logger: logger) + end + end + + test "the group registers the last two assertions, after group six's four" do + assert_equal([%w[TRANSPORT-12], %w[TRANSPORT-13]], Suite.assertions[32, 2].map(&:ids)) + assert_equal(34, Suite.assertions.size) + end + + test "TRANSPORT-12: a dropped non-token name passes; a native refusal escaping fails; a native " \ + "client that sends it is vacuous by measurement" do + assertion = find("TRANSPORT-12") + + assert_passes(assertion, build: raw_logged(drop_non_token: true)) + assert_fails(assertion, build: raw(:refuse_non_token), matching: /native exception escaped/) + assert_vacuous(assertion, build: raw, matching: /antecedent is absent/) + end + + test "TRANSPORT-13: once-per-name, bounded drops pass; every-drop-warns fails; a silent drop " \ + "fails; a native client that sends the name is vacuous by measurement" do + assertion = find("TRANSPORT-13") + + assert_passes(assertion, build: raw_logged(drop_non_token: true)) + assert_fails(assertion, build: raw_logged(:drop_non_token_loudly), + matching: /warn once then go quiet/,) + assert_fails(assertion, build: raw_logged(:drop_non_token_silently), + matching: /without a drop record/,) + assert_vacuous(assertion, build: raw, matching: /antecedent is absent/) + end +end diff --git a/gems/dexpace-conformance/test/dexpace/conformance/transport_suite/lifecycle_test.rb b/gems/dexpace-conformance/test/dexpace/conformance/transport_suite/lifecycle_test.rb index 21aa0b3..3c3516f 100644 --- a/gems/dexpace-conformance/test/dexpace/conformance/transport_suite/lifecycle_test.rb +++ b/gems/dexpace-conformance/test/dexpace/conformance/transport_suite/lifecycle_test.rb @@ -9,9 +9,10 @@ require "dexpace/conformance" # Group 5 (8a plan Task 13): TRANSPORT-5, 6, 15, 16, 29 and phase 7c's PAGE-36, each proven in -# both directions. This is also the file that closes the registry: the whole suite is 28 -# assertions, one of them vacuous by measurement, none of them TRANSPORT-12's or TRANSPORT-13's -# (phase 8c's rows, asserted by 8c's driver). +# both directions. This is also the file that pins the registry's size: the whole suite was 28 +# assertions in five groups, one of them vacuous by measurement, until phase 8c appended its two +# (asynchronous_test.rb, header_drops_test.rb) -- 34 in seven, TRANSPORT-8 still absent by +# decision. class DexpaceConformanceLifecycleAssertionsTest < DexpaceTestCase include AssertionProbe @@ -22,15 +23,19 @@ def borrow_with(probe_answer) end end - test "the suite is exactly 28 assertions in five groups, with no TRANSPORT-12 or TRANSPORT-13" do - assert_equal(28, Suite.assertions.size) + # Twenty-eight in five groups until phase 8c added its six assertions in two groups after this + # one (TRANSPORT-7, 9, 21, 23, then 12 and 13 -- and not TRANSPORT-8, which PREAMBLE names as + # the third thing a green run does not prove): 34 in seven. + test "the suite is exactly 34 assertions in seven groups, this one fifth" do + assert_equal(34, Suite.assertions.size) assert_equal([%w[TRANSPORT-5], %w[TRANSPORT-6], %w[TRANSPORT-15], %w[TRANSPORT-15 TRANSPORT-16], %w[TRANSPORT-29], %w[PAGE-36],], Suite.assertions[22, 6].map(&:ids),) ids = Suite.assertions.flat_map(&:ids).uniq - assert_empty(ids & %w[TRANSPORT-12 TRANSPORT-13]) - assert_equal(22, ids.grep(/\ATRANSPORT-/).size, - "the 23 own IDs minus TRANSPORT-30, whose proxy assertions are the adapter's own",) + assert_equal(28, ids.grep(/\ATRANSPORT-/).size, + "8a's 23 own IDs minus TRANSPORT-30, whose proxy assertions are the adapter's " \ + "own, plus 8c's six portable ones",) + refute_includes(ids, "TRANSPORT-8") end test "TRANSPORT-5: two concurrent calls each bounded by its own timeout pass; sticky fails" do diff --git a/gems/dexpace-conformance/test/support/raw_wire_transport.rb b/gems/dexpace-conformance/test/support/raw_wire_transport.rb index ea1ab44..e867fef 100644 --- a/gems/dexpace-conformance/test/support/raw_wire_transport.rb +++ b/gems/dexpace-conformance/test/support/raw_wire_transport.rb @@ -11,14 +11,19 @@ # item 6: "a deliberately non-conforming fake transport ... proving the suite detects rather than # merely runs"). The gem's own double, never a lift of dexpace-core's (design, "Work phase 8a # postponed"). It is not an adapter: no pump, no proxy, no TLS, bodies read eagerly. -class RawWireTransport # rubocop:disable Metrics/ClassLength -- one client with twenty switchable defects; splitting it would hide which defect lives where +class RawWireTransport # rubocop:disable Metrics/ClassLength -- one client with twenty-six switchable defects; splitting it would hide which defect lives where DEFECTS = %i[ form_type override_type ignore_body_type no_zero_length forward_host drop_pass_through skip_validation retry_once send_twice misclassify_cancel cancel_on_timeout non_retryable_timeout sticky_timeout ignore_window pretend_followed send_after_close cached_response leave_open - short_read bare_errno + short_read bare_errno ignore_cancel refuse_non_token drop_non_token_silently + drop_non_token_loudly null_success bare_adaptation ].freeze + # TRANSPORT-13's bound, as the drop mode below applies it: the first drop per folded name warns, + # the rest are verbose, and after this many distinct names every drop is verbose. + DROP_TRACKED_NAMES = 64 + # The framing headers this client always recomputes and never copies (TRANSPORT-11). FRAMING = %w[content-length transfer-encoding].freeze @@ -30,13 +35,21 @@ class Bare < StandardError; end attr_reader :calls - def initialize(*defects, default_timeout: 1.5, owned: true) + # `drop_non_token:` switches on the CORRECT TRANSPORT-12/13 behaviour -- a non-token name is + # dropped before the wire and logged through `logger:` once per folded name, then quietly, over a + # bounded latch -- which the plain double does not have: it copies every model-valid name, as + # Net::HTTP does, and phase 8c's two assertions must read that as vacuous by measurement. + def initialize(*defects, default_timeout: 1.5, owned: true, drop_non_token: false, + logger: Dexpace::Instrumentation::Logger::NULL) unknown = defects - DEFECTS raise ArgumentError, "unknown defects: #{unknown.inspect}" unless unknown.empty? @defects = defects @default_timeout = default_timeout @owned = owned + @drop_non_token = drop_non_token + @logger = logger + @warned = {} @timeout_for = nil @cached = nil @closed = false @@ -70,7 +83,7 @@ def call(request, options, cancellation) return build_response(request, @cached) if defect?(:cached_response) && @cached validate!(request) unless defect?(:skip_validation) - bytes = wire_bytes(request) + bytes = adapt(request) response = exchange(request, bytes, timeout_for(options), cancellation) response = exchange(request, bytes, timeout_for(options), cancellation) if defect?(:send_twice) response @@ -92,6 +105,17 @@ def validate!(request) end end + # TRANSPORT-21: a failure raised while the request is adapted -- a body that raises -- is + # classified as the retryable transport failure (P6-4's "wrap, and default to retryable"), + # unless the defect lets the raw exception escape. + def adapt(request) + wire_bytes(request) + rescue StandardError => error + raise error if defect?(:bare_adaptation) + + raise Dexpace::TransportError.new("adaptation failed: #{error.message}", phase: :connect) + end + # The request head and body as bytes, under this double's header policy. def wire_bytes(request) body = body_bytes(request) @@ -113,7 +137,7 @@ def header_lines(request, body) explicit_type = nil request.headers.each_entry do |name, value| folded = name.downcase - next unless copied?(folded) + next unless copied?(name, folded) explicit_type = value if folded == "content-type" lines << "#{name}: #{value}" unless folded == "content-type" && defect?(:override_type) @@ -124,15 +148,49 @@ def header_lines(request, body) end # Whether a caller's header is copied onto the wire: framing is recomputed, Host is this - # client's own unless the defect forwards it, and everything else is pass-through unless the - # defect drops it. - def copied?(folded) + # client's own unless the defect forwards it, a non-token name is the drop mode's business, and + # everything else is pass-through unless the defect drops it. + def copied?(name, folded) return false if FRAMING.include?(folded) return defect?(:forward_host) if folded == "host" + return false if non_token_dropped?(name) folded == "content-type" || !defect?(:drop_pass_through) end + # TRANSPORT-12/13's three shapes: the plain double copies a non-token name like any other (so + # the assertions measure the antecedent absent); the `refuse_non_token` defect lets a native + # refusal escape; `drop_non_token:` drops it and logs through the policy; the two loud/silent + # defects drop it and log wrongly or not at all. + def non_token_dropped?(name) + return false if Dexpace::HeaderSyntax.token?(name) + raise "native client refused the header name #{name}" if defect?(:refuse_non_token) + return false unless @drop_non_token || defect?(:drop_non_token_silently) || + defect?(:drop_non_token_loudly) + + log_drop(name) unless defect?(:drop_non_token_silently) + true + end + + def log_drop(name) + severity = drop_severity(name.downcase) + event = Dexpace::Instrumentation::Events::TRANSPORT_HEADER_DROPPED + Dexpace::Instrumentation.contain(@logger, event: event) do + @logger.event(severity).event(event).field("header", name).field("reason", "not a token") + .emit + end + end + + def drop_severity(folded) + return Dexpace::Instrumentation::Severity::WARNING if defect?(:drop_non_token_loudly) + if @warned.key?(folded) || @warned.size >= DROP_TRACKED_NAMES + return Dexpace::Instrumentation::Severity::VERBOSE + end + + @warned[folded] = true + Dexpace::Instrumentation::Severity::WARNING + end + def content_type(request, body, explicit_type) return nil if body.nil? || (explicit_type && !defect?(:override_type)) @@ -172,7 +230,7 @@ def exchange(request, bytes, timeout, cancellation) def attempt(request, bytes, timeout, cancellation) socket = connect(request) - subscription = cancellation.on_cancel { socket.close } + subscription = subscribe(cancellation, socket) begin socket.write(bytes) read_response(request, socket, timeout, cancellation) @@ -193,6 +251,14 @@ def attempt(request, bytes, timeout, cancellation) end end + # The ignore_cancel defect subscribes nothing and classifies nothing by the token: a cancelled + # send blocks until the server answers, and the answer is DELIVERED (TRANSPORT-9's failure). + def subscribe(cancellation, socket) + return Dexpace::Cancellation.none.on_cancel { nil } if defect?(:ignore_cancel) + + cancellation.on_cancel { socket.close } + end + def connect(request) TCPSocket.new(request.url.hostname, request.url.port) rescue SystemCallError => error @@ -202,7 +268,7 @@ def connect(request) end def classify(error, cancellation) - if cancellation.cancelled? + if cancellation.cancelled? && !defect?(:ignore_cancel) return Dexpace::TransportError.new("cancelled", phase: :read) if defect?(:misclassify_cancel) return Dexpace::CancelledError.new(cancellation.reason) @@ -260,12 +326,17 @@ def read_body(socket, headers) end def build_response(request, head) + return nil if defect?(:null_success) && head.body.empty? + body = Dexpace::ResponseBody.new(source: Dexpace::IO::BufferedSource.of_bytes(head.body), media_type: media_type(head.headers), content_length: head.body.bytesize,) Dexpace::Response.build(request: request, protocol: "HTTP/1.1", status: head.code, - reason: head.status_line.split(" ", 3)[2]&.strip, - headers: head.headers, body: body,) + reason: reason_for(head), headers: head.headers, body: body,) + end + + def reason_for(head) + head.status_line.split(" ", 3)[2]&.strip end def media_type(headers) diff --git a/gems/dexpace-core/lib/dexpace/configuration/keys.rb b/gems/dexpace-core/lib/dexpace/configuration/keys.rb index 063a3c8..c88d9d1 100644 --- a/gems/dexpace-core/lib/dexpace/configuration/keys.rb +++ b/gems/dexpace-core/lib/dexpace/configuration/keys.rb @@ -52,6 +52,14 @@ module Keys # caller who means thirty seconds writes `30s` or `PT30S`. The precedent for a # transport-facing key nothing in core reads is HTTP_PROXY/HTTPS_PROXY/NO_PROXY above. REQUEST_TIMEOUT = "REQUEST_TIMEOUT" + + # The per-origin connection-pool bound the asynchronous transport reads at construction, + # added by phase 8c in the change that reads it: async-http's own pool is unbounded by + # default, and an SDK that hands a caller an unbounded file-descriptor budget has decided on + # the caller's behalf. Read with `#integer`; genuinely that adapter's alone, because + # dexpace-transport-net_http builds a client per call and has no pool to bound. Nothing in + # core reads it. + TRANSPORT_CONNECTION_LIMIT = "TRANSPORT_CONNECTION_LIMIT" end end end diff --git a/gems/dexpace-core/sig/dexpace/configuration/keys.rbs b/gems/dexpace-core/sig/dexpace/configuration/keys.rbs index 996efca..4f7c7e3 100644 --- a/gems/dexpace-core/sig/dexpace/configuration/keys.rbs +++ b/gems/dexpace-core/sig/dexpace/configuration/keys.rbs @@ -10,6 +10,7 @@ module Dexpace MAX_TRACKED_CONTEXTS: String LOG_PREVIEW_BYTES: String REQUEST_TIMEOUT: String + TRANSPORT_CONNECTION_LIMIT: String end end end diff --git a/gems/dexpace-core/test/dexpace/async_transport_bare_require_test.rb b/gems/dexpace-core/test/dexpace/async_transport_bare_require_test.rb new file mode 100644 index 0000000..352174c --- /dev/null +++ b/gems/dexpace-core/test/dexpace/async_transport_bare_require_test.rb @@ -0,0 +1,83 @@ +# frozen_string_literal: true +# SPDX-License-Identifier: MIT + +require_relative "../test_helper" +require_relative "../support/bare_require" +require "dexpace" + +# SEAM-1, SEAM-2, SEAM-6 on the AsyncTransport seam: the properties that hold on a BARE +# `require "dexpace"` and on nothing else -- an empty registry that is not the sync seam's, the +# zero-candidate SeamError that follows from one, and a swap or an install that leaves nothing +# resolved behind it. Each runs in a fresh process, for the reason transport_bare_require_test.rb +# gives about the sync seam since phase 8a: since phase 8c `dexpace-transport-async_http`'s entry +# file registers its factory under :async_http the moment it is required (design §3.6), so in +# the one `rake test:gems` process the registry is NOT empty and #resolve builds an adapter +# instead of raising. The in-process form was the pin phase 8c's registration invalidated; +# async_transport_test.rb keeps the in-process halves. No lib/ mirror: it asserts the process, +# not a file. +class DexpaceAsyncTransportBareRequireTest < DexpaceTestCase + include BareRequire + + test "the registry starts empty and is not the sync seam's (SEAM-1, P2-1)" do + out = bare_require(<<~RUBY) + print Dexpace::AsyncTransport.registered_keys.inspect + Dexpace::Transport.swap(->(_r, _o, _c) { :sync }) do + begin + Dexpace::AsyncTransport.resolve + print " resolved through the sync seam" + rescue Dexpace::SeamError + print " SeamError" + end + end + RUBY + + assert_equal("[] SeamError", out) + end + + test "the zero-candidate error names this seam and no gem (SEAM-2)" do + message = bare_require(<<~RUBY) + begin + Dexpace::AsyncTransport.resolve + print "resolved" + rescue Dexpace::SeamError => error + print error.message + end + RUBY + + assert_match(/no async transport provider is registered/, message) + assert_match(/Dexpace::AsyncTransport\.install/, message) + refute_match(%r{async_http|net_http|async/http}i, message, "SEAM-2") + end + + test "after a swap block nothing is resolved again (SEAM-6)" do + out = bare_require(<<~RUBY) + t = ->(_request, _options, _cancellation) { :future } + Dexpace::AsyncTransport.swap(t) do + abort("the override was not resolved") unless Dexpace::AsyncTransport.resolve.equal?(t) + end + begin + Dexpace::AsyncTransport.resolve + print "resolved after the block" + rescue Dexpace::SeamError + print "SeamError" + end + RUBY + + assert_equal("SeamError", out) + end + + test "an install inside a swap block is restored with it, leaving nothing resolved" do + out = bare_require(<<~RUBY) + i = ->(_request, _options, _cancellation) { :installed } + Dexpace::AsyncTransport.swap(:override) { Dexpace::AsyncTransport.install(i) } + begin + Dexpace::AsyncTransport.resolve + print "resolved after the block" + rescue Dexpace::SeamError + print "SeamError" + end + RUBY + + assert_equal("SeamError", out) + end +end diff --git a/gems/dexpace-core/test/dexpace/async_transport_test.rb b/gems/dexpace-core/test/dexpace/async_transport_test.rb index cd5230d..d6b76ec 100644 --- a/gems/dexpace-core/test/dexpace/async_transport_test.rb +++ b/gems/dexpace-core/test/dexpace/async_transport_test.rb @@ -11,6 +11,12 @@ # would merge two concerns the requirement separates. The name is not Dexpace::Transport::Async, # because that constant would sit beside the adapter namespaces Dexpace::Transport::NetHTTP and # ::AsyncHTTP -- a seam beside its own implementations. +# +# The properties of a BARE `require "dexpace"` -- an empty registry that is not the sync seam's, +# the zero-candidate SeamError and nothing resolved after a swap -- live in +# async_transport_bare_require_test.rb since phase 8c, whose adapter registers a factory the +# moment it is required: in one `rake test:gems` process the registry is not empty, and the +# in-process assertions were the pin that registration invalidated (8a's shape, applied here). class DexpaceAsyncTransportTest < DexpaceTestCase CORE = "~> 0.0" @@ -24,32 +30,37 @@ class DexpaceAsyncTransportTest < DexpaceTestCase assert(Dexpace::Transport.conforms?(FakeAsyncTransport.new)) end - test "the registry starts empty and is not the sync seam's" do - assert_empty(Dexpace::AsyncTransport.registered_keys) + # The two seams are two registries: a sync override resolves nothing here. Whether this one + # starts empty is a property of the bare require (see the class comment). + test "the registry is not the sync seam's" do + keys_before = Dexpace::AsyncTransport.registered_keys Dexpace::Transport.swap(->(_r, _o, _c) { :sync }) do - assert_raises(Dexpace::SeamError) { Dexpace::AsyncTransport.resolve } + assert_equal(keys_before, Dexpace::AsyncTransport.registered_keys) + resolved = begin + Dexpace::AsyncTransport.resolve + rescue Dexpace::SeamError + :none + end + + refute_equal(:sync, resolved, "the sync override reached the async seam") end end - test "the zero-candidate error names this seam and no gem" do - error = assert_raises(Dexpace::SeamError) { Dexpace::AsyncTransport.resolve } - - assert_match(/no async transport provider is registered/, error.message) - assert_match(/Dexpace::AsyncTransport\.install/, error.message) - refute_match(/async_http|net_http/i, error.message, "SEAM-2") - end - + # The override itself; "and afterwards nothing is resolved" is a property of the bare require + # and lives in async_transport_bare_require_test.rb. test "install refuses a non-conforming object and swap scopes an override" do assert_raises(Dexpace::InvalidArgumentError) { Dexpace::AsyncTransport.install(Object.new) } transport = FakeAsyncTransport.new + keys_before = Dexpace::AsyncTransport.registered_keys Dexpace::AsyncTransport.swap(transport) do assert_same(transport, Dexpace::AsyncTransport.resolve) end - assert_raises(Dexpace::SeamError) { Dexpace::AsyncTransport.resolve } + assert_equal(keys_before, Dexpace::AsyncTransport.registered_keys, + "the swap registered nothing",) end test "install goes through the module, returns it, and is scoped by an enclosing swap" do @@ -59,8 +70,6 @@ class DexpaceAsyncTransportTest < DexpaceTestCase assert_same(Dexpace::AsyncTransport, Dexpace::AsyncTransport.install(transport)) assert_same(transport, Dexpace::AsyncTransport.resolve) end - - assert_raises(Dexpace::SeamError) { Dexpace::AsyncTransport.resolve } end # The registration is scoped inside a swap and cleaned out of the private registry afterwards, diff --git a/gems/dexpace-core/test/dexpace/configuration/keys_test.rb b/gems/dexpace-core/test/dexpace/configuration/keys_test.rb index 09af81d..e15583b 100644 --- a/gems/dexpace-core/test/dexpace/configuration/keys_test.rb +++ b/gems/dexpace-core/test/dexpace/configuration/keys_test.rb @@ -8,7 +8,8 @@ # name 5b's body-logging wiring reads (LOG_PREVIEW_BYTES, added by phase 5b in the change that # reads it -- the eighth constant, and this suite's count grew with it) and the one name phase # 8's transport adapters read (REQUEST_TIMEOUT, added by phase 8a in the change that reads it, -# the ninth; the same rule, the same growth). +# the ninth; the same rule, the same growth), and the one name phase 8c's asynchronous transport +# alone reads (TRANSPORT_CONNECTION_LIMIT, the tenth, added in the change that reads it). class DexpaceConfigurationKeysTest < DexpaceTestCase test "CFG-14: the five well-known keys and the four wiring keys are frozen, non-empty Strings" do expected = { @@ -21,6 +22,7 @@ class DexpaceConfigurationKeysTest < DexpaceTestCase MAX_TRACKED_CONTEXTS: "MAX_TRACKED_CONTEXTS", LOG_PREVIEW_BYTES: "LOG_PREVIEW_BYTES", REQUEST_TIMEOUT: "REQUEST_TIMEOUT", + TRANSPORT_CONNECTION_LIMIT: "TRANSPORT_CONNECTION_LIMIT", } assert_equal(expected.keys.sort, Dexpace::Configuration::Keys.constants.sort) diff --git a/gems/dexpace-core/test/dexpace/instrumentation/downstream_wirings_test.rb b/gems/dexpace-core/test/dexpace/instrumentation/downstream_wirings_test.rb index f995b8a..89d13a6 100644 --- a/gems/dexpace-core/test/dexpace/instrumentation/downstream_wirings_test.rb +++ b/gems/dexpace-core/test/dexpace/instrumentation/downstream_wirings_test.rb @@ -221,12 +221,13 @@ def configuration(environment: {}, properties: {}) assert_empty(sink.entries) end - # Nine keys since phase 8a added REQUEST_TIMEOUT (its R3); this pin was eight until then. + # Nine keys since phase 8a added REQUEST_TIMEOUT (its R3), ten since phase 8c added + # TRANSPORT_CONNECTION_LIMIT; this pin was eight before either. test "the body-logging caps' key: LOG_PREVIEW_BYTES is the eighth key, LOG_LEVEL is 5a's" do assert_equal("LOG_PREVIEW_BYTES", ConfigKeys::LOG_PREVIEW_BYTES) assert_equal("LOG_LEVEL", ConfigKeys::LOG_LEVEL) assert_predicate(ConfigKeys::LOG_PREVIEW_BYTES, :frozen?) - assert_equal(9, ConfigKeys.constants.size) + assert_equal(10, ConfigKeys.constants.size) # The reference default is the CALLER's, resolved through 5a's typed accessor and never # baked into a 5b signature (the plan's open question 5). assert_equal(8192, configuration.integer(ConfigKeys::LOG_PREVIEW_BYTES, default: 8 * 1024)) diff --git a/gems/dexpace-transport-async_http/README.md b/gems/dexpace-transport-async_http/README.md index 135989c..f37abcc 100644 --- a/gems/dexpace-transport-async_http/README.md +++ b/gems/dexpace-transport-async_http/README.md @@ -3,9 +3,21 @@ Part of the [dexpace Ruby SDK](../../README.md): an HTTP-client toolkit, not an HTTP client. This gem is the asynchronous transport seam's reference adapter, over `async-http`. -**Status: skeleton at `0.0.0`.** Nothing is published yet, and `lib/` holds the namespace and a -`VERSION` constant and nothing else. The phase that fills it is named in the YARD block of -`lib/dexpace/transport/async_http.rb`. +**Status: `0.0.0`, unpublished; the adapter is built.** `lib/` holds phase 8c's +`Dexpace::Transport::AsyncHTTP`: the two constructions +`.build(timeout:, logger:, drop_policy:, connection_limit:, ssl_context:, configuration:)` over a +client map the adapter owns -- one `Async::HTTP::Client` per (reactor, origin), bounded at +`MAX_ORIGINS` -- and `.using(client, logger:, drop_policy:)` over a caller's own client, the +`Adapter` behind both with its owning-or-borrowing lifecycle, the header policy the wire carries +(`FRAMING_HEADERS`, the RFC 7230 token predicate applied on both protocols, and `DropPolicy`'s +once-per-name reporting), a per-call exchange task under the caller's own reactor task with one +total per-call budget, the queue-marshalled cancellation bridge that lets a token cancelled from any +thread reach the exchange, the lazy pull-shaped response body, the classifier that asks the +cancellation token first and wraps every other failure as a retryable `Dexpace::TransportError`, +the lenient inbound mapping, the default TLS context offering HTTP/2 by ALPN (`ALPN_PROTOCOLS`), and +the registration under `REGISTRY_KEY` (`:async_http`) that requiring the gem performs. Proven over +HTTP/1.1, plaintext prior-knowledge HTTP/2 and TLS HTTP/2 on Ruby 3.3, 3.4 and 4.0, and against the +shared conformance suite in `dexpace-conformance` as its second driver. ## Install @@ -14,21 +26,119 @@ This gem is the asynchronous transport seam's reference adapter, over `async-htt gem "dexpace-transport-async_http" ``` +This gem requires **Ruby >= 3.3** -- narrower than the SDK's 3.2 floor, because `async-http` +and `async` require it (the SDK's `P8-36`). A consumer on 3.2 composes `dexpace-core` with +`dexpace-transport-net_http` and loses only the reactor transport. + ## The smallest thing that works today ```ruby require "dexpace/transport/async_http" -puts Dexpace::Transport::AsyncHTTP::VERSION # => "0.0.0" +adapter = Dexpace::Transport::AsyncHTTP.build +request = Dexpace::Request.build(method: "GET", url: "http://127.0.0.1:8080/pets?limit=2", + headers: Dexpace::Headers::EMPTY) +Sync do + future = adapter.call(request, Dexpace::RequestOptions::EMPTY, nil) # against a server on 8080 + response = future.value + response.status.code # => 200 + response.body_string # => "[]" -- decoded through the one decode boundary, then closed +end +adapter.close # => nil, reactor-free ``` +The response body streams from the connection until it is drained or closed, on the fiber that +reads it; `Response#body_string` does both. `AsyncPipeline.standard(adapter, redirect: :unsupported)` +puts retry and authentication around it. + +## What a reactor means here + +**Every call needs a running reactor on the calling thread.** `Adapter#call` outside `Sync { }` or +`Async { }` does not raise: it returns an **already-failed** future carrying `Dexpace::SeamError` +whose message names the fix (`TRANSPORT-21`, `P8-39`). This gem creates no reactor of its own -- a +`Sync` inside the adapter would block the caller's thread and defeat the seam, and a reactor +thread the adapter owned would be a thread pool by another name, carrying the constructor's +diagnostic context rather than the caller's. + +**A response does not outlive the reactor that produced it.** `Sync { }` returns only when the +reactor has no work left, and on its way out it drains the connection pool, which waits for every +busy connection -- and the connection behind an unread body is busy until that body is read or +closed. So read or close the response **inside** the block that made the call; a `Sync` that hands +a streaming response out to code after it never returns. `Response#body_string`, `#body_bytes` and +`#close` all release the connection. + +**`Dexpace::AsyncTransport.sync_over(adapter)` still requires a reactor.** The bridge awaits the +future this adapter returns, and the exchange behind that future cannot run without a reactor, so +a caller with no reactor gets the same `SeamError` through the bridge as through `#call`. + +**`Dexpace::Transport.async_over` accepts this adapter silently.** That bridge takes a synchronous +transport, and handing it an object whose `#call` already returns a future yields a future of a +future; the return-type check that would refuse it is phase 2's open finding in `dexpace-core`, +not this gem's to fix. Use the adapter as the `Dexpace::AsyncTransport` it is. + +## ASYNC-7: what a cancellation interrupts + +The SDK's design (§3.3) fixes the contrast between its two async adapters in one sentence: **"the +thread adapter lets an in-flight blocking read finish, the reactor-backed ones abort at the next +scheduler checkpoint."** This gem is the reactor-backed one. A cancellation -- the token's, the +future's, or one the runtime originates by cancelling a parent task -- reaches the exchange task as +`Async::Cancel` at its next scheduler checkpoint, which is where a blocked socket read is suspended, +and `ensure` blocks run there. That is the property `Thread#raise` lacks and the reason it is banned +in this SDK: the interrupt lands only where the task can be interrupted, never inside a connection's +release. The adapter's own suite measures it (a blocked read is abandoned well under the time the +response would have taken), so this sentence and the assertion cannot drift apart. + +A cancellation from another OS thread -- `Cancellation::Source#cancel` on a thread that is not the +reactor's -- is marshalled through a queue to a watcher task inside the reactor, because +`Async::Task#cancel` from a foreign thread is not supported by the runtime; the effect is the same +and it is prompt. + +## Timeouts, headers and what the wire carries + +- **The per-call budget** is `RequestOptions#timeout`, then `.build(timeout:)`, then the + configuration key `REQUEST_TIMEOUT`, then `DEFAULT_TIMEOUT_SECONDS` (60). A bare number in + `REQUEST_TIMEOUT` is **milliseconds** (`CFG-7`): thirty seconds is `30s` or `PT30S`. The budget + is one `Async::Task#with_timeout` over the whole exchange, so it bounds the head; a body read + after delivery is bounded by the caller's own reactor discipline. +- **Framing headers are never copied** from a request: the ten folded names in `FRAMING_HEADERS` + (`host`, `content-length`, `transfer-encoding`, `connection`, `keep-alive`, `proxy-connection`, + `te`, `trailer`, `upgrade`, `expect`) are dropped and logged at verbose, because `async-http` + would append a caller's `Host` beside its own rather than replace it (`TRANSPORT-11`). +- **A header name the RFC 7230 token grammar refuses** (`HTTP-17` admits seventeen bytes the + grammar does not, `:` among them) is dropped **on both protocols** before dispatch + (`TRANSPORT-12`, `P8-40`) and reported through the adapter's `logger:` once per name, then + quietly, over a bounded latch (`TRANSPORT-13`, `DropPolicy`). On HTTP/1.1 `protocol-http1` would + refuse the whole request after the request line is on the wire; on HTTP/2 `protocol-http2` would + transmit it unvalidated. +- **No `Content-Type` is invented.** The caller's explicit header wins, then the body's own media + type; a body with neither goes out with no `Content-Type` -- `async-http` stamps none, unlike + `Net::HTTP`, whose form-urlencoded default is what `dexpace-transport-net_http` pre-empts with + `application/octet-stream`. +- **A body-less `GET` carries `content-length: 0`.** `async-http` writes it, and suppressing it + would mean reaching under the body layer; `TRANSPORT-26` does not forbid it and `HTTP-7` is about + the model, not the wire. Recorded rather than fixed. +- **HTTP/2** is negotiated by ALPN over TLS through the default context (`VERIFY_PEER`, + `ALPN_PROTOCOLS`); a caller's `ssl_context:` is used verbatim, so a context without + `alpn_protocols` negotiates HTTP/1.1. Plaintext prior-knowledge HTTP/2 is a property of a + caller's own client, reachable through `.using`. +- **On Ruby 4.0**, the first use of `IO::Buffer` under a scheduler prints Ruby's once-per-process + "IO::Buffer is experimental" warning to stderr. The adapter cannot suppress it for a host; a + host that runs with warnings fatal should expect it once, as this gem's own suite does. + ## Depends on -`dexpace-core`, and later `async-http` -- the one third-party gem `NFR-2` budgets for this adapter, -declared by phase 8 with the code that needs it. +`dexpace-core`, and `async-http ~> 0.104` -- the one gem `NFR-2` budgets for this adapter, whose +closure brings `async`, `async-pool`, `protocol-http`, `protocol-http1`, `protocol-http2`, +`io-event` and `openssl`; on Ruby 3.3 that resolution compiles the `openssl` gem, because the +interpreter's own is older than `io-stream` requires. ## Where to read next +- `docs/sdk-documentation/transport-async_http.md` -- the as-built page: the reactor discipline, + what the wire carries, what a cancellation interrupts, what a failure means, and what is + deliberately not here. +- `docs/sdk-documentation/conformance.md` -- the suite this adapter is proven against, and the + three things a green run does not prove. - `docs/sdk-documentation/architecture.md` -- how the gems compose and which one to install. - `docs/sdk-design-ruby/02-gem-and-workspace-layout.md` -- the gem layout and the zero-dependency invariant every gem here is built under. diff --git a/gems/dexpace-transport-async_http/dexpace-transport-async_http.gemspec b/gems/dexpace-transport-async_http/dexpace-transport-async_http.gemspec index 41dd355..19a724a 100644 --- a/gems/dexpace-transport-async_http/dexpace-transport-async_http.gemspec +++ b/gems/dexpace-transport-async_http/dexpace-transport-async_http.gemspec @@ -12,12 +12,17 @@ Gem::Specification.new do |spec| spec.summary = "The asynchronous transport adapter for dexpace, over async-http." spec.description = <<~TEXT The reference asynchronous transport for the dexpace HTTP-client toolkit, implemented over - the async-http gem. It depends on dexpace-core and, once the adapter lands, on async-http - and nothing else. + the async-http gem. It depends on dexpace-core and on async-http and nothing else. TEXT spec.homepage = "https://github.com/dexpace/ruby-sdk" spec.license = "MIT" - spec.required_ruby_version = ">= #{DexpaceVersions.ruby_floor}" + # P8-36: narrower than the repository's 3.2 floor, and the one gemspec that reads its OWN + # `floor:` row of VERSIONS rather than the global one. async-http 0.95.0 and async 2.38.0 + # both raised required_ruby_version to >= 3.3, and the highest release that still admits 3.2 is + # eleven minor versions behind the one every fact in this gem's design was verified against. A + # floor a gem declares must be a floor it is tested on (8c's R15); the gates read the per-gem + # row, and the 3.2 CI row leaves this gem out of the bundle, test:gems and gates:clean_bundle. + spec.required_ruby_version = ">= #{DexpaceVersions.ruby_floor("dexpace-transport-async_http")}" spec.metadata = { "homepage_uri" => spec.homepage, @@ -32,7 +37,10 @@ Gem::Specification.new do |spec| spec.add_dependency "dexpace-core", DexpaceVersions.core_constraint - # NFR-2: dexpace-core plus at most one third-party gem. The third-party half of this - # adapter's budget is declared by the phase that writes the code needing it (design P0-9); - # `rake gates:gemspec_audit` enforces the whole budget either way. + # NFR-2: dexpace-core plus at most one third-party gem, and this is the one (phase 8c). The + # `~>` on a two-segment version admits every 0.x release from 0.104 on (`>= 0.104, < 1`), so + # 0.105.0 and a future 0.200.0 alike; the adapter is proven on 0.105.0, whose closure is fifteen + # further gems including io-event's C extension and, below Ruby 3.4's default openssl 3.3, an + # installed openssl gem -- both stated in docs/first-release.md. + spec.add_dependency "async-http", "~> 0.104" end diff --git a/gems/dexpace-transport-async_http/lib/dexpace/transport/async_http.rb b/gems/dexpace-transport-async_http/lib/dexpace/transport/async_http.rb index 390186c..27eaea6 100644 --- a/gems/dexpace-transport-async_http/lib/dexpace/transport/async_http.rb +++ b/gems/dexpace-transport-async_http/lib/dexpace/transport/async_http.rb @@ -1,20 +1,153 @@ # frozen_string_literal: true # SPDX-License-Identifier: MIT +require "dexpace" +require "async/http" require_relative "async_http/version" +require_relative "async_http/errors" +require_relative "async_http/drop_policy" +require_relative "async_http/endpoints" +require_relative "async_http/clients" +require_relative "async_http/request_body" +require_relative "async_http/request_mapper" +require_relative "async_http/response_body" +require_relative "async_http/response_mapper" +require_relative "async_http/exchange" +require_relative "async_http/adapter" module Dexpace # Transport adapters: the synchronous and asynchronous transport seams' shipped # implementations. The seam contracts themselves live in dexpace-core. module Transport - # The reference asynchronous transport, over the `async-http` gem. Phase 0 ships the - # namespace and VERSION only; the adapter itself lands in phase 8. + # The reference asynchronous transport, over the `async-http` gem (phase 8c): the one MVP gem + # whose reason for existing is to prove the properties a thread pool cannot -- HTTP/2 + # multiplexing, a structured cancellation tree, and suspension at a scheduler checkpoint. One + # `Async::HTTP::Client` per (reactor, origin) pair, each built with `retries: 0` and a bounded + # pool; every call is a child task of the caller's own `Async::Task` under its own + # `with_timeout`, and the returned `Dexpace::Async::Future` is settled from inside it. The + # adapter creates no reactor: a call from a thread with no `Fiber.scheduler` comes back as an + # already-failed future carrying Dexpace::SeamError (P8-39, TRANSPORT-21). # # CAUTION: once dexpace-async-thread is loaded in the same process, `Dexpace::Async` exists, # and an unqualified `Async::HTTP` written anywhere under this module resolves through the - # lexical scope to `Dexpace::Async` before it ever reaches the socketry gem. Every reference - # to that gem from inside here is written `::Async::HTTP`. + # lexical scope to `Dexpace::Async` before it ever reaches the socketry gem -- and `Dexpace::Async` + # exists already, in core, so the shadowing is not hypothetical. Every reference to that gem, + # to `Protocol` and to `OpenSSL` from inside here is written `::`-qualified, and the gem's + # smoke suite scans lib/ for an unqualified one. module AsyncHTTP + extend self + + # The configured tier's fallback for RequestOptions#timeout: the per-call budget when + # neither the call nor the transport nor Configuration::Keys::REQUEST_TIMEOUT says + # otherwise. The same key and the same default as dexpace-transport-net_http, so one caller + # setting governs both transports (the charter's shared transport contracts, item 4). + DEFAULT_TIMEOUT_SECONDS = 60.0 + + # The per-origin connection limit when Configuration::Keys::TRANSPORT_CONNECTION_LIMIT is + # unset: async-http's own default is an UNBOUNDED pool (eight concurrent requests opened + # seven connections in the design's measurement), and an SDK that hands a caller an + # unbounded file-descriptor budget has decided on the caller's behalf. Chosen, not derived. + DEFAULT_CONNECTION_LIMIT = 8 + + # XCUT-14's hard cap on the client map: (reactor, origin) pairs, drained back under it in a + # loop after every insert, each evicted client's pool retired. A memory backstop and never + # the primary cleanup -- `#close` is. + MAX_ORIGINS = 32 + + # The key the require-time registration below uses, and the key a caller passes to + # Dexpace::AsyncTransport when they want this adapter by name. + REGISTRY_KEY = :async_http + + # TRANSPORT-11's drop set: the same ten folded names dexpace-transport-net_http drops under + # its MANAGED_HEADERS (the charter's shared transport contracts, item 1; the membership is + # shared, the constant is per gem because NFR-2 forbids either gem depending on the other). + # More load-bearing here than there: async-http APPENDS a caller-set `host`, `content-length` + # or `transfer-encoding` beside the one it writes itself, which is the canonical + # Host-duplication and CL.CL / TE.CL request-smuggling shape (the design's verified fact 5). + # A drop from this set is always logged at VERBOSE and never goes through DropPolicy. + FRAMING_HEADERS = %w[ + host content-length transfer-encoding connection keep-alive proxy-connection te trailer + upgrade expect + ].freeze + + # What the adapter's own TLS context offers by ALPN: HTTP/2 preferred, HTTP/1.1 admitted. + # A caller-supplied `ssl_context:` is used verbatim and never has this set on it. + ALPN_PROTOCOLS = %w[h2 http/1.1].freeze + + # The SDK-managed construction: builds and owns its clients, one per (reactor, origin), and + # closes them all on `#close` by retiring every pooled connection without waiting (P8-37). + # + # The budget's three tiers, highest first: `RequestOptions#timeout` on the call, `timeout:` + # here, then `Configuration::Keys::REQUEST_TIMEOUT` -- read through `Configuration#duration`, + # whose grammar treats a BARE number as milliseconds (CFG-7): `REQUEST_TIMEOUT=30` is thirty + # milliseconds, and thirty seconds is `30s` or `PT30S` -- and finally + # DEFAULT_TIMEOUT_SECONDS. The budget bounds the exchange up to the response head: the + # body streams under no deadline (IO-40), and the deadline interrupts only at a scheduler + # checkpoint (design §8.3). + # + # `ssl_context:` replaces the adapter's own TLS context for every https origin, whole rather + # than per knob, so the SDK never becomes a partial re-export of OpenSSL's surface; the + # caller's context is used verbatim, ALPN included -- a context with no `alpn_protocols` + # negotiates HTTP/1.1. With none, the adapter's own context verifies the peer against the + # default certificate store for EVERY host, including `localhost`, which async-http's own + # default would have silently left unverified. + # + # @param timeout [Numeric, nil] the per-transport default budget in seconds; nil defers to + # the configuration chain + # @param logger [Dexpace::Instrumentation::Logger] where header drops are logged; defaults + # to the null logger so no caller holds a nil + # @param drop_policy [DropPolicy, nil] TRANSPORT-13's drop-logging policy; nil is + # DropPolicy's once-per-name default + # @param connection_limit [Integer, nil] the per-origin pool bound; nil reads + # Configuration::Keys::TRANSPORT_CONNECTION_LIMIT, then DEFAULT_CONNECTION_LIMIT + # @param ssl_context [OpenSSL::SSL::SSLContext, nil] a caller's whole TLS context, or nil + # @param configuration [Dexpace::Configuration, nil] the chain the limit and the configured + # timeout are read from; nil is the process-wide Dexpace.configuration + # @return [Adapter] an owning adapter (`#owned?` is true) + # @raise [Dexpace::InvalidArgumentError] for a `timeout:` that is not nil or a finite, + # positive number + def build(timeout: nil, logger: ::Dexpace::Instrumentation::Logger::NULL, drop_policy: nil, + connection_limit: nil, ssl_context: nil, configuration: nil) + Adapter.owning(timeout: timeout, logger: logger, drop_policy: drop_policy, + connection_limit: connection_limit, ssl_context: ssl_context, + configuration: configuration,) + end + + # The borrowing construction: the caller's own `Async::HTTP::Client` carries every request + # whatever its origin, and the adapter never closes it -- the caller may keep using it after + # the transport is closed (TRANSPORT-15, XCUT-22). Two consequences are the contract: the + # client is bound to the reactor it was built for, exactly as any async-http client is; and + # `retries` must already be zero, which the adapter asserts rather than sets, because + # TRANSPORT-2 scopes the disable to an SDK-managed transport and XCUT-22 forbids mutating a + # caller's object. + # + # @param client [Async::HTTP::Client] the caller's client, with `retries` zero + # @param logger [Dexpace::Instrumentation::Logger] as for `.build` + # @param drop_policy [DropPolicy, nil] as for `.build` + # @return [Adapter] a borrowing adapter (`#owned?` is false) + # @raise [Dexpace::InvalidArgumentError] when the client's `retries` is not zero + def using(client, logger: ::Dexpace::Instrumentation::Logger::NULL, drop_policy: nil) + Adapter.borrowing(client, logger: logger, drop_policy: drop_policy) + end + + # SEAM-5's zero-argument factory the registry calls: a FRESH owning adapter every call, + # never a memoized one, because a memoized default would be one adapter shared across + # every unconfigured consumer in the process. + # + # @return [Adapter] + def default + build + end + + # Require-time self-registration, the one load-time side effect this repository permits: + # the LAST statement inside the module, so `method(:default)` resolves against it. The + # `core:` keyword is phase 2's version-skew guard and raises Dexpace::SeamError on a + # Dexpace::VERSION mismatch (design §2.4). Explicit, never presence-gated: this adapter + # registers because it was required, never because `Async::HTTP` happens to be defined. + ::Dexpace::AsyncTransport.register( + REGISTRY_KEY, method(:default), + core: "~> #{::Dexpace::VERSION.split(".").first(2).join(".")}", + ) end end end diff --git a/gems/dexpace-transport-async_http/lib/dexpace/transport/async_http/adapter.rb b/gems/dexpace-transport-async_http/lib/dexpace/transport/async_http/adapter.rb new file mode 100644 index 0000000..6325da4 --- /dev/null +++ b/gems/dexpace-transport-async_http/lib/dexpace/transport/async_http/adapter.rb @@ -0,0 +1,169 @@ +# frozen_string_literal: true +# SPDX-License-Identifier: MIT + +module Dexpace + module Transport + module AsyncHTTP + # The Dexpace::AsyncTransport seam implementation (SEAM-16). `#call` returns a + # Dexpace::Async::Future before doing anything fallible, and nothing between minting the + # pivot and settling it raises to the caller: a pre-dispatch failure -- a forged header the + # wire-boundary re-validation refuses, a URL this transport cannot dispatch, a call with no + # reactor -- is delivered through the future (TRANSPORT-21), and only `Async::Cancel`, + # NoMemoryError, SystemExit, SignalException and Interrupt may propagate synchronously, which + # `rescue ::StandardError` excludes without a hand-written list. + # + # Two named entry points make ownership a construction-time fact (design §3.7): an owning + # adapter builds one `Async::HTTP::Client` per (reactor, origin) and closes them all; a + # borrowing one carries every request through the caller's client and never closes it + # (TRANSPORT-15, XCUT-22). Frozen in effect after construction -- the only writes are the + # client map's own and Closeable's latch, each under its own mutex -- and every per-call + # value lives on the Exchange and its Completer (TRANSPORT-29, ASYNC-22). + # + # The adapter creates no reactor (P8-39): `Sync {}` would block the caller's thread and + # defeat the seam, and an owned reactor thread is a thread pool by another name and a + # long-lived fiber carrying the constructor's diagnostic context rather than the caller's. + # A call from a thread with no `Fiber.scheduler` settles Dexpace::SeamError naming the fix. + class Adapter + include ::Dexpace::Closeable + + # What a call outside a reactor settles with. + REACTOR_MESSAGE = "Dexpace::Transport::AsyncHTTP requires a running Async reactor on " \ + "the calling thread; wrap the call in `Sync { }` or `Async { }`" + + private_class_method :new + + # The SDK-managed construction behind AsyncHTTP.build, which is the entry point a caller + # uses; public because the module function calls it with an explicit receiver. + # + # @api private + # @return [Adapter] + def self.owning(timeout:, logger:, drop_policy:, connection_limit:, ssl_context:, + configuration:) + timeout!(timeout) + configuration ||= ::Dexpace.configuration + clients = Clients.build(configuration: configuration, connection_limit: connection_limit, + ssl_context: ssl_context,) + new(clients: clients, client: nil, timeout: timeout, logger: logger, + drop_policy: drop_policy, configuration: configuration, owned: true,) + end + + # The borrowing construction behind AsyncHTTP.using: refuses the client here, before an + # instance exists. + # + # @api private + # @return [Adapter] + def self.borrowing(client, logger:, drop_policy:) + unless client.respond_to?(:call) && client.respond_to?(:retries) && client.retries.zero? + raise ::Dexpace::InvalidArgumentError, + "a borrowed Async::HTTP::Client must already have retries == 0 (TRANSPORT-2): " \ + "the adapter may not set it on a client it does not own (XCUT-22)" + end + + new(clients: nil, client: client, timeout: nil, logger: logger, + drop_policy: drop_policy, configuration: ::Dexpace.configuration, owned: false,) + end + + def self.timeout!(timeout) + return if timeout.nil? + return if timeout.is_a?(::Numeric) && timeout.finite? && timeout.positive? + + raise ::Dexpace::InvalidArgumentError, + "timeout must be a finite, positive number of seconds or nil, got " \ + "#{timeout.inspect}" + end + private_class_method :timeout! + + def initialize(clients:, client:, timeout:, logger:, drop_policy:, configuration:, owned:) + @clients = clients + @client = client + @timeout = timeout + @logger = logger + @drop_policy = drop_policy || DropPolicy.build + @configuration = configuration + initialize_closeable(owned: owned) + end + + # The seam: mints the pivot, refuses what must be refused through it, maps the request on + # the caller's own fiber, and hands the exchange to a child task of the caller's. + # + # @param request [Dexpace::Request] + # @param options [Dexpace::RequestOptions, nil] `#timeout` is this call's budget up to the + # response head, in seconds; nil takes the transport's default + # @param cancellation [Dexpace::Cancellation, nil] nil is the never-cancelled token + # @return [Dexpace::Async::Future] settled with a Dexpace::Response, failed with a + # Dexpace::TransportError (retryable), a Dexpace::InvalidArgumentError, a + # Dexpace::ClosedError or a Dexpace::SeamError, or cancelled with Dexpace::CancelledError + def call(request, options, cancellation) + cancellation ||= ::Dexpace::Cancellation.none + completer = ::Dexpace::Async::Completer.new + dispatch(completer, request, options, cancellation) + completer.future + end + + private + + # Steps 2 to 10, on the caller's fiber. The post-close guard settles through the future, + # because SEAM-15's "a send after close raises" names the sync seam's channel and + # TRANSPORT-21 governs this one; a borrowing adapter stays usable after its own close, as + # dexpace-transport-net_http's does, because the client is the caller's. + def dispatch(completer, request, options, cancellation) + if closed? && owned? + return completer.fail(::Dexpace::ClosedError.new("this transport is closed")) + end + + task = ::Async::Task.current? + return completer.fail(::Dexpace::SeamError.new(REACTOR_MESSAGE)) if task.nil? + + cancellation.check! + exchange_for(completer, request, options, cancellation).start(task) + rescue ::StandardError => error + # Errors.settle, never a bare #fail: Endpoints and OpenSSL raise ArgumentError, + # OpenSSL::X509::StoreError and Errno::* here -- none a Dexpace:: error -- and P6-4's + # obligation is "wrap, and default to retryable" at every site; a Dexpace:: error, the + # re-validation's InvalidArgumentError included, passes through unchanged, and a + # cancelled token -- `check!`'s raise, or a cancel racing the mapping -- settles a + # cancellation, never a failure carrying one. + Errors.settle(completer, error, phase: :connect, cancellation: cancellation) + end + + # The three tiers, highest first: the call, the transport, then the configuration chain + # through Configuration#duration, whose bare-number grammar is milliseconds (CFG-7). + # Steps 4 to 7: the native request (the re-validation and the header drops happen inside + # the mapping), the client for this origin under the calling fiber's reactor, and this + # call's budget. + def exchange_for(completer, request, options, cancellation) + Exchange.new( + completer: completer, cancellation: cancellation, request: request, + native_request: RequestMapper.call(request, logger: @logger, drop_policy: @drop_policy), + client: client_for(request.url, ::Fiber.scheduler), deadline: resolve_timeout(options), + logger: @logger, + ) + end + + def resolve_timeout(options) + options&.timeout || @timeout || configured_timeout + end + + def configured_timeout + key = ::Dexpace::Configuration::Keys::REQUEST_TIMEOUT + @configuration.duration(key, default: DEFAULT_TIMEOUT_SECONDS) || DEFAULT_TIMEOUT_SECONDS + end + + def client_for(url, reactor) + clients = @clients + return @client if clients.nil? + + clients.fetch(url, reactor: reactor) + end + + # The owning construction releases every client it built, without waiting on any of + # them (P8-37); the borrowing one never reaches here, because Closeable#close skips + # #release when `owned?` is false. + def release + @clients&.close + nil + end + end + end + end +end diff --git a/gems/dexpace-transport-async_http/lib/dexpace/transport/async_http/clients.rb b/gems/dexpace-transport-async_http/lib/dexpace/transport/async_http/clients.rb new file mode 100644 index 0000000..7213741 --- /dev/null +++ b/gems/dexpace-transport-async_http/lib/dexpace/transport/async_http/clients.rb @@ -0,0 +1,153 @@ +# frozen_string_literal: true +# SPDX-License-Identifier: MIT + +module Dexpace + module Transport + module AsyncHTTP + # The client map: one `Async::HTTP::Client` per (reactor, origin) pair, and the object + # P8-37 and the XCUT-14 cap live on. + # + # Keyed by the REACTOR as well as the origin, because one async-http client cannot be + # shared across OS threads that each run their own reactor: its pool waits on a + # Thread::Mutex and a ConditionVariable that are not the scheduler's to interrupt, and a + # fiber resumed across threads raises. Measured: two threads with a reactor each, ten GETs + # apiece through one client, never completed; one client per thread, forty of forty. So + # every thread running its own reactor gets its own client per origin, fibers inside ONE + # reactor share one multiplexed client, and ASYNC-22's "concurrent calls from multiple + # threads" holds structurally. The reactor is `Fiber.scheduler` at the call, compared by + # identity through Data's member equality, never by class. + # + # #fetch's critical section is a Hash read and a Hash insert and nothing else: the client is + # built OUTSIDE the lock, because Endpoints loads the default certificate store from disk + # for an https origin -- filesystem I/O, a scheduler suspension point, which + # concurrency-and-async/ee54cb68 forbids a lock being held across. A lost race discards an + # unused client, which costs nothing: Client.new opens no socket. A private_constant of + # AsyncHTTP. + class Clients + # The map's key: the calling fiber's scheduler and the request's origin. + Key = ::Data.define(:reactor, :origin) + private_constant :Key + + private_class_method :new + + # @param configuration [Dexpace::Configuration] + # @param connection_limit [Integer, nil] an explicit per-origin bound, else the + # configuration's TRANSPORT_CONNECTION_LIMIT, else DEFAULT_CONNECTION_LIMIT + # @param ssl_context [OpenSSL::SSL::SSLContext, nil] a caller's TLS context, or nil + # @return [Clients] + # @raise [Dexpace::InvalidArgumentError] for a limit that is not a positive Integer + def self.build(configuration:, connection_limit: nil, ssl_context: nil) + limit = connection_limit || configuration.integer( + ::Dexpace::Configuration::Keys::TRANSPORT_CONNECTION_LIMIT, + default: DEFAULT_CONNECTION_LIMIT, + ) + unless limit.is_a?(::Integer) && limit.positive? + raise ::Dexpace::InvalidArgumentError, + "connection_limit must be a positive Integer, got #{limit.inspect}" + end + + new(limit: limit, ssl_context: ssl_context) + end + + def initialize(limit:, ssl_context:) + @limit = limit + @ssl_context = ssl_context + @mutex = ::Thread::Mutex.new + @by_key = {} + end + + # @return [Integer] the per-origin pool bound every client is built with + attr_reader :limit + + # The client for this request's origin under the calling fiber's reactor, built on first + # use. XCUT-14: the insert drains back to MAX_ORIGINS in a LOOP under the same lock as the + # insert -- a client whose reactor has since closed first, then the oldest -- and every + # evicted client's pool is retired OUTSIDE the lock, because the values own pools and a + # dropped pool is a connection leak wearing a cap. + # + # @param url [URI::Generic] the request's URL + # @param reactor [Object] the calling fiber's scheduler, compared by identity + # @return [Async::HTTP::Client] + def fetch(url, reactor:) + key = Key.new(reactor: reactor, origin: Endpoints.origin_for(url)) + existing = @mutex.synchronize { @by_key[key] } + return existing if existing + + candidate = build_client(url) + client = nil + evicted = [] #: Array[untyped] + @mutex.synchronize do + client = (@by_key[key] ||= candidate) + drain(evicted) + end + evicted.each { |old| Clients.release(old) } + client + end + + # XCUT-14's bound, asserted rather than assumed. + # + # @return [Integer] + def size + @mutex.synchronize { @by_key.size } + end + + # Releases every client without waiting on any of them (P8-37). + # + # @return [nil] + def close + clients = @mutex.synchronize { @by_key.values.tap { @by_key.clear } } + clients.each { |client| Clients.release(client) } + nil + end + + # P8-37, as built: `Async::HTTP::Client#close` is `@pool.wait_until_free { Console.warn … }` + # then `@pool.close`, and async-pool's `Controller#close` is itself a `drain` that waits on + # a condition while any resource is busy -- both bounded only by the SERVER, and the first + # writes a JSON warning to the host's stderr on the way. Measured with one request in + # flight against a two-second server: 1951 ms each. The route that returns at once is to + # retire every resource first -- `Controller#retire` and `#resources` are public API -- + # after which `#close` has nothing to wait for. A connection still in flight is retired + # rather than waited on: the exchange holding it surfaces a wrapped, retryable + # Dexpace::TransportError, which is what closing a transport mid-flight means + # (TRANSPORT-16, XCUT-13). Works outside any reactor too, which `Adapter#close` from a + # test's teardown or a caller's `ensure` relies on. + # + # @param client [Async::HTTP::Client] + # @return [nil] + def self.release(client) + pool = client.pool + pool.resources.each_key { |resource| pool.retire(resource) } + ::Dexpace.close_quietly(pool) + end + + private + + # Under the lock: back to MAX_ORIGINS, closed reactors first, then the oldest. + def drain(evicted) + return if @by_key.size <= MAX_ORIGINS + + evict_closed_reactors(evicted) + evicted << @by_key.delete(@by_key.keys.first) while @by_key.size > MAX_ORIGINS + end + + def evict_closed_reactors(evicted) + @by_key.delete_if do |key, client| + next false unless key.reactor.respond_to?(:closed?) && key.reactor.closed? + + evicted << client + true + end + end + + # `retries: 0` is TRANSPORT-2, TRANSPORT-17 and TRANSPORT-18 on this adapter: the default + # is 3 and the SDK pipeline is the single retry authority. `limit:` reaches the pool. + def build_client(url) + endpoint = Endpoints.for(url, ssl_context: @ssl_context) + ::Async::HTTP::Client.new(endpoint, retries: 0, limit: @limit) + end + end + + private_constant :Clients + end + end +end diff --git a/gems/dexpace-transport-async_http/lib/dexpace/transport/async_http/drop_policy.rb b/gems/dexpace-transport-async_http/lib/dexpace/transport/async_http/drop_policy.rb new file mode 100644 index 0000000..95ee742 --- /dev/null +++ b/gems/dexpace-transport-async_http/lib/dexpace/transport/async_http/drop_policy.rb @@ -0,0 +1,119 @@ +# frozen_string_literal: true +# SPDX-License-Identifier: MIT + +module Dexpace + module Transport + module AsyncHTTP + # TRANSPORT-13 (SHOULD), and the OBS-19 header-drop policy phase 5b postponed to the first + # adapter that drops a caller-set header rather than raising on it: how a TRANSPORT-12 drop + # is logged, in one of three modes, with the per-name dedup mode case-insensitive and bounded + # so an attacker synthesising distinct names cannot grow it without limit. A TRANSPORT-11 + # framing drop never goes through this policy and is always VERBOSE: three modes over a set + # the caller controls would let an attacker suppress a WARNING by exhausting the bound. + # + # Not itself a Data.define value, although phase 5b's hand-forward reads that way: a Data is + # frozen at the end of its #initialize, and this object's whole job is to grow a bounded + # per-name table over its own lifetime. What IS frozen Data is the SNAPSHOT it holds in one + # ivar and replaces wholesale under its own mutex on the write path, reading it without a + # lock everywhere else -- concurrency-and-async/f414b864's shape, and phase 2's Registry's. + class DropPolicy + # The frozen table of folded names already warned about. + Snapshot = ::Data.define(:seen) + private_constant :Snapshot + + # TRANSPORT-13's bound: after this many distinct folded names the once-per-name mode stops + # tracking and degrades to the quiet mode for every further name, rather than growing. + MAX_TRACKED_NAMES = 64 + + # Every drop at WARNING. + EVERY = :every + + # The default: the first drop per distinct folded name at WARNING, the rest at VERBOSE. + ONCE_PER_NAME = :once_per_name + + # Every drop at VERBOSE. + QUIET = :quiet + + # The closed set of modes. + MODES = [EVERY, ONCE_PER_NAME, QUIET].freeze + + private_class_method :new + + # The validating factory; a mode outside MODES is refused here rather than degraded. + # + # @param mode [Symbol] one of MODES + # @return [DropPolicy] + # @raise [Dexpace::InvalidArgumentError] for a mode outside MODES + def self.build(mode: ONCE_PER_NAME) + unless MODES.include?(mode) + raise ::Dexpace::InvalidArgumentError, + "mode must be one of #{MODES.inspect}, got #{mode.inspect}" + end + + new(mode: mode) + end + + def initialize(mode:) + @mode = mode + @mutex = ::Thread::Mutex.new + seen = {} #: Hash[String, bool] + @snapshot = Snapshot.new(seen: seen.freeze) + end + + # @return [Symbol] the mode this policy was built with + attr_reader :mode + + # Logs one drop under the shared transport event, at the severity this policy's mode + # picks for this name -- TRANSPORT-13's own conformance clause: "under once-per-header + # assert the same name warns once then goes quiet, a different name warns once." Every + # emission runs inside Instrumentation.contain, because OBS-20's "every log-emission site" + # is not scoped to phase 5's own sites. + # + # @param logger [Dexpace::Instrumentation::Logger] + # @param name [String] the header name as the caller spelled it + # @param reason [String] why it was dropped + # @return [nil] + def report(logger, name, reason) + severity = severity_for(name) + event = ::Dexpace::Instrumentation::Events::TRANSPORT_HEADER_DROPPED + ::Dexpace::Instrumentation.contain(logger, event: event) do + logger.event(severity).event(event).field("header", name.to_s).field("reason", reason) + .emit + end + nil + end + + private + + def severity_for(name) + case @mode + when EVERY then ::Dexpace::Instrumentation::Severity::WARNING + when QUIET then ::Dexpace::Instrumentation::Severity::VERBOSE + else once_per_name_severity(name) + end + end + + def once_per_name_severity(name) + if first_sighting?(name.to_s.downcase) + ::Dexpace::Instrumentation::Severity::WARNING + else + ::Dexpace::Instrumentation::Severity::VERBOSE + end + end + + # Folded with `downcase` and no locale argument (HTTP-13). The mutex is held across the + # snapshot swap and nothing else; the bound is checked before the table grows, so the + # 65th distinct name is neither tracked nor warned about. + def first_sighting?(folded) + @mutex.synchronize do + seen = @snapshot.seen + next false if seen.key?(folded) || seen.size >= MAX_TRACKED_NAMES + + @snapshot = Snapshot.new(seen: seen.merge(folded => true).freeze) + true + end + end + end + end + end +end diff --git a/gems/dexpace-transport-async_http/lib/dexpace/transport/async_http/endpoints.rb b/gems/dexpace-transport-async_http/lib/dexpace/transport/async_http/endpoints.rb new file mode 100644 index 0000000..06657da --- /dev/null +++ b/gems/dexpace-transport-async_http/lib/dexpace/transport/async_http/endpoints.rb @@ -0,0 +1,81 @@ +# frozen_string_literal: true +# SPDX-License-Identifier: MIT + +require "openssl" + +module Dexpace + module Transport + module AsyncHTTP + # Dexpace::Request#url -> Async::HTTP::Endpoint, and the origin the client map is keyed on. + # + # Never calls Async::HTTP::Endpoint.parse: that method routes through URI.parse, i.e. + # URI::DEFAULT_PARSER, which IS URI::RFC3986_PARSER on 3.4 and RFC2396_PARSER below it -- + # the straddle design §3.5 pins against everywhere else. Dexpace::URL.parse! already parsed + # the request's URL with URI::RFC3986_PARSER at phase 1's construction time, and + # Endpoint.new takes that object as it is (the design's verified fact 13). A private_constant + # of AsyncHTTP. + module Endpoints + extend self + + # The two schemes this transport can dispatch. Dexpace::URL.parse! admits `ftp://` and + # Endpoint.new would dial its port 21, so the screen is here and the answer is + # InvalidArgumentError through the future (TRANSPORT-21), never a connect to the wrong + # service (the same screen 6b's Location and 7c's next-page target apply). + SCHEMES = %w[http https].freeze + private_constant :SCHEMES + + # @param url [URI::Generic] the request's already-parsed URL + # @param ssl_context [OpenSSL::SSL::SSLContext, nil] a caller's context, used verbatim + # @return [Async::HTTP::Endpoint] + # @raise [Dexpace::InvalidArgumentError] for a scheme other than http or https + def for(url, ssl_context: nil) + screen!(url) + return ::Async::HTTP::Endpoint.new(url) if url.scheme.to_s.downcase == "http" + + ::Async::HTTP::Endpoint.new(url, ssl_context: ssl_context || default_ssl_context) + end + + # The screen, run by RequestMapper before the request target is read -- a URI::FTP has + # no `#request_uri` -- and by #for before an endpoint is built. + # + # @param url [URI::Generic] + # @return [nil] + # @raise [Dexpace::InvalidArgumentError] for a scheme other than http or https + def screen!(url) + return nil if SCHEMES.include?(url.scheme.to_s.downcase) + + raise ::Dexpace::InvalidArgumentError, + "the async transport dispatches http and https only, got the scheme " \ + "#{url.scheme.inspect} (TRANSPORT-21)" + end + + # The origin: scheme, host and port, folded, so two URLs on one origin share one client + # and one pool (TRANSPORT-29's "same native client" is per origin here). + # + # @param url [URI::Generic] + # @return [Array(String, String, Integer)] frozen + def origin_for(url) + [url.scheme.to_s.downcase, url.host.to_s.downcase, url.port].freeze + end + + private + + # The two library defaults this adapter overrides (the design's TLS defaults): the + # endpoint's own `ssl_verify_mode` is VERIFY_NONE for any hostname matching `localhost`, + # which is a convenience in a web framework and a silent downgrade in an SDK; and a + # caller-supplied context is used verbatim with `alpn_protocols` never set on it, so + # HTTP/2 over TLS is unreachable unless something sets it. This is the something, and + # `set_params` with VERIFY_PEER also installs the default certificate store and hostname + # verification -- filesystem I/O, which is why Clients builds outside its lock. + def default_ssl_context + context = ::OpenSSL::SSL::SSLContext.new + context.set_params(verify_mode: ::OpenSSL::SSL::VERIFY_PEER) + context.alpn_protocols = ALPN_PROTOCOLS + context + end + end + + private_constant :Endpoints + end + end +end diff --git a/gems/dexpace-transport-async_http/lib/dexpace/transport/async_http/errors.rb b/gems/dexpace-transport-async_http/lib/dexpace/transport/async_http/errors.rb new file mode 100644 index 0000000..887a74e --- /dev/null +++ b/gems/dexpace-transport-async_http/lib/dexpace/transport/async_http/errors.rb @@ -0,0 +1,109 @@ +# frozen_string_literal: true +# SPDX-License-Identifier: MIT + +module Dexpace + module Transport + module AsyncHTTP + # TRANSPORT-3, TRANSPORT-4 and TRANSPORT-20 in one function, and P6-4's obligation read + # literally: "wrap, and default to retryable, not wrap and get the classification right by + # hand". The wrap is a CATCH-ALL, never a lookup against an enumerated list -- a list has + # exactly one failure mode and it is the bad one: a family nobody thought of escapes + # unwrapped and classifies NOT retryable through RETRY-2's capability query. The families + # this adapter actually meets are documented here rather than branched on: Async::TimeoutError + # (a StandardError, the deadline), Errno::* through SystemCallError, SocketError (which + # covers Socket::ResolutionError where it exists), OpenSSL::SSL::SSLError, EOFError (a stale + # keep-alive connection, or a body shorter than its Content-Length), IOError (a connection + # retired or closed under a read), Protocol::HTTP::Error and everything beneath it -- + # RefusedError, RemoteError, Protocol::HTTP1::BadRequest, Protocol::HTTP2::StreamError -- + # and the NoMethodError a read on a retired connection raises. Not one of them but EOFError + # and IOError is an ::IOError (the design's verified fact 11), which is what makes + # Dexpace::TransportError the only thing that lets RETRY-2 see any of them as retryable. + # Async::Cancel never reaches here: it is not a StandardError, and every caller of #wrap is + # a `rescue ::StandardError` arm. A private_constant of AsyncHTTP. + module Errors + extend self + + # The error to settle the future with for whatever escaped the native dispatch. + # + # @param error [Exception] whatever escaped + # @param phase [Symbol] :connect, :write, :read or :close, for TransportError#phase + # @param cancellation [Dexpace::Cancellation] the token this call was given + # @return [Exception] a Dexpace::CancelledError when the token is cancelled (the caller + # settles it through Completer#request_cancel), the error itself when it is already a + # Dexpace::Error, and a retryable Dexpace::TransportError carrying the original as + # `#cause` otherwise + def wrap(error, phase:, cancellation:) + # TRANSPORT-3: ask the TOKEN first, never the exception. A cancel delivered by retiring + # the connection and a peer reset arrive as the SAME IOError with the SAME message, so + # discrimination by class or by message cannot work at all. + return ::Dexpace::CancelledError.new(cancellation.reason) if cancellation.cancelled? + # Never re-wrap what is already ours: double-wrapping a stream-contract violation or a + # caller's own InvalidArgumentError into an always-retryable transport failure would + # make RETRY-2 re-send on a caller's own bug. + return error if own?(error) + + # TRANSPORT-4 and TRANSPORT-20: retryable, and the token is never written here, so a + # deadline leaves the cancellation flag exactly as it found it. + with_cause(::Dexpace::TransportError.new(error.message, phase: phase), error) + end + + # Settles the pivot from whatever escaped: a cancelled token settles a CANCELLATION + # through Completer#request_cancel -- the one settlement Future#cancelled? reads as true + # (ASYNC-6) -- and everything else a failure through Completer#fail, wrapped as #wrap + # wraps it. The one place on this adapter a failure meets the pivot. + # + # @param completer [Dexpace::Async::Completer] + # @param error [Exception] whatever escaped + # @param phase [Symbol] as for #wrap + # @param cancellation [Dexpace::Cancellation] as for #wrap + # @return [Boolean] whether this settlement won the race + def settle(completer, error, phase:, cancellation:) + wrapped = wrap(error, phase: phase, cancellation: cancellation) + if wrapped.is_a?(::Dexpace::CancelledError) + completer.request_cancel(wrapped.reason) + else + completer.fail(wrapped) + end + end + + # A body read that failed after the head arrived is phase 3a's contract, never the + # transport family (P3-3: StreamError is a sibling of TransportError, not a subclass): the + # response existed, so RETRY-2 has nothing to re-send. The token still comes first, because + # the cancellation watcher closes the native body from inside the reactor and the blocked + # read wakes with a bare IOError. + # + # @param error [Exception] what `Protocol::HTTP::Body::Readable#read` raised + # @param cancellation [Dexpace::Cancellation] the token the response was obtained under + # @return [Exception] a Dexpace::CancelledError, the error itself when it is already a + # Dexpace::Error, or a Dexpace::StreamError carrying the original as `#cause` + def classify_read(error, cancellation:) + return ::Dexpace::CancelledError.new(cancellation.reason) if cancellation.cancelled? + return error if own?(error) + + with_cause(::Dexpace::StreamError.new("the response body failed mid-stream: " \ + "#{error.class}: #{error.message}"), error,) + end + + private + + # `#cause` is set by raising inside a rescue of the original, because on the async path the + # wrapped error is handed to Completer#fail rather than raised from inside a `rescue` where + # Ruby would attach it for free (pipeline/7ce4431d's discipline, from the other side: an + # error CARRIED is never raised bare). Exception.new takes no `cause:` keyword. + def with_cause(wrapped, original) + raise wrapped, cause: original + rescue ::Dexpace::TransportError, ::Dexpace::StreamError => error + error + end + + # Dexpace::Error is a module, so an `is_a?` on it in the caller would narrow the local to + # the module's type and lose `Exception` for Steep; the predicate keeps the type whole. + def own?(error) + error.is_a?(::Dexpace::Error) + end + end + + private_constant :Errors + end + end +end diff --git a/gems/dexpace-transport-async_http/lib/dexpace/transport/async_http/exchange.rb b/gems/dexpace-transport-async_http/lib/dexpace/transport/async_http/exchange.rb new file mode 100644 index 0000000..9d5d2ef --- /dev/null +++ b/gems/dexpace-transport-async_http/lib/dexpace/transport/async_http/exchange.rb @@ -0,0 +1,212 @@ +# frozen_string_literal: true +# SPDX-License-Identifier: MIT + +module Dexpace + module Transport + module AsyncHTTP + # One call's exchange: steps 11 to 18 of the dispatch path, run inside a CHILD task of the + # caller's own Async::Task under this call's `with_timeout`, and the cancellation bridge in + # both directions (ASYNC-6, TRANSPORT-7, TRANSPORT-8, TRANSPORT-9). Every per-call value + # lives here and on the Completer, never on the adapter (ASYNC-22, TRANSPORT-29). + # + # The bridge, as built. `Async::Task#cancel` on a task whose fiber is not current runs + # `Fiber.scheduler.raise`, and `Fiber.scheduler` is nil on any OS thread but the reactor's + # -- so a cancellation hook that reached the task directly would raise NoMethodError on the + # canceller's thread and leave the exchange running, and the conformance suite cancels its + # token from an OS thread. Instead every cancellation arrives through a Thread::Queue: the + # token's hook settles the Completer cancelled and pushes its reason, `Future#cancel` settles + # the Completer whose own hook pushes the reason, and a transient WATCHER task on the + # caller's task pops the queue -- a scheduler-aware wait from inside the reactor, wakeable + # from any thread -- and acts ON THE REACTOR'S THREAD: it cancels the exchange task while the + # exchange is in flight, and closes the delivered response afterwards, which is what wakes a + # consumer blocked in a body read. Closing the native body from the canceller's thread + # instead corrupts the reactor's selector (measured: `IOError: stream closed in another + # thread` out of the reactor itself). The queue is closed when the exchange ends undelivered + # or when the delivered body is released, so the watcher wakes with nil and exits; + # `transient: true` keeps a body a caller never closes from holding the caller's `Sync` + # block open. A hook can still run after that close -- the source and the completer both + # steal their hooks before they notify -- so the push is total over it (#signal) and a cancel + # that lost the race against the exchange's own end never raises back into the canceller. + # + # `Async::Cancel` is not a StandardError and leaves this task through the + # `rescue ::Exception` arm below, which re-raises it unchanged after settling the pivot + # cancelled -- the repository's own shape for an exit that must run on a cancellation and + # must not swallow it (typed_response.rb, redirect/step.rb, page/items.rb). No `$!` is read + # anywhere. A private_constant of AsyncHTTP. + class Exchange + # @param completer [Dexpace::Async::Completer] the pivot this call settles + # @param cancellation [Dexpace::Cancellation] the caller's token + # @param client [Async::HTTP::Client] the client this origin's exchange goes through + # @param native_request [Protocol::HTTP::Request] the mapped request + # @param request [Dexpace::Request] the request the response answers + # @param deadline [Float] this call's budget, in seconds + # @param logger [Dexpace::Instrumentation::Logger] + def initialize(completer:, cancellation:, client:, native_request:, request:, deadline:, + logger:) + @completer = completer + @cancellation = cancellation + @client = client + @native_request = native_request + @request = request + @deadline = deadline + @logger = logger + @queue = ::Thread::Queue.new + @subscription = nil + @task = nil + @native = nil + @adapted = nil + @delivered = false + @finishing = false + end + + # The watcher task's annotation: what a reactor's task tree shows for it, and what the + # adapter's suite reads to prove the watcher is gone once an exchange ends undelivered. + WATCHER_ANNOTATION = "Dexpace::Transport::AsyncHTTP exchange watcher" + + # Spawns the watcher and then the exchange, both children of `caller_task` through its own + # `#async` (on async 2.46 `Kernel#Async` inside a task delegates to the same call, so the + # spelling is the honest one rather than a distinction the runtime still draws), and + # returns; a child runs to its first suspension point before `async` returns, so a client + # that answers without suspending has already settled the pivot when this method does. + # + # @param caller_task [Async::Task] the caller's current task + # @return [void] + def start(caller_task) + caller_task.async(transient: true, annotation: WATCHER_ANNOTATION) { watch } + @task = caller_task.async { |task| run(task) } + nil + end + + private + + # Steps 11 to 18. An already-cancelled pivot is noticed before any I/O; every exit closes + # what was not delivered and settles what was not settled. + def run(task) + subscribe + return finish { nil } if @completer.settled? + + task.with_timeout(@deadline) { perform } + @finishing = true + rescue ::Dexpace::CancelledError => error + # Check-after-resume's own discovery path: the token was cancelled while the native + # call was suspended and `check!` saw it before the watcher's cancel landed. Settled + # through #request_cancel, never #fail, so Future#cancelled? reads true (ASYNC-6). + finish { @completer.request_cancel(error.reason) } + rescue ::StandardError => error + # Step 17: everything else is wrapped retryable, or passed through when it is already a + # Dexpace:: error; the token is asked first (TRANSPORT-3), and a cancelled token settles + # a cancellation rather than a failure carrying one. + finish { Errors.settle(@completer, error, phase: :connect, cancellation: @cancellation) } + rescue ::Exception => error # rubocop:disable Lint/RescueException -- Async::Cancel < Exception: the runtime's own cancellation (a parent task cancelled, the reactor torn down, or the watcher acting on the pivot) must close the undelivered response and settle the pivot cancelled, then propagate unchanged so the task settles :cancelled (R13, TRANSPORT-8) + finish { @completer.request_cancel(@cancellation.reason || :async_cancelled) } + raise + ensure + net + end + + # A cancellation raised inside one of #run's arms would have left the pivot unsettled: + # the net that makes "the future always settles" a property of the code. The watcher + # stays for the life of a DELIVERED response (the body's release ends it) and is released + # here on every other exit. + def net + @completer.request_cancel(:async_cancelled) unless @completer.settled? + release_watch unless @delivered + end + + # The native call, check-after-resume, adaptation and delivery, under the deadline. + def perform + @native = @client.call(@native_request) + @cancellation.check! + @adapted = ResponseMapper.call(@native, request: @request, logger: @logger, + cancellation: @cancellation, + head: @request.method.token == "HEAD", + on_release: -> { release_watch },) + # Step 18: Settlement's own "exactly one of response/error" makes a null success + # unreachable (TRANSPORT-23); a lost race closes the response it was handed (SEAM-30). + @delivered = @completer.fulfil(@adapted) + end + + # The token drives the pivot AND the queue -- after delivery the pivot is settled and + # `request_cancel` is a no-op, so the queue is what still reaches a body read -- and the + # pivot drives the queue, for `Future#cancel`. Both hooks run inline when their subject is + # already cancelled, and both may run AFTER the exchange has ended (#signal). + def subscribe + @subscription = @cancellation.on_cancel do |reason| + @completer.request_cancel(reason) + signal(reason) + end + @completer.on_cancel { |reason| signal(reason) } + end + + # A hook's push, total over the exchange's end. `Cancellation::Source#cancel` and + # `Completer#settle` each steal their hook list under their own mutex and run it outside, + # on the CANCELLER's thread; an exchange that finished in between -- check-after-resume + # saw the flag the cancel had already flipped, settled the pivot cancelled and closed this + # queue through #release_watch, whose detach reached a list the source no longer held; or + # a delivered body released in that same window -- has nothing left for the hook to do, + # and `Thread::Queue#push` on the closed queue raises `ClosedQueueError`, which + # `Hooks.notify` would hand back to the caller's own `Source#cancel` or `Future#cancel`. + # A cancel that lost that race is not the caller's failure: the pivot is settled -- + # cancelled, or with a response whose body is already released -- and the token reads + # cancelled, so the raise is swallowed here and nowhere else. A `closed?` check first + # would be the same race one instruction later. + def signal(reason) + @queue.push(reason) + nil + rescue ::ClosedQueueError + nil + end + + # R13: the undelivered response is closed on EVERY exit, before the pivot is settled, so + # a waiter that wakes finds the connection already released. Once adapted, the + # Dexpace::Response owns the native body and a lost `fulfil` race has closed it already; + # before that, the native body is the exchange's to close. + def finish + @finishing = true + ::Dexpace.close_quietly(@native&.body, logger: @logger) if @adapted.nil? + yield + end + + # The watcher, on the reactor's thread: a reason means the pivot was cancelled -- from + # the token, from Future#cancel, or from a pre-dispatch cancellation -- and nil means the + # exchange ended or the delivered body was released. While the exchange is in flight the + # task is cancelled, which is what reaches a native call blocked in a read or in the + # pool's acquire (TRANSPORT-7); once the response is delivered the response itself is + # closed, which is what reaches a consumer blocked in a body read, and never a delivered + # response whose pivot was not cancelled (ASYNC-20). An exchange already in its exit path + # is left to it: a cancel landing inside a close would leave the pivot to the ensure's net. + # + # The `cause:` is the SDK's own reason as an exception so the task's `Async::Cancel` + # names it for whoever reads the task -- a debugger, the reactor's task tree -- rather + # than the runtime's generic "Cancelling task!", which is what a non-Exception cause is + # replaced with. Nothing in this adapter reads it back: by the time the watcher acts the + # pivot is already settled cancelled with the reason (the token's hook settles it before + # it pushes; Future#cancel settled it to run its hook at all), so #run's exit arm has + # nothing left to settle and the reason a caller sees travelled through the token and the + # completer, never through the Cancel. + def watch + reason = @queue.pop + return if reason.nil? + + if @delivered + ::Dexpace.close_quietly(@adapted, logger: @logger) + elsif !@finishing + @task&.cancel(cause: ::Dexpace::CancelledError.new(reason)) + end + end + + # Idempotent: the queue's close wakes the watcher with nil, and Subscription#detach is a + # no-op the second time. Reached from the exchange's own exit when nothing was delivered, + # and from the delivered body's release otherwise -- through the mapper at once when + # there is no body at all. + def release_watch + @queue.close + @subscription&.detach + nil + end + end + + private_constant :Exchange + end + end +end diff --git a/gems/dexpace-transport-async_http/lib/dexpace/transport/async_http/request_body.rb b/gems/dexpace-transport-async_http/lib/dexpace/transport/async_http/request_body.rb new file mode 100644 index 0000000..659e922 --- /dev/null +++ b/gems/dexpace-transport-async_http/lib/dexpace/transport/async_http/request_body.rb @@ -0,0 +1,80 @@ +# frozen_string_literal: true +# SPDX-License-Identifier: MIT + +module Dexpace + module Transport + module AsyncHTTP + # A `Protocol::HTTP::Body::Readable` over a Dexpace::Body, so an outbound body is never + # materialised: the library pulls one `#read` per chunk plus one for end of stream and + # frames a body of unknown length as chunked (the design's verified fact 7). + # `Protocol::HTTP::Body::Buffered.wrap` is deliberately NOT used -- it materialises an + # `#each`-yielding object in full, which is SEAM-11's streaming intent broken on the write + # side. + # + # The pull is phase 3a's own `Dexpace::IO::BufferedSource.over(body)`, which owns nothing + # and reads `#each` on demand; a replayable body is re-read through a fresh source on + # `#rewind`, and a single-use one refuses -- the second of TRANSPORT-17's two independent + # guarantees on this adapter beside `retries: 0`. A private_constant of AsyncHTTP. + class RequestBody < ::Protocol::HTTP::Body::Readable + # One pull's ceiling; a chunk the body yields is never split below it and never coalesced + # beyond it. + READ_SIZE = 65_536 + private_constant :READ_SIZE + + # @param body [Dexpace::Body] + def initialize(body) + super() + @body = body + @source = ::Dexpace::IO::BufferedSource.over(body) + end + + # The library's own protocol: an Integer frames the body with Content-Length, nil frames + # it chunked. BODY-35's -1 sentinel is this adapter's nil. + # + # @return [Integer, nil] + def length + value = @body.content_length + value.negative? ? nil : value + end + + # @return [Boolean] whether `#rewind` will succeed -- the body's own BODY-1 answer + def rewindable? + @body.replayable? + end + + # A fresh source over a replayable body; false for a single-use one, so the library's own + # `Request#retry!` never re-reads it even at a non-zero retry count. + # + # @return [Boolean] + def rewind # rubocop:disable Naming/PredicateMethod -- the library's own name for a command reporting whether it took effect, as Async::Pool's own bodies spell it + return false unless rewindable? + + @source = ::Dexpace::IO::BufferedSource.over(@body) + true + end + + # One chunk, BINARY, or nil at end of stream. + # + # @return [String, nil] + def read + @source.readpartial(READ_SIZE).b + rescue ::Dexpace::EndOfStreamError + nil + end + + # Releases the pull source; the Dexpace::Body itself is the caller's and is never closed + # here (design §10.12: a body closes exactly the sources it opened, and this opened none). + # + # @param error [Exception, nil] the library's own argument, unused + # @return [nil] + def close(error = nil) + super + @source.close + nil + end + end + + private_constant :RequestBody + end + end +end diff --git a/gems/dexpace-transport-async_http/lib/dexpace/transport/async_http/request_mapper.rb b/gems/dexpace-transport-async_http/lib/dexpace/transport/async_http/request_mapper.rb new file mode 100644 index 0000000..039e5c3 --- /dev/null +++ b/gems/dexpace-transport-async_http/lib/dexpace/transport/async_http/request_mapper.rb @@ -0,0 +1,127 @@ +# frozen_string_literal: true +# SPDX-License-Identifier: MIT + +module Dexpace + module Transport + module AsyncHTTP + # Steps 4 to 9 of the dispatch path as one function returning a `Protocol::HTTP::Request`: + # the wire-boundary re-validation, the framing-header drop (TRANSPORT-11), the wire-grammar + # drop (TRANSPORT-12, TRANSPORT-13, P8-40), Content-Type authority (TRANSPORT-10) and the + # native request. The only place in this gem that touches Dexpace::HeaderSyntax, the only + # place that reads FRAMING_HEADERS, and the only place that constructs a native request. + # Runs on the caller's own fiber before any task exists, so a raise here reaches the + # future without a reactor turn. A private_constant of AsyncHTTP. + module RequestMapper + extend self + + # @param request [Dexpace::Request] the request to send, or anything duck-typed like one + # @param logger [Dexpace::Instrumentation::Logger] where each framing drop is logged at + # VERBOSE + # @param drop_policy [DropPolicy] how a wire-grammar drop is logged + # @return [Protocol::HTTP::Request] + # @raise [Dexpace::InvalidArgumentError] when an outbound header name or value fails the + # wire-boundary re-validation (HTTP-17, HTTP-18) + def call(request, logger:, drop_policy:) + Endpoints.screen!(request.url) + revalidate!(request) + + fields = ::Protocol::HTTP::Headers.new + content_type = copy_headers(request, fields, logger, drop_policy) + set_content_type(request, fields) unless content_type + # The screen above admitted http and https alone, and URL.parse! builds the class from + # the scheme, so the URL is a URI::HTTP and #request_uri is its own. + url = request.url #: URI::HTTP + ::Protocol::HTTP::Request.new( + url.scheme, authority_for(url), request.method.token, url.request_uri, nil, fields, + body_for(request), + ) + end + + private + + # The wire-boundary re-validation phase 1 postponed to the adapters (HTTP-17, HTTP-18, + # XCUT-18): every outbound name and value is checked again immediately before dispatch, + # BEFORE anything is copied, because HTTP-2's constructor privacy is bypassable and a + # duck-typed impostor can reach this code with no Dexpace validation ever having run. On + # the HTTP/2 path this is the ONLY validation between the model and the wire: + # protocol-http2 transmits a CRLF-bearing value verbatim (the design's verified fact 4). + def revalidate!(request) + request.headers.each_entry do |name, value| + ::Dexpace::HeaderSyntax.validate_name!(name) + ::Dexpace::HeaderSyntax.validate_outbound_value!(value, name: name) + end + end + + # Three fates for a caller's header, decided on the folded name and then on the RFC 7230 + # token grammar: a framing header is dropped and logged at VERBOSE (TRANSPORT-11); a name + # the token grammar refuses is dropped and reported through the policy (TRANSPORT-12/13); + # everything else is copied as spelled. The token predicate is applied on BOTH protocols + # (P8-40): HTTP-17 admits seventeen bytes the tchar set refuses (`"(),/:;<=>?@[\]{}`), + # protocol-http1 raises on such a name only AFTER the request line and `host:` are on the + # socket, and protocol-http2 transmits it lowercased and unvalidated -- so the drop is a + # pre-dispatch predicate, never a rescue, and one request produces one header set whatever + # ALPN negotiated. `HeaderSyntax.token?` is byte-exact and total over invalid UTF-8, which + # a regexp is not. Returns whether the caller set a Content-Type of their own. + def copy_headers(request, fields, logger, drop_policy) + content_type = false + request.headers.each_entry do |name, value| + folded = ::Dexpace::HeaderName.of(name).folded + if FRAMING_HEADERS.include?(folded) + log_framing_drop(logger, name) + elsif !::Dexpace::HeaderSyntax.token?(name) + drop_policy.report(logger, name, "not an RFC 7230 token (TRANSPORT-12)") + else + content_type ||= folded == "content-type" + fields.add(name, value) + end + end + content_type + end + + # TRANSPORT-10: the caller's explicit header wins (already copied, matched folded); failing + # that the body's own media type. No default is invented for a body with no media type: + # async-http stamps none itself, and the four-member Request is the whole truth about what + # goes out (HTTP-6) -- the one departure from dexpace-transport-net_http, whose + # octet-stream default exists to pre-empt Net::HTTP's form-type fallback. + def set_content_type(request, fields) + media = request.body&.media_type + fields.add("content-type", media.render) if media + end + + # The body as the library pulls it, or nil: a body-less request goes out as a zero-length + # body with `content-length: 0`, which async-http writes itself (TRANSPORT-26). The framing + # is the library's, derived from RequestBody#length, and never copied from a header. + def body_for(request) + body = request.body + return nil if body.nil? || request.method.body_forbidden? + + RequestBody.new(body) + end + + # `host:` is written by async-http from the request's authority and a caller's value would + # be APPENDED beside it (fact 5), which is why `host` is in FRAMING_HEADERS. The + # scheme-default port is elided as HTTP wants it. + def authority_for(url) + port = url.port + port && port != url.default_port ? "#{url.host}:#{port}" : url.host.to_s + end + + # TRANSPORT-11's SHOULD: each drop at VERBOSE through the one containment helper (OBS-20), + # under the shared transport event with the shared field pair, so one conformance + # assertion reads a drop record from either adapter. Never through DropPolicy. + def log_framing_drop(logger, name) + event = ::Dexpace::Instrumentation::Events::TRANSPORT_HEADER_DROPPED + ::Dexpace::Instrumentation.contain(logger, event: event) do + logger.event(::Dexpace::Instrumentation::Severity::VERBOSE) + .event(event) + .field("header", name.to_s) + .field("reason", "transport framing header (TRANSPORT-11)") + .emit + end + end + end + + private_constant :RequestMapper + end + end +end diff --git a/gems/dexpace-transport-async_http/lib/dexpace/transport/async_http/response_body.rb b/gems/dexpace-transport-async_http/lib/dexpace/transport/async_http/response_body.rb new file mode 100644 index 0000000..380ff24 --- /dev/null +++ b/gems/dexpace-transport-async_http/lib/dexpace/transport/async_http/response_body.rb @@ -0,0 +1,133 @@ +# frozen_string_literal: true +# SPDX-License-Identifier: MIT + +module Dexpace + module Transport + module AsyncHTTP + # A Dexpace::Body over the native `Protocol::HTTP::Body::Readable`: lazy, pull-shaped, + # BINARY, one native `#read` per unit of consumer demand (the design's verified fact 6, and + # SSE-39's backpressure surviving the transport). Not core's own Dexpace::ResponseBody: that + # class closes the Dexpace::IO::BufferedSource it was handed, which is right for a source + # built with `.wrapping` and wrong for one built with `.over`, which owns nothing. This class + # holds the native body itself, closes IT through its own Closeable latch, and hands out one + # memoised `.over` source -- a fresh view per call would strand the bytes the first view had + # buffered. + # + # The include order is Dexpace::Body THEN Dexpace::Closeable, so Closeable#close sits nearer + # the class than the module's no-op default and wins (P3-23). Every failure out of the native + # read is classified through the token first and is a Dexpace::StreamError otherwise (P3-3): + # a body shorter than its Content-Length raises a bare EOFError from the library, and a body + # closed under a blocked read -- which is what a cancellation does, from inside the reactor + # -- a bare IOError. + # + # The native close goes through Dexpace.close_quietly, the one sanctioned quiet exit, and + # never bare: on HTTP/2, closing a body before it was read to the end resets the stream, and + # async-http 0.105.0 writes the RST_STREAM frame BEFORE transitioning the stream's state, so + # a peer's END_STREAM landing during that write closes the stream twice and releases the + # pooled connection twice -- `RuntimeError: Trying to reuse unacquired resource` out of + # `Input#close`, measured through a response obtained in a child task and closed unread by + # its parent, five of five. The connection is retired rather than reused, the client stays + # sound, and the raise is a library artefact a caller can do nothing with, so it is reported + # as one `http.instrumentation.close` WARNING through the adapter's logger and never reaches + # Response#close (TRANSPORT-16's idempotent, non-raising close). A private_constant of + # AsyncHTTP. + class ResponseBody + include ::Dexpace::Body + include ::Dexpace::Closeable + + attr_reader :media_type, :content_length + + # @param native [Protocol::HTTP::Body::Readable] the native body, owned from here on + # @param media_type [Dexpace::MediaType, nil] + # @param content_length [Integer] the native length, or -1 when unknown (BODY-35) + # @param cancellation [Dexpace::Cancellation] the token the response was obtained under + # @param logger [Dexpace::Instrumentation::Logger] where a native close failure is reported + # @param on_release [#call, nil] run once, after the native body is closed, whichever path + # closed it -- the exchange's watcher hands its own release over through this + def initialize(native:, media_type:, content_length:, cancellation:, + logger: ::Dexpace::Instrumentation::Logger::NULL, on_release: nil) + @native = native + @media_type = media_type + @content_length = content_length + @cancellation = cancellation + @logger = logger + @on_release = on_release + @source = nil + initialize_closeable(owned: true) + initialize_single_use + end + + # One native `#read` per yield, retagged BINARY, closed through the latch on natural + # exhaustion so a later explicit `#close` is a no-op. Check-after-resume: a token cancelled + # while a read was blocked is honoured before the chunk it returned is yielded. A read + # after `#close` raises rather than re-opening anything. + # + # @yield [String] each chunk + # @return [nil] + def each + return to_enum(:each) unless block_given? + + loop do + chunk = pull + break if chunk.nil? + + yield chunk.b + end + close + nil + end + + # HTTP-36's single write, over `#each`; single-use, as every response body is (BODY-6). + # + # @param sink [#write] + # @return [Integer] the byte count written + def write_to(sink) + claim_single_use! + written = 0 + each do |chunk| + sink.write(chunk) + written += chunk.bytesize + end + written + end + + # The same handle every call (BODY-14), built with `.over`, never `.wrapping`, so no byte + # is read ahead of demand and no one-byte-per-read defect is inherited (3a's residue). + # + # @return [Dexpace::IO::BufferedSource] + def source + @source ||= ::Dexpace::IO::BufferedSource.over(self) + end + + private + + def pull + raise ::Dexpace::ClosedError, "the response body is closed" if closed? + + chunk = @native.read + if @cancellation.cancelled? + close + raise ::Dexpace::CancelledError, @cancellation.reason + end + chunk + rescue ::Dexpace::Error + raise + rescue ::StandardError => error + ::Dexpace.close_quietly(self, logger: @logger) + raise Errors.classify_read(error, cancellation: @cancellation) + end + + # BODY-15: releases the native body -- which returns its connection to the pool, or resets + # the stream when the body was not read to the end -- idempotent through Closeable's latch, + # quietly (see the class comment), and then the exchange's own release hook. + def release + ::Dexpace.close_quietly(@native, logger: @logger) + ensure + @on_release&.call + end + end + + private_constant :ResponseBody + end + end +end diff --git a/gems/dexpace-transport-async_http/lib/dexpace/transport/async_http/response_mapper.rb b/gems/dexpace-transport-async_http/lib/dexpace/transport/async_http/response_mapper.rb new file mode 100644 index 0000000..402f679 --- /dev/null +++ b/gems/dexpace-transport-async_http/lib/dexpace/transport/async_http/response_mapper.rb @@ -0,0 +1,109 @@ +# frozen_string_literal: true +# SPDX-License-Identifier: MIT + +module Dexpace + module Transport + module AsyncHTTP + # Step 15: the native response to a Dexpace::Response -- TRANSPORT-14's lenient inbound + # copy, TRANSPORT-24's status mapping and TRANSPORT-27's two downgrades. The body handed to + # ResponseBody is the native BODY, a `Protocol::HTTP::Body::Readable`, never the response: + # `Protocol::HTTP::Response#read` is the WHOLE body joined into one String. Two clauses never + # reach here on this adapter, both raised by protocol-http1 out of the read before a + # response object exists: a malformed inbound header NAME (`Protocol::HTTP1::BadHeader`, + # P8-38, the TRANSPORT-14 waiver) and a non-numeric Content-Length + # (`Protocol::HTTP1::BadRequest`, the TRANSPORT-27 waiver) -- both wrap as retryable + # transport failures. Dexpace::Protocol admits HTTP/1.1 and HTTP/2 only (HTTP-33) and + # Dexpace::Status 100-599 (HTTP-10's reading), so an `HTTP/1.0` head and a `999` status each + # raise Dexpace::InvalidArgumentError here, after the head, with the native body closed by + # the exchange's own release -- the disposition dexpace-transport-net_http records, and the + # two phase-1 questions on phase 10's inbound list. A private_constant of AsyncHTTP. + module ResponseMapper + extend self + + # @param native [Protocol::HTTP::Response] the head, its body unread + # @param request [Dexpace::Request] the request the response answers + # @param logger [Dexpace::Instrumentation::Logger] where each malformed-header drop is + # logged at VERBOSE (TRANSPORT-14) + # @param cancellation [Dexpace::Cancellation] the token the body is read under + # @param head [Boolean] whether the request was a HEAD, whose response carries no body + # @param on_release [#call, nil] handed to the ResponseBody; run once it is closed + # @return [Dexpace::Response] + def call(native, request:, logger:, cancellation:, head: false, on_release: nil) + headers = inbound_headers(native, logger) + body = body_for(native, head, cancellation: cancellation, on_release: on_release, + media_type: media_type_for(headers), logger: logger,) + on_release&.call if body.nil? + ::Dexpace::Response.build( + request: request, protocol: native.version.to_s, status: native.status, + reason: reason_for(native), headers: headers, body: body, + ) + end + + private + + # TRANSPORT-14: a name HeaderSyntax refuses or a value the inbound grammar refuses -- a + # control byte -- is dropped, that header only, BEFORE it reaches Headers::Builder, which + # would raise on exactly those bytes and fail the whole response; obs-text is an admitted + # inbound byte and is kept. Names arrive as the protocol spelled them: preserved over + # HTTP/1.1, lowercased over HTTP/2 (RFC 9113 §8.2.1), which is why every lookup folds. + # Each drop is logged at VERBOSE by name -- never the value. + def inbound_headers(native, logger) + builder = ::Dexpace::Headers.inbound_builder + native.headers.each do |name, value| + next log_drop(logger, name) unless ::Dexpace::HeaderSyntax.valid_name?(name) + next log_drop(logger, name) unless ::Dexpace::HeaderSyntax.valid_inbound_value?(value) + + builder.add(name, value) + end + builder.build + end + + # A HEAD response, a 204 or anything else the library delivers with no body gets + # `body: nil` and the native body, when one exists, is closed at once; otherwise a + # ResponseBody over the native body, its length mapped from the library's nil to + # BODY-35's -1 sentinel. + def body_for(native, head, cancellation:, on_release:, media_type:, logger:) + body = native.body + if head || body.nil? || body.is_a?(::Protocol::HTTP::Body::Head) + ::Dexpace.close_quietly(body, logger: logger) + return nil + end + + ResponseBody.new(native: body, media_type: media_type, content_length: body.length || -1, + cancellation: cancellation, logger: logger, on_release: on_release,) + end + + # An HTTP/1.1 response carries its status line's reason phrase; an HTTP/2 one has none + # (RFC 9113 §8.3.2), and the protocol-neutral response class declares no reader for it. + def reason_for(native) + native.respond_to?(:reason) ? native.reason : nil + end + + # TRANSPORT-27: MediaType.parse RAISES on a malformed value rather than returning nil, so + # this rescue is what produces "downgraded to no media type"; the malformed value still + # reaches the caller verbatim in Dexpace::Headers. + def media_type_for(headers) + raw = headers["content-type"]&.first + return nil if raw.nil? + + ::Dexpace::MediaType.parse(raw) + rescue ::Dexpace::InvalidArgumentError + nil + end + + def log_drop(logger, name) + event = ::Dexpace::Instrumentation::Events::TRANSPORT_HEADER_DROPPED + ::Dexpace::Instrumentation.contain(logger, event: event) do + logger.event(::Dexpace::Instrumentation::Severity::VERBOSE) + .event(event) + .field("header", name.to_s.b) + .field("reason", "malformed inbound header (TRANSPORT-14)") + .emit + end + end + end + + private_constant :ResponseMapper + end + end +end diff --git a/gems/dexpace-transport-async_http/sig/dexpace/transport/async_http.rbs b/gems/dexpace-transport-async_http/sig/dexpace/transport/async_http.rbs index 97248cc..2b52a4a 100644 --- a/gems/dexpace-transport-async_http/sig/dexpace/transport/async_http.rbs +++ b/gems/dexpace-transport-async_http/sig/dexpace/transport/async_http.rbs @@ -1,6 +1,23 @@ module Dexpace module Transport + # The reference asynchronous transport over async-http (phase 8c). Nothing from async, + # async-http, protocol-http or openssl names a public signature here (NFR-11): `.using`'s + # client and `.build`'s `ssl_context:` are untyped, with their YARD blocks saying what each + # accepts. module AsyncHTTP + DEFAULT_TIMEOUT_SECONDS: Float + DEFAULT_CONNECTION_LIMIT: Integer + MAX_ORIGINS: Integer + REGISTRY_KEY: Symbol + FRAMING_HEADERS: Array[String] + ALPN_PROTOCOLS: Array[String] + + def self?.build: (?timeout: Numeric?, ?logger: Instrumentation::Logger, + ?drop_policy: DropPolicy?, ?connection_limit: Integer?, + ?ssl_context: untyped, ?configuration: Dexpace::Configuration?) -> Adapter + def self?.using: (untyped client, ?logger: Instrumentation::Logger, + ?drop_policy: DropPolicy?) -> Adapter + def self?.default: () -> Adapter end end end diff --git a/gems/dexpace-transport-async_http/sig/dexpace/transport/async_http/adapter.rbs b/gems/dexpace-transport-async_http/sig/dexpace/transport/async_http/adapter.rbs new file mode 100644 index 0000000..7f6997c --- /dev/null +++ b/gems/dexpace-transport-async_http/sig/dexpace/transport/async_http/adapter.rbs @@ -0,0 +1,51 @@ +module Dexpace + module Transport + module AsyncHTTP + # The transport (SEAM-12): private .new, two construction entry points reached through + # AsyncHTTP.build and AsyncHTTP.using, and Closeable's latch as the only state written + # after construction (TRANSPORT-29). The async-http client and the reactor are untyped: + # nothing from async-http may name a signature (NFR-11). + class Adapter + include Dexpace::Closeable + + REACTOR_MESSAGE: String + + @clients: Clients? + @client: untyped + @timeout: Numeric? + @logger: Instrumentation::Logger + @drop_policy: DropPolicy + @configuration: Dexpace::Configuration + + def self.owning: (timeout: Numeric?, logger: Instrumentation::Logger, + drop_policy: DropPolicy?, connection_limit: Integer?, + ssl_context: untyped, configuration: Dexpace::Configuration?) -> Adapter + def self.borrowing: (untyped client, logger: Instrumentation::Logger, + drop_policy: DropPolicy?) -> Adapter + private def self.timeout!: (untyped timeout) -> void + private def self.new: (clients: Clients?, client: untyped, timeout: Numeric?, + logger: Instrumentation::Logger, drop_policy: DropPolicy?, + configuration: Dexpace::Configuration, owned: bool) -> instance + def initialize: (clients: Clients?, client: untyped, timeout: Numeric?, + logger: Instrumentation::Logger, drop_policy: DropPolicy?, + configuration: Dexpace::Configuration, owned: bool) -> void + + def call: (Dexpace::Request request, Dexpace::RequestOptions? options, + Dexpace::Cancellation? cancellation) -> Dexpace::Async::Future + + private + + def dispatch: (Dexpace::Async::Completer completer, Dexpace::Request request, + Dexpace::RequestOptions? options, Dexpace::Cancellation cancellation) + -> untyped + def exchange_for: (Dexpace::Async::Completer completer, Dexpace::Request request, + Dexpace::RequestOptions? options, Dexpace::Cancellation cancellation) + -> Exchange + def resolve_timeout: (Dexpace::RequestOptions? options) -> Numeric + def configured_timeout: () -> Numeric + def client_for: (URI::Generic url, untyped reactor) -> untyped + def release: () -> nil + end + end + end +end diff --git a/gems/dexpace-transport-async_http/sig/dexpace/transport/async_http/clients.rbs b/gems/dexpace-transport-async_http/sig/dexpace/transport/async_http/clients.rbs new file mode 100644 index 0000000..7bf3aed --- /dev/null +++ b/gems/dexpace-transport-async_http/sig/dexpace/transport/async_http/clients.rbs @@ -0,0 +1,44 @@ +# Dexpace::Transport::AsyncHTTP::Clients is a private_constant and not public API: this +# declaration exists because the `async_http` Steep target checks every file under lib/ and needs +# the class declared to type Adapter's call sites. RBS has no visibility for a constant, so the +# privacy lives in lib/dexpace/transport/async_http/clients.rb alone. The reactor, the SSL context +# and the async-http clients are untyped: nothing from async-http may name a signature (NFR-11). +module Dexpace + module Transport + module AsyncHTTP + class Clients + # The map's key: the reactor by identity and the origin triple. A private_constant Data + # snapshot (phase 2's P2-9 exemption), so no Model and no .build. + class Key < Data + attr_reader reactor: untyped + attr_reader origin: Array[String | Integer | nil] + + def self.new: (reactor: untyped, origin: Array[String | Integer | nil]) -> instance + end + + @limit: Integer + @ssl_context: untyped + @mutex: Thread::Mutex + @by_key: Hash[Key, untyped] + + attr_reader limit: Integer + + def self.build: (configuration: Dexpace::Configuration, ?connection_limit: Integer?, + ?ssl_context: untyped) -> Clients + private def self.new: (limit: Integer, ssl_context: untyped) -> instance + def initialize: (limit: Integer, ssl_context: untyped) -> void + + def fetch: (URI::Generic url, reactor: untyped) -> untyped + def size: () -> Integer + def close: () -> nil + def self.release: (untyped client) -> nil + + private + + def drain: (Array[untyped] evicted) -> void + def evict_closed_reactors: (Array[untyped] evicted) -> void + def build_client: (URI::Generic url) -> untyped + end + end + end +end diff --git a/gems/dexpace-transport-async_http/sig/dexpace/transport/async_http/drop_policy.rbs b/gems/dexpace-transport-async_http/sig/dexpace/transport/async_http/drop_policy.rbs new file mode 100644 index 0000000..cf86ae4 --- /dev/null +++ b/gems/dexpace-transport-async_http/sig/dexpace/transport/async_http/drop_policy.rbs @@ -0,0 +1,41 @@ +module Dexpace + module Transport + module AsyncHTTP + # TRANSPORT-13's once-per-name, bounded drop reporting, as a policy value an adapter is + # built with. Private .new; one frozen snapshot replaced under the mutex. + class DropPolicy + # The names warned about so far, as one frozen Data replaced whole under the mutex. A + # private_constant snapshot (phase 2's P2-9 exemption), so no Model and no .build. + class Snapshot < Data + attr_reader seen: Hash[String, bool] + + def self.new: (seen: Hash[String, bool]) -> instance + end + + MAX_TRACKED_NAMES: Integer + EVERY: Symbol + ONCE_PER_NAME: Symbol + QUIET: Symbol + MODES: Array[Symbol] + + @mode: Symbol + @mutex: Thread::Mutex + @snapshot: Snapshot + + attr_reader mode: Symbol + + def self.build: (?mode: Symbol) -> DropPolicy + private def self.new: (mode: Symbol) -> instance + def initialize: (mode: Symbol) -> void + + def report: (Instrumentation::Logger logger, String name, String reason) -> nil + + private + + def severity_for: (String name) -> Instrumentation::Severity + def once_per_name_severity: (String name) -> Instrumentation::Severity + def first_sighting?: (String folded) -> bool + end + end + end +end diff --git a/gems/dexpace-transport-async_http/sig/dexpace/transport/async_http/endpoints.rbs b/gems/dexpace-transport-async_http/sig/dexpace/transport/async_http/endpoints.rbs new file mode 100644 index 0000000..b30599e --- /dev/null +++ b/gems/dexpace-transport-async_http/sig/dexpace/transport/async_http/endpoints.rbs @@ -0,0 +1,23 @@ +# Dexpace::Transport::AsyncHTTP::Endpoints is a private_constant and not public API: this +# declaration exists because the `async_http` Steep target checks every file under lib/ and needs +# the module declared to type Clients' and RequestMapper's call sites. RBS has no visibility for a +# constant, so the privacy lives in lib/dexpace/transport/async_http/endpoints.rb alone. The +# endpoint and the SSL context are untyped: nothing from async-http or openssl may name a +# signature (NFR-11). +module Dexpace + module Transport + module AsyncHTTP + module Endpoints + SCHEMES: Array[String] + + def self?.for: (URI::Generic url, ?ssl_context: untyped) -> untyped + def self?.screen!: (URI::Generic url) -> nil + def self?.origin_for: (URI::Generic url) -> Array[String | Integer | nil] + + private + + def default_ssl_context: () -> untyped + end + end + end +end diff --git a/gems/dexpace-transport-async_http/sig/dexpace/transport/async_http/errors.rbs b/gems/dexpace-transport-async_http/sig/dexpace/transport/async_http/errors.rbs new file mode 100644 index 0000000..0e2ed08 --- /dev/null +++ b/gems/dexpace-transport-async_http/sig/dexpace/transport/async_http/errors.rbs @@ -0,0 +1,23 @@ +# Dexpace::Transport::AsyncHTTP::Errors is a private_constant and not public API: this +# declaration exists because the `async_http` Steep target checks every file under lib/ and needs +# the module and its three functions declared to type Adapter's, Exchange's and ResponseBody's +# call sites. RBS has no visibility for a constant, so the privacy lives in +# lib/dexpace/transport/async_http/errors.rb alone. +module Dexpace + module Transport + module AsyncHTTP + module Errors + def self?.wrap: (Exception error, phase: Symbol, cancellation: Dexpace::Cancellation) + -> Exception + def self?.settle: (Dexpace::Async::Completer completer, Exception error, phase: Symbol, + cancellation: Dexpace::Cancellation) -> bool + def self?.classify_read: (Exception error, cancellation: Dexpace::Cancellation) -> Exception + + private + + def with_cause: (Exception wrapped, Exception original) -> Exception + def own?: (Exception error) -> bool + end + end + end +end diff --git a/gems/dexpace-transport-async_http/sig/dexpace/transport/async_http/exchange.rbs b/gems/dexpace-transport-async_http/sig/dexpace/transport/async_http/exchange.rbs new file mode 100644 index 0000000..ea6fc01 --- /dev/null +++ b/gems/dexpace-transport-async_http/sig/dexpace/transport/async_http/exchange.rbs @@ -0,0 +1,46 @@ +# Dexpace::Transport::AsyncHTTP::Exchange is a private_constant and not public API: this +# declaration exists because the `async_http` Steep target checks every file under lib/ and needs +# the class declared to type Adapter's call site. RBS has no visibility for a constant, so the +# privacy lives in lib/dexpace/transport/async_http/exchange.rb alone. The tasks, the client and +# the native request and response are untyped: nothing from async or async-http may name a +# signature (NFR-11). +module Dexpace + module Transport + module AsyncHTTP + class Exchange + WATCHER_ANNOTATION: String + + @completer: Dexpace::Async::Completer + @cancellation: Dexpace::Cancellation + @client: untyped + @native_request: untyped + @request: Dexpace::Request + @deadline: Numeric + @logger: Instrumentation::Logger + @queue: Thread::Queue + @subscription: Dexpace::Cancellation::Subscription? + @task: untyped + @native: untyped + @adapted: Dexpace::Response? + @delivered: bool + @finishing: bool + + def initialize: (completer: Dexpace::Async::Completer, cancellation: Dexpace::Cancellation, + client: untyped, native_request: untyped, request: Dexpace::Request, + deadline: Numeric, logger: Instrumentation::Logger) -> void + def start: (untyped caller_task) -> nil + + private + + def run: (untyped task) -> untyped + def perform: () -> bool + def subscribe: () -> Dexpace::Async::Completer + def signal: (untyped reason) -> nil + def net: () -> nil + def finish: () { () -> untyped } -> untyped + def watch: () -> untyped + def release_watch: () -> nil + end + end + end +end diff --git a/gems/dexpace-transport-async_http/sig/dexpace/transport/async_http/request_body.rbs b/gems/dexpace-transport-async_http/sig/dexpace/transport/async_http/request_body.rbs new file mode 100644 index 0000000..a8fd436 --- /dev/null +++ b/gems/dexpace-transport-async_http/sig/dexpace/transport/async_http/request_body.rbs @@ -0,0 +1,27 @@ +# Dexpace::Transport::AsyncHTTP::RequestBody is a private_constant and not public API: this +# declaration exists because the `async_http` Steep target checks every file under lib/ and needs +# the class declared to type RequestMapper's call site. RBS has no visibility for a constant, so +# the privacy lives in lib/dexpace/transport/async_http/request_body.rb alone. Its superclass is +# Protocol::HTTP::Body::Readable, which rbs cannot see (protocol-http ships no sig/ and the +# collection ignores the async family), so the class is declared without it here and the target +# downgrades the unknown constant to information; the methods declared are the ones the +# superclass's contract names. +module Dexpace + module Transport + module AsyncHTTP + class RequestBody + READ_SIZE: Integer + + @body: Dexpace::Body + @source: Dexpace::IO::BufferedSource + + def initialize: (Dexpace::Body body) -> void + def length: () -> Integer? + def rewindable?: () -> bool + def rewind: () -> bool + def read: () -> String? + def close: (?Exception? error) -> nil + end + end + end +end diff --git a/gems/dexpace-transport-async_http/sig/dexpace/transport/async_http/request_mapper.rbs b/gems/dexpace-transport-async_http/sig/dexpace/transport/async_http/request_mapper.rbs new file mode 100644 index 0000000..b268bc3 --- /dev/null +++ b/gems/dexpace-transport-async_http/sig/dexpace/transport/async_http/request_mapper.rbs @@ -0,0 +1,26 @@ +# Dexpace::Transport::AsyncHTTP::RequestMapper is a private_constant and not public API: this +# declaration exists because the `async_http` Steep target checks every file under lib/ and needs +# the module and its one function declared to type Adapter's call site. RBS has no visibility for +# a constant, so the privacy lives in lib/dexpace/transport/async_http/request_mapper.rb alone. +# The native request and its header table are untyped: nothing from protocol-http may name a +# signature (NFR-11). +module Dexpace + module Transport + module AsyncHTTP + module RequestMapper + def self?.call: (Dexpace::Request request, logger: Instrumentation::Logger, + drop_policy: DropPolicy) -> untyped + + private + + def revalidate!: (Dexpace::Request request) -> void + def copy_headers: (Dexpace::Request request, untyped fields, Instrumentation::Logger logger, + DropPolicy drop_policy) -> bool + def set_content_type: (Dexpace::Request request, untyped fields) -> void + def body_for: (Dexpace::Request request) -> RequestBody? + def authority_for: (URI::Generic url) -> String + def log_framing_drop: (Instrumentation::Logger logger, String name) -> nil + end + end + end +end diff --git a/gems/dexpace-transport-async_http/sig/dexpace/transport/async_http/response_body.rbs b/gems/dexpace-transport-async_http/sig/dexpace/transport/async_http/response_body.rbs new file mode 100644 index 0000000..c042eed --- /dev/null +++ b/gems/dexpace-transport-async_http/sig/dexpace/transport/async_http/response_body.rbs @@ -0,0 +1,45 @@ +# Dexpace::Transport::AsyncHTTP::ResponseBody is a private_constant and not public API: this +# declaration exists because the `async_http` Steep target checks every file under lib/ and needs +# the class declared to type ResponseMapper's call site. RBS has no visibility for a constant, so +# the privacy lives in lib/dexpace/transport/async_http/response_body.rb alone. The native body +# is untyped: nothing from protocol-http may name a signature (NFR-11). +module Dexpace + module Transport + module AsyncHTTP + # What the mapper hands a body to run when the body releases its native resource: the + # exchange's own watcher release, as a lambda. + interface _Release + def call: () -> untyped + end + + class ResponseBody + include Dexpace::Body + include Dexpace::Closeable + + @native: untyped + @media_type: Dexpace::MediaType? + @content_length: Integer + @cancellation: Dexpace::Cancellation + @logger: Instrumentation::Logger + @on_release: _Release? + @source: Dexpace::IO::BufferedSource? + + attr_reader media_type: Dexpace::MediaType? + attr_reader content_length: Integer + + def initialize: (native: untyped, media_type: Dexpace::MediaType?, content_length: Integer, + cancellation: Dexpace::Cancellation, ?logger: Instrumentation::Logger, + ?on_release: _Release?) -> void + def each: () { (String) -> void } -> nil + | () -> Enumerator[String, nil] + def write_to: (Dexpace::IO::_Sink sink) -> Integer + def source: () -> Dexpace::IO::BufferedSource + + private + + def pull: () -> String? + def release: () -> untyped + end + end + end +end diff --git a/gems/dexpace-transport-async_http/sig/dexpace/transport/async_http/response_mapper.rbs b/gems/dexpace-transport-async_http/sig/dexpace/transport/async_http/response_mapper.rbs new file mode 100644 index 0000000..c1cab97 --- /dev/null +++ b/gems/dexpace-transport-async_http/sig/dexpace/transport/async_http/response_mapper.rbs @@ -0,0 +1,27 @@ +# Dexpace::Transport::AsyncHTTP::ResponseMapper is a private_constant and not public API: this +# declaration exists because the `async_http` Steep target checks every file under lib/ and needs +# the module and its one function declared to type Exchange's call site. RBS has no visibility for +# a constant, so the privacy lives in lib/dexpace/transport/async_http/response_mapper.rb alone. +# The native response and its body are untyped: nothing from protocol-http may name a signature +# (NFR-11). +module Dexpace + module Transport + module AsyncHTTP + module ResponseMapper + def self?.call: (untyped native, request: Dexpace::Request, logger: Instrumentation::Logger, + cancellation: Dexpace::Cancellation, ?head: bool, + ?on_release: _Release?) -> Dexpace::Response + + private + + def inbound_headers: (untyped native, Instrumentation::Logger logger) -> Dexpace::Headers + def body_for: (untyped native, bool head, cancellation: Dexpace::Cancellation, + on_release: _Release?, media_type: Dexpace::MediaType?, + logger: Instrumentation::Logger) -> ResponseBody? + def reason_for: (untyped native) -> String? + def media_type_for: (Dexpace::Headers headers) -> Dexpace::MediaType? + def log_drop: (Instrumentation::Logger logger, String name) -> nil + end + end + end +end diff --git a/gems/dexpace-transport-async_http/test/dexpace/transport/async_http/adapter_test.rb b/gems/dexpace-transport-async_http/test/dexpace/transport/async_http/adapter_test.rb new file mode 100644 index 0000000..4ae5174 --- /dev/null +++ b/gems/dexpace-transport-async_http/test/dexpace/transport/async_http/adapter_test.rb @@ -0,0 +1,307 @@ +# frozen_string_literal: true +# SPDX-License-Identifier: MIT + +require_relative "../../../test_helper" +require_relative "../../../support/async_http_recording_body" +require_relative "../../../support/async_http_recording_sink" +require_relative "../../../support/async_http_holding_server" +require_relative "../../../support/async_http_hermetic_configuration" +require_relative "../../../support/async_http_reactor" +require "dexpace/transport/async_http" + +# Dispatch steps 1 to 3 and 10, TRANSPORT-15/16/21/29 and ASYNC-22's structural half: the pivot +# is minted and returned before anything fallible runs, every pre-dispatch failure is delivered +# through it, ownership is a construction-time fact, and nothing per-call lives on the adapter. +# Three nested classes under Metrics/ClassLength (8a's shape): the constructions, the +# pre-dispatch settlements, and the deadline's tiers -- the last over a hermetic configuration, +# so the 60-second default it pins is the default and not the host's REQUEST_TIMEOUT. +module DexpaceTransportAsyncHTTPAdapterTest + # The request builder and the three doubles every class here shares: a client answering + # inline, a forged request that met no builder, and a native response. + module AdapterTestSupport + include AsyncHTTPReactor + include AsyncHTTPHermeticConfiguration + + AsyncHTTP = Dexpace::Transport::AsyncHTTP + Adapter = Dexpace::Transport::AsyncHTTP::Adapter + + def request(url: "https://example.test/", headers: {}, method: "GET", body: nil) + builder = Dexpace::Request.builder + builder.method = method + builder.url = url + headers.each { |name, value| builder.header(name, value) } + builder.body = body + builder.build + end + + # A client that answers inline, with no reactor turn, so a future comes back already settled. + # Built with its behaviour up front: redefining a singleton method warns under -w. + def fake_client(response = nil, retries: 0, on_pool_close: nil, &block) + pool = Object.new + pool.define_singleton_method(:close) { on_pool_close&.call } + client = Object.new + client.define_singleton_method(:retries) { retries } + client.define_singleton_method(:pool) { pool } + client.define_singleton_method(:call) { |native| block ? yield(native) : response } + client + end + + # A request-shaped object answering the four readers, carrying headers no Dexpace validation + # has seen -- design §10.10's admitted hole, the wire-boundary re-validation's own subject. + def forged_request(name, value) + template = request + headers = Object.new + headers.define_singleton_method(:each_entry) { |&block| block.call(name, value) } + forged = Object.new + forged.define_singleton_method(:method) { template.method } + forged.define_singleton_method(:url) { template.url } + forged.define_singleton_method(:headers) { headers } + forged.define_singleton_method(:body) { nil } + forged + end + + def native_response(body = AsyncHTTPRecordingBody.new(["ok".b], length: 2), status: 200) + ::Protocol::HTTP::Response.new("HTTP/1.1", status, ::Protocol::HTTP::Headers.new, body) + end + end + + # .build, .using, .default, what each refuses, and the per-call statelessness ASYNC-22 rests on. + class DexpaceTransportAsyncHTTPAdapterConstructionTest < DexpaceTestCase + include AdapterTestSupport + + test ".build builds and owns its own clients; .using borrows a caller's client and never " \ + "closes it (TRANSPORT-15, XCUT-22)" do + owning = AsyncHTTP.build + + assert_predicate(owning, :owned?) + assert(Dexpace::AsyncTransport.conforms?(owning)) + owning.close + + closed = false + client = fake_client(on_pool_close: -> { closed = true }) + borrowing = AsyncHTTP.using(client) + + refute_predicate(borrowing, :owned?) + borrowing.close + + refute(closed, "a borrowing adapter must never close the caller's client") + assert_predicate(borrowing, :closed?) + end + + test ".using refuses a client whose retries is not zero, and never sets it (TRANSPORT-2, " \ + "P8-10)" do + client = fake_client(retries: 3) + + error = assert_raises(Dexpace::InvalidArgumentError) { AsyncHTTP.using(client) } + + assert_match(/retries == 0/, error.message) + assert_equal(3, client.retries) + end + + test ".build validates timeout: at construction" do + assert_raises(Dexpace::InvalidArgumentError) { AsyncHTTP.build(timeout: 0) } + assert_raises(Dexpace::InvalidArgumentError) { AsyncHTTP.build(timeout: Float::INFINITY) } + assert_raises(Dexpace::InvalidArgumentError) { AsyncHTTP.build(timeout: "1") } + AsyncHTTP.build(timeout: 0.5).close + end + + test "Adapter.new is private behind the two validating factories" do + assert_raises(NoMethodError) { Adapter.new } + end + + test "#close is idempotent, does not block, and reports closed" do + adapter = AsyncHTTP.build + adapter.close + adapter.close + + assert_predicate(adapter, :closed?) + end + + # ASYNC-22 / TRANSPORT-29: the adapter's own state is its clients, its settings and Closeable's + # latch -- nothing per call lives on self. + test "ASYNC-22: nothing per-call lives on the adapter" do + adapter = AsyncHTTP.build + ivars = adapter.instance_variables.sort + + assert_equal( + %i[@client @clients @configuration @dexpace_close_mutex @dexpace_closed @dexpace_owned + @drop_policy @logger @timeout].sort, ivars, + ) + ensure + adapter&.close + end + end + + # Every failure before the exchange starts settles through the future the caller already holds. + class DexpaceTransportAsyncHTTPAdapterPreDispatchTest < DexpaceTestCase + include AdapterTestSupport + + # The positive outcome, never assert_nothing_raised: what TRANSPORT-21 asserts is that the + # future comes back already settled and carries the failure. + test "TRANSPORT-21: calling outside a reactor settles a SeamError through the future, never " \ + "a synchronous raise (P8-39)" do + adapter = AsyncHTTP.build + + future = adapter.call(request, nil, Dexpace::Cancellation.none) + + assert_predicate(future, :settled?) + error = assert_raises(Dexpace::SeamError) { future.value } + assert_match(/Async reactor/, error.message) + assert_match(/Sync \{ \}/, error.message) + ensure + adapter&.close + end + + test "TRANSPORT-21 / HTTP-17: a header HeaderSyntax rejects settles through the future " \ + "(the wire-boundary re-validation), and no client is touched" do + calls = 0 + adapter = AsyncHTTP.using(fake_client { calls += 1 }) + forged = forged_request("X-Evil\r\nInjected", "v") + + Sync do + future = adapter.call(forged, nil, Dexpace::Cancellation.none) + + assert_predicate(future, :settled?) + assert_raises(Dexpace::InvalidArgumentError) { future.value } + end + + assert_equal(0, calls) + end + + test "TRANSPORT-21: a URL the endpoint cannot dispatch settles InvalidArgumentError through " \ + "the future -- never a connection to port 21" do + adapter = AsyncHTTP.build + + Sync do + future = adapter.call(request(url: "ftp://example.test/"), nil, Dexpace::Cancellation.none) + + assert_predicate(future, :settled?) + error = assert_raises(Dexpace::InvalidArgumentError) { future.value } + assert_match(/http and https only/, error.message) + end + ensure + adapter&.close + end + + test "a send after close settles ClosedError through the future on an owning adapter, and " \ + "a borrowing adapter stays usable after its own close (SEAM-15, boundary 17)" do + owning = AsyncHTTP.build + owning.close + borrowing = AsyncHTTP.using(fake_client(native_response)) + borrowing.close + + Sync do + failed = owning.call(request, nil, Dexpace::Cancellation.none) + + assert_predicate(failed, :settled?) + assert_raises(Dexpace::ClosedError) { failed.value } + assert_equal(200, borrowing.call(request, nil, nil).value.status.code) + end + end + + test "an already-cancelled token settles a CANCELLATION before anything is mapped or sent" do + calls = 0 + sink = AsyncHTTPRecordingSink.new + adapter = AsyncHTTP.using(fake_client { calls += 1 }, + logger: Dexpace::Instrumentation::Logger.build(sink: sink),) + source = Dexpace::Cancellation.source + source.cancel(:already_gone) + + Sync do + future = adapter.call(request(headers: { "Expect" => "100-continue" }), nil, source.token) + + assert_predicate(future, :settled?) + assert_predicate(future, :cancelled?) + assert_equal(:already_gone, assert_raises(Dexpace::CancelledError) { future.value }.reason) + end + + assert_equal(0, calls) + assert_empty(sink.entries, "the request was never mapped: no managed-header drop was logged") + end + + test "a nil cancellation is the never-cancelled token (P8-57's shape)" do + adapter = AsyncHTTP.using(fake_client(native_response)) + + Sync { assert_equal(200, adapter.call(request, nil, nil).value.status.code) } + end + + test "a native failure raised inline settles a retryable TransportError with the cause" do + adapter = AsyncHTTP.using(fake_client { raise ::Errno::ECONNREFUSED }) + + Sync do + future = adapter.call(request, nil, nil) + error = assert_raises(Dexpace::TransportError) { future.value } + + assert_predicate(error, :retryable?) + assert_kind_of(::Errno::ECONNREFUSED, error.cause) + assert_equal(:connect, error.phase) + end + end + + test "TRANSPORT-23: a successful dispatch never settles with a nil response" do + adapter = AsyncHTTP.using(fake_client(native_response)) + + Sync do + response = adapter.call(request, nil, nil).value + + refute_nil(response) + assert_instance_of(Dexpace::Response, response) + assert_equal("ok", response.body_string) + end + end + end + + # The deadline's three tiers, and the pair TRANSPORT-8 shares with TRANSPORT-4. + class DexpaceTransportAsyncHTTPAdapterTimeoutTest < DexpaceTestCase + include AdapterTestSupport + + # The three tiers: the call, the transport, the configuration chain (a bare number is + # milliseconds, CFG-7), then the default -- read through the adapter's own private resolver + # because the deadline is applied inside the exchange task. Both chains are hermetic: the + # default tier is reached only when the environment tier answered nothing. + test "the deadline's three tiers resolve highest first, and the configured tier reads " \ + "REQUEST_TIMEOUT through #duration" do + key = Dexpace::Configuration::Keys::REQUEST_TIMEOUT + configuration = hermetic_configuration({ key => "250ms" }) + adapter = AsyncHTTP.build(timeout: 2.5, configuration: configuration) + per_call = Dexpace::RequestOptions.builder.tap { |b| b.timeout = 0.75 }.build + + assert_in_delta(0.75, adapter.send(:resolve_timeout, per_call)) + assert_in_delta(2.5, adapter.send(:resolve_timeout, nil)) + configured = AsyncHTTP.build(configuration: configuration) + + assert_in_delta(0.25, configured.send(:resolve_timeout, Dexpace::RequestOptions::EMPTY)) + assert_in_delta(60.0, AsyncHTTP.build(configuration: hermetic_configuration) + .send(:resolve_timeout, nil),) + assert_in_delta(60.0, AsyncHTTP::DEFAULT_TIMEOUT_SECONDS) + ensure + adapter&.close + configured&.close + end + + test "TRANSPORT-4/TRANSPORT-8's pair: a deadline that expires with the head withheld settles " \ + "a RETRYABLE TransportError and leaves the token clear" do + server = AsyncHTTPHoldingServer.new(hold: :head) + adapter = AsyncHTTP.build + source = Dexpace::Cancellation.source + reactor_over(server) do |task| + options = Dexpace::RequestOptions.builder.tap { |b| b.timeout = 0.1 }.build + future = adapter.call(request(url: "http://127.0.0.1:#{server.port}/"), options, + source.token,) + server.wait_for_accept + error = assert_raises(Dexpace::TransportError) do + future.value(deadline: Dexpace::Clock.deadline_in(5)) + end + + assert_predicate(error, :retryable?) + assert_kind_of(::Async::TimeoutError, error.cause) + refute_predicate(source.token, :cancelled?) + refute_predicate(future, :cancelled?) + assert_exchange_released(task) + end + ensure + adapter&.close + server&.close + end + end +end diff --git a/gems/dexpace-transport-async_http/test/dexpace/transport/async_http/cancellation_test.rb b/gems/dexpace-transport-async_http/test/dexpace/transport/async_http/cancellation_test.rb new file mode 100644 index 0000000..b35fe84 --- /dev/null +++ b/gems/dexpace-transport-async_http/test/dexpace/transport/async_http/cancellation_test.rb @@ -0,0 +1,285 @@ +# frozen_string_literal: true +# SPDX-License-Identifier: MIT + +require_relative "../../../test_helper" +require_relative "../../../support/async_http_holding_server" +require_relative "../../../support/async_http_recording_body" +require_relative "../../../support/async_http_reactor" +require "dexpace/transport/async_http" + +# R13: "writes a test asserting the close ran on a cancellation specifically -- not only on a +# timeout, which is the test a correct-looking wrong implementation passes." The cancellation +# cases and the timeout pair (adapter_test.rb) are kept in this gem so neither can be deleted +# without the other being missed. TRANSPORT-7: cancel an in-flight future and the native call is +# cancelled. TRANSPORT-9: a native response obtained after the pivot was cancelled is closed, +# never delivered. ASYNC-6: both directions -- the token and the future reach the exchange, and +# the exchange's own end settles the pivot. Deterministic, per the design's three techniques: +# the SERVER decides when the client is blocked (AsyncHTTPHoldingServer#wait_for_accept), a +# Cancellation::Source the test owns fires the cancel, and the assertions are on counts and +# outcomes, never on elapsed time. Every wait is bounded, because "hangs" is the failure mode. +# Two nested classes under Metrics/ClassLength: the cancel reaching a blocked exchange, and the +# cancel around and after a delivery. +module DexpaceTransportAsyncHTTPCancellationTests + # The request builder and the bounded wait both classes share. + module CancellationTestSupport + include AsyncHTTPReactor + + AsyncHTTP = Dexpace::Transport::AsyncHTTP + BOUND = 5.0 + + def request(url) + builder = Dexpace::Request.builder + builder.url = url + builder.build + end + + def value_within(future, cancellation: nil) + future.value(cancellation: cancellation, deadline: Dexpace::Clock.deadline_in(BOUND)) + end + end + + # A cancel -- the token's, the future's, one from a foreign thread -- reaching an exchange blocked + # on the head. + class DexpaceTransportAsyncHTTPCancellationTest < DexpaceTestCase + include CancellationTestSupport + + # TRANSPORT-7 / ASYNC-6 (token -> exchange): cancelling the token aborts a native call blocked + # waiting for the head and settles a terminal, non-retryable cancellation. + test "TRANSPORT-7/ASYNC-6: cancelling the token aborts a blocked native call and settles a " \ + "terminal, non-retryable CancelledError with the token's reason" do + server = AsyncHTTPHoldingServer.new(hold: :head) + adapter = AsyncHTTP.build + source = Dexpace::Cancellation.source + + reactor_over(server) do |task| + future = adapter.call(request("http://127.0.0.1:#{server.port}/"), nil, source.token) + server.wait_for_accept # provably blocked waiting for the head; no sleep needed + source.cancel(:token_cancelled) + + error = assert_raises(Dexpace::CancelledError) { value_within(future) } + assert_equal(:token_cancelled, error.reason) + assert_predicate(future, :cancelled?) + refute_respond_to(error, :retryable?) + assert_exchange_released(task) + end + ensure + adapter&.close + server&.close + end + + # ASYNC-6 (future -> exchange): cancelling the FUTURE itself, not the token, must also reach + # the in-flight exchange -- the direction only an async transport can satisfy for real. + test "ASYNC-6: cancelling the future reaches the native exchange and settles it cancelled" do + server = AsyncHTTPHoldingServer.new(hold: :head) + adapter = AsyncHTTP.build + + reactor_over(server) do |task| + future = adapter.call(request("http://127.0.0.1:#{server.port}/"), nil, nil) + server.wait_for_accept + future.cancel(:future_cancelled) + + error = assert_raises(Dexpace::CancelledError) { value_within(future) } + assert_equal(:future_cancelled, error.reason) + assert_predicate(future, :cancelled?) + assert_exchange_released(task) + end + ensure + adapter&.close + server&.close + end + + # The bridge as built: the token's hook runs on the CANCELLER's thread, where Fiber.scheduler + # is nil and Async::Task#cancel raises NoMethodError, so the cancel is marshalled through a + # queue to a watcher task on the reactor. The conformance suite's TRANSPORT-3 assertion cancels + # from an OS thread exactly like this. + test "a token cancelled from a foreign OS thread still reaches the exchange, promptly" do + server = AsyncHTTPHoldingServer.new(hold: :head) + adapter = AsyncHTTP.build + source = Dexpace::Cancellation.source + canceller = nil + + reactor_over(server) do |task| + future = adapter.call(request("http://127.0.0.1:#{server.port}/"), nil, source.token) + server.wait_for_accept + canceller = ::Thread.new { source.cancel(:from_another_thread) } + + error = assert_raises(Dexpace::CancelledError) { value_within(future) } + assert_equal(:from_another_thread, error.reason) + assert_exchange_released(task) + end + canceller.join + ensure + adapter&.close + server&.close + end + + # Review round 2's R2-1. `Cancellation::Source#cancel` steals its hooks under its mutex, flips + # the flag and runs them OUTSIDE it, on the canceller's thread -- so the exchange can end + # between the flip and the adapter's hook: check-after-resume sees the flag, the pivot + # settles cancelled and the exchange closes its queue before the hook pushes onto it, and a + # push onto a closed queue raises `ClosedQueueError`, which `Hooks.notify` would hand back to + # the caller's `Source#cancel`. Deterministic through an ordinary caller's hook registered + # FIRST, which parks the canceller across exactly that window; the same ordering needs no + # slow hook when the canceller is merely descheduled there. A cancel that lost the race + # against the exchange's own end is not the caller's failure. + test "a token cancel in flight while the exchange finishes never raises out of " \ + "Source#cancel on the canceller's thread, and the future is cancelled" do + source = Dexpace::Cancellation.source + flagged = ::Thread::Queue.new # the canceller has flipped the flag and is in its hooks + finished = ::Thread::Queue.new # the exchange has ended: let the adapter's hook run now + outcome = ::Thread::Queue.new + source.token.on_cancel do |_reason| + flagged.push(true) + finished.pop + end + client = Object.new + client.define_singleton_method(:retries) { 0 } + client.define_singleton_method(:pool) { Object.new.tap { |pool| def pool.close = nil } } + client.define_singleton_method(:call) do |_native| + flagged.pop # scheduler-aware: the reactor keeps turning until the cancel is in flight + ::Protocol::HTTP::Response.new("HTTP/1.1", 204, ::Protocol::HTTP::Headers.new, nil) + end + adapter = AsyncHTTP.using(client) + canceller = nil + + Sync do |task| + future = adapter.call(request("http://example.test/"), nil, source.token) + canceller = ::Thread.new do + outcome.push(begin + source.cancel(:racing) + rescue ::StandardError => error + error + end) + end + + error = assert_raises(Dexpace::CancelledError) { value_within(future) } + + assert_equal(:racing, error.reason) + assert_predicate(future, :cancelled?) + assert_exchange_released(task) # the exchange ended and closed its queue... + ensure + finished.push(true) # ...and only now does the adapter's hook run (on every path) + end + canceller.join + result = outcome.pop + + assert_same(true, result, "Source#cancel raised #{result.inspect} out of the adapter's hook") + end + end + + # A cancel racing the delivery (TRANSPORT-9), landing after it (ASYNC-20), or reaching a blocked + # body read (TRANSPORT-7's body path). + class DexpaceTransportAsyncHTTPCancellationDeliveryTest < DexpaceTestCase + include CancellationTestSupport + + # TRANSPORT-9: the token is cancelled WHILE the native call is in flight and the native call + # returns a response anyway -- the exact race the requirement names. Check-after-resume sees + # the token before the response is delivered, and the response is closed exactly once. + test "TRANSPORT-9: a native response obtained after the token was cancelled mid-flight is " \ + "closed exactly once, never delivered" do + native = AsyncHTTPRecordingBody.new(["late".b], length: 4) + source = Dexpace::Cancellation.source + client = Object.new + client.define_singleton_method(:retries) { 0 } + client.define_singleton_method(:pool) { Object.new.tap { |pool| def pool.close = nil } } + client.define_singleton_method(:call) do |_native| + source.cancel(:mid_flight) + ::Protocol::HTTP::Response.new("HTTP/1.1", 200, ::Protocol::HTTP::Headers.new, native) + end + adapter = AsyncHTTP.using(client) + + Sync do + future = adapter.call(request("http://example.test/"), nil, source.token) + + assert_predicate(future, :cancelled?) + assert_raises(Dexpace::CancelledError) { value_within(future) } + end + + assert_equal(1, native.close_count) + end + + # The negative twin (ASYNC-20): a response already delivered to the caller must not be closed + # by a late cancellation of the future -- a cancel on a settled pivot is a no-op. The watcher + # that would close it on a TOKEN cancel stays for the life of the delivered response, and + # transient: a body a caller never closes must not hold the caller's `Sync` block open -- + # without `transient: true` that property fails as a hang and never by name (review round + # 2's R2-3), so it is asserted here before the body is released and its release is asserted + # to end the watcher. + test "ASYNC-20: cancelling the future after delivery does not close the delivered response; " \ + "its watcher stays, transient, until the body is released" do + server = AsyncHTTPHoldingServer.new(hold: :head) + adapter = AsyncHTTP.build + + reactor_over(server) do |task| + future = adapter.call(request("http://127.0.0.1:#{server.port}/"), nil, nil) + server.wait_for_accept + server.release("ok") + response = value_within(future) + + future.cancel(:too_late) # no-op: the pivot is already settled successfully + + refute_predicate(response.body, :closed?) + refute_predicate(future, :cancelled?) + # Read while the body is open and asserted only after its release: a non-transient + # watcher under a still-open body would hold the reactor open on the failing assertion. + transient = watcher_tasks(task).map(&:transient?) + + assert_equal("ok", response.body_string) + assert_equal([true], transient, "one watcher per delivered response, and transient") + assert_exchange_released(task) # the body's release is what ends the watcher + end + ensure + adapter&.close + server&.close + end + + # A cancel that lands after delivery but through the TOKEN reaches a consumer blocked in a + # body read: the watcher closes the response from inside the reactor, the blocked read wakes, + # and the token is asked first, so the reader sees the cancellation and not a stream failure. + test "TRANSPORT-7 on the body path: a token cancelled under a blocked body read wakes the " \ + "reader with CancelledError and releases the body" do + server = AsyncHTTPHoldingServer.new(hold: :body) + adapter = AsyncHTTP.build + source = Dexpace::Cancellation.source + arrived = ::Thread::Queue.new + canceller = ::Thread.new do + arrived.pop # the first chunk was yielded: the reader is now blocked on the held second one + source.cancel(:reader_cancelled) + end + + reactor_over(server) do |task| + response = value_within(adapter.call(request("http://127.0.0.1:#{server.port}/"), nil, + source.token,)) + server.wait_for_accept + chunks = [] + # Bounded: a watcher that never closed the delivered response would leave the read blocked + # for good. The bound's Async::TimeoutError is a StandardError the token-first classifier + # turns into the SAME CancelledError, so the cause is what tells the two wakes apart. + error = assert_raises(Dexpace::CancelledError) do + task.with_timeout(BOUND) do + response.body.each do |chunk| + chunks << chunk + arrived.push(true) + end + end + end + + assert_equal(:reader_cancelled, error.reason) + # The wake must be the WATCHER's close and never this test's own bound: with the watcher's + # close of the delivered response deleted (the reviewer's mutation 37), the bound fires + # inside the native read five seconds later and the classifier still answers + # CancelledError(:reader_cancelled) with the body closed -- every assertion around this + # one passes. A close under a blocked read surfaces from the library as a bare IOError, + # which the classifier saw and Ruby attached; the bound would have attached its own. + refute_kind_of(::Async::TimeoutError, error.cause, + "the reader was woken by the test's bound, not by the watcher's close",) + assert_equal(["first"], chunks) + assert_predicate(response.body, :closed?) + end + canceller.join + ensure + adapter&.close + server&.close + end + end +end diff --git a/gems/dexpace-transport-async_http/test/dexpace/transport/async_http/clients_test.rb b/gems/dexpace-transport-async_http/test/dexpace/transport/async_http/clients_test.rb new file mode 100644 index 0000000..5158788 --- /dev/null +++ b/gems/dexpace-transport-async_http/test/dexpace/transport/async_http/clients_test.rb @@ -0,0 +1,253 @@ +# frozen_string_literal: true +# SPDX-License-Identifier: MIT + +require_relative "../../../test_helper" +require_relative "../../../support/async_http_holding_server" +require_relative "../../../support/async_http_hermetic_configuration" +require_relative "../../../support/async_http_reactor" +require "dexpace/transport/async_http" + +# The (reactor, origin) client map. concurrency-and-async/c0fab747 (smallest critical section) +# and /ee54cb68 (never hold a lock across I/O): #fetch's mutex guards a Hash read and a Hash +# insert and nothing else -- the client is built outside the lock, because building an https one +# loads the default certificate store from disk. Async::HTTP::Client.new opens no socket, so a +# client discarded by a lost insert race costs nothing. XCUT-14: the map is bounded at +# MAX_ORIGINS and drains back under it in a loop after each insert, retiring what it evicts. +# P8-37, as built: #close retires every pooled connection and never waits. Every configuration a +# case builds is hermetic -- the environment tier answers nothing -- so the default the limit case +# pins is the default and not the host's TRANSPORT_CONNECTION_LIMIT. Two nested classes under +# Metrics/ClassLength: what #fetch builds, and what #close and the cap release. +module DexpaceTransportAsyncHTTPClientsTest + # The stand-in reactor and the two builders both classes share. + module ClientsTestSupport + include AsyncHTTPReactor + include AsyncHTTPHermeticConfiguration + + Clients = Dexpace::Transport::AsyncHTTP.const_get(:Clients, false) + AsyncHTTP = Dexpace::Transport::AsyncHTTP + + # A stand-in reactor: the key compares it by identity, never by class. + REACTOR = Object.new + + def url(string) = Dexpace::URL.parse!(string) + + def clients(**) + Clients.build(configuration: hermetic_configuration, **) + end + end + + # One client per (reactor, origin), built with the native retry loop off, the configured limit + # and the caller's TLS context. + class FetchTest < DexpaceTestCase + include ClientsTestSupport + + test "fetch memoises one client per origin under one reactor" do + map = clients + + first = map.fetch(url("https://example.test/a"), reactor: REACTOR) + second = map.fetch(url("https://EXAMPLE.test:443/b"), reactor: REACTOR) + third = map.fetch(url("https://example.test:8443/a"), reactor: REACTOR) + + assert_same(first, second) + refute_same(first, third) + assert_equal(2, map.size) + ensure + map&.close + end + + # ASYNC-22's multi-thread clause, structurally: one async-http client cannot be shared across + # reactors on different OS threads (its pool waits on a Thread::Mutex the scheduler cannot + # interrupt, measured), so the key is the reactor too. + test "the same origin under a different reactor is a different client" do + map = clients + + first = map.fetch(url("http://example.test/"), reactor: REACTOR) + second = map.fetch(url("http://example.test/"), reactor: Object.new) + + refute_same(first, second) + assert_equal(2, map.size) + ensure + map&.close + end + + test "every client disables the native retry loop (TRANSPORT-2, TRANSPORT-17, TRANSPORT-18)" do + map = clients + client = map.fetch(url("https://example.test/"), reactor: REACTOR) + + assert_equal(0, client.retries) + assert_operator(::Async::HTTP::DEFAULT_RETRIES, :>, 0, + "the default is what makes this load-bearing",) + ensure + map&.close + end + + test "the connection limit reads TRANSPORT_CONNECTION_LIMIT off the configuration, default 8" do + map = clients + client = map.fetch(url("https://example.test/"), reactor: REACTOR) + + assert_equal(8, map.limit) + assert_equal(AsyncHTTP::DEFAULT_CONNECTION_LIMIT, client.pool.instance_variable_get(:@limit)) + ensure + map&.close + end + + test "a configured connection limit overrides the default, and an explicit keyword " \ + "overrides both" do + key = Dexpace::Configuration::Keys::TRANSPORT_CONNECTION_LIMIT + configuration = hermetic_configuration({ key => "3" }) + configured = Clients.build(configuration: configuration) + explicit = Clients.build(configuration: configuration, connection_limit: 2) + + assert_equal(3, configured.fetch(url("https://example.test/"), reactor: REACTOR).pool + .instance_variable_get(:@limit),) + assert_equal(2, explicit.fetch(url("https://example.test/"), reactor: REACTOR).pool + .instance_variable_get(:@limit),) + ensure + configured&.close + explicit&.close + end + + test "a limit that is not a positive Integer is refused at construction" do + assert_raises(Dexpace::InvalidArgumentError) { clients(connection_limit: 0) } + assert_raises(Dexpace::InvalidArgumentError) { clients(connection_limit: "8") } + end + + test "a caller's ssl_context reaches every https client verbatim" do + context = ::OpenSSL::SSL::SSLContext.new + map = clients(ssl_context: context) + client = map.fetch(url("https://example.test/"), reactor: REACTOR) + + assert_same(context, client.endpoint.ssl_context) + ensure + map&.close + end + end + + # P8-37's close, XCUT-14's cap, the lock's extent, and a close under an open response. + class ReleaseTest < DexpaceTestCase + include ClientsTestSupport + + # P8-37 as built: Client#close waits on the pool and writes a Console warning; pool.close + # alone still drains while any resource is busy. #close retires every resource FIRST. + test "close retires every pooled resource and closes the pool, never through Client#close" do + map = clients + client = map.fetch(url("https://example.test/"), reactor: REACTOR) + calls = [] + client.define_singleton_method(:close) { calls << :client_close } + client.pool.define_singleton_method(:close) { calls << :pool_close } + client.pool.define_singleton_method(:retire) { |resource| calls << [:retire, resource] } + client.pool.define_singleton_method(:resources) { { a: 1, b: 1 } } + + map.close + + assert_equal([%i[retire a], %i[retire b], :pool_close], calls) + assert_equal(0, map.size) + end + + test "close is idempotent, and a fetch afterwards builds afresh" do + map = clients + first = map.fetch(url("https://example.test/"), reactor: REACTOR) + + map.close + map.close + + refute_same(first, map.fetch(url("https://example.test/"), reactor: REACTOR)) + ensure + map&.close + end + + # XCUT-14 (MUST): "bounded by a hard cap and MUST drain back under the cap after each insert + # using a loop (not a single pre-insert check-then-evict)". Both influences are present: a + # caller's URLs choose origins and a server's redirect Location chooses new ones. + test "XCUT-14: the map is bounded at MAX_ORIGINS and drains back to the cap after each " \ + "insert" do + map = clients + + (AsyncHTTP::MAX_ORIGINS + 5).times { |i| map.fetch(url("https://h#{i}.test/"), reactor: REACTOR) } + + assert_equal(32, AsyncHTTP::MAX_ORIGINS) + assert_equal(AsyncHTTP::MAX_ORIGINS, map.size) + ensure + map&.close + end + + # The values own pools, so eviction that does not retire is a connection leak wearing a cap. + test "XCUT-14: an evicted client's pool is retired and closed, never merely dropped" do + map = clients + first = map.fetch(url("https://h0.test/"), reactor: REACTOR) + closed = false + first.pool.define_singleton_method(:close) { closed = true } + + (AsyncHTTP::MAX_ORIGINS + 1).times { |i| map.fetch(url("https://evict#{i}.test/"), reactor: REACTOR) } + + assert(closed, "the evicted client's pool must be closed") + ensure + map&.close + end + + # A reactor that has exited leaves its clients useless: they are the first victims when the + # cap is reached, before the oldest live one. + test "XCUT-14: a client whose reactor has closed is evicted before a live one" do + map = clients + dead = Object.new + def dead.closed? = true + stale = map.fetch(url("https://stale.test/"), reactor: dead) + live = map.fetch(url("https://live.test/"), reactor: REACTOR) + + (AsyncHTTP::MAX_ORIGINS - 1).times { |i| map.fetch(url("https://fill#{i}.test/"), reactor: REACTOR) } + + refute_same(stale, map.fetch(url("https://stale.test/"), reactor: dead), "the stale one went") + assert_same(live, map.fetch(url("https://live.test/"), reactor: REACTOR), + "the live one stayed",) + ensure + map&.close + end + + # concurrency-and-async/ee54cb68: the source is the assertion, because no timing test can tell + # a client built under the lock from one built beside it. + test "the client is built outside the mutex: no Endpoints call inside a synchronize block" do + source = File.read(File.expand_path("../../../../lib/dexpace/transport/async_http/clients.rb", + __dir__,)) + blocks = source.scan(/synchronize do(.*?)^\s*end$/m).flatten + + source.scan(/synchronize \{(.*?)\}/).flatten + + refute_empty(blocks) + blocks.each { |block| refute_match(/Endpoints|Client\.new|build_client/, block) } + end + + # Measured against a real connection: a client with one response open is released at once. + test "close returns at once with a response still open, and the open read then fails" do + server = AsyncHTTPHoldingServer.new(hold: :body) + map = clients + elapsed = nil + read_error = nil + reactor_over(server) do |task| + client = map.fetch(url("http://127.0.0.1:#{server.port}/"), reactor: ::Fiber.scheduler) + response = client.get("/") + server.wait_for_accept + response.body.read + started = Process.clock_gettime(Process::CLOCK_MONOTONIC) + begin + task.with_timeout(2) { map.close } + rescue ::Async::TimeoutError + flunk("close waited on the open response instead of retiring it (P8-37)") + end + elapsed = Process.clock_gettime(Process::CLOCK_MONOTONIC) - started + begin + task.with_timeout(2) { response.body.read } + rescue StandardError => error + read_error = error + end + ensure + # A close that waited would have left the connection busy, and the reactor cannot exit + # while it is: releasing the response here is what lets that flunk surface. + ::Dexpace.close_quietly(response) if response + end + + assert_operator(elapsed, :<, 1.0, "close waited on the open response") + refute_nil(read_error) + ensure + server&.close + end + end +end diff --git a/gems/dexpace-transport-async_http/test/dexpace/transport/async_http/conformance_test.rb b/gems/dexpace-transport-async_http/test/dexpace/transport/async_http/conformance_test.rb new file mode 100644 index 0000000..9bcb2d0 --- /dev/null +++ b/gems/dexpace-transport-async_http/test/dexpace/transport/async_http/conformance_test.rb @@ -0,0 +1,195 @@ +# frozen_string_literal: true +# SPDX-License-Identifier: MIT + +require_relative "../../../test_helper" +require "dexpace/transport/async_http" +require "dexpace/conformance" + +# The second driver over the shared conformance suite (8a's R16, the suite contract): every +# assertion 8a wrote and the six 8c added, run unchanged against the real asynchronous adapter +# through MinitestDriver, with the three mechanisms the suite contract added for exactly this +# driver -- `settle:` awaits the future, `around:` opens the reactor an assertion's body runs +# inside and bounds it (see `.bounded`), and `borrow:` builds the caller's own client -- and two +# NAMED WAIVERS, both this adapter's alone (§9.3's mechanism, the id listed so the gap stays +# visible): +# +# - TRANSPORT-14's malformed-inbound-NAME clause: protocol-http1 raises BadHeader out of the read +# before a response exists to adapt (P8-38). The suite carries TWO assertions under that id, +# the per-header leniency and the multi-valued Set-Cookie, both over the same script whose +# non-ASCII name refuses the whole head here, so the one waiver skips both (a waiver is by id, +# design §9.3); the obs-text, control-byte-in-a-value and multi-valued halves are asserted in +# response_mapper_test.rb. +# - TRANSPORT-27's invalid-Content-Length clause: Protocol::HTTP1::BadRequest out of the read, no +# response object to downgrade; the malformed-Content-Type half is asserted in +# response_mapper_test.rb. 8a's driver waives nothing: Net::HTTP delivers both heads. +# +# TRANSPORT-18 reports vacuous by measurement (a skip): with `retries: 0` no native +# re-subscription exists to measure, exactly as on 8a's adapter. So this driver's run is four +# skips -- three waived over two ids, one vacuous -- and every other assertion passes, +# TRANSPORT-12 and TRANSPORT-13 included, which are vacuous by measurement on 8a's adapter and +# real here. +# +# The reactor shape, decided 2026-09-21: an assertion that settles from another OS thread +# (TRANSPORT-5's pair, TRANSPORT-29's eight) opens a reactor of its own per settle, because the +# adapter creates none (P8-39) and one async-http client cannot be shared across reactors on +# different threads -- the client map is keyed by (reactor, origin), so each such reactor gets +# its own client and the map's cap retires the ones whose reactor has exited. `kase.teardown` +# runs outside `around:`, so `Adapter#close` is reactor-free by construction. +# +# And a response never outlives the reactor that produced it: `Sync` returns only when the +# reactor has no work left, stopping the pool's transient gardener first, whose `ensure` closes +# the pool by DRAINING it -- a wait on every busy connection, and the connection behind an +# unread body is busy until that body is read or closed. Hand a streaming response out of a +# `Sync` and the `Sync` never returns (found by TRANSPORT-29's hang, 2026-09-21; the README +# states the same rule for a consumer). So the per-settle reactor materialises the body inside +# itself through Response#body_bytes -- the connection is released there, the reactor exits -- +# and hands the assertion a replayable BufferBody with the same media type, which reads exactly +# as the streamed one would have. `body_bytes` is loud past the materialisation ceiling, never a +# markerless truncation, which is why it is that and not Body.buffer_bounded. +class DexpaceTransportAsyncHTTPConformanceTest < DexpaceTestCase + extend Dexpace::Conformance::MinitestDriver + + AsyncHTTP = Dexpace::Transport::AsyncHTTP + WAIVED = %w[TRANSPORT-14 TRANSPORT-27].freeze + # Seconds an assertion's body may take before the driver fails it: generous against a loaded + # CI row, an order of magnitude above the slowest legitimate assertion here (TRANSPORT-29's + # eight settles, each in a reactor of its own, well under a second), and far below what a run + # that hangs costs. + AROUND_BOUND = 30.0 + + # Raised into the driver's OWN waiting fiber when the bound expires -- never into the + # assertion's, which is the point of `.bounded`. + class BoundExpired < ::StandardError; end + + # Runs the block inside the calling fiber's reactor, or a fresh one when the thread has none. + def self.in_reactor(&) + ::Async::Task.current? ? yield : Sync(&) + end + + # Clause 9's wrapper: the reactor the adapter needs (P8-39), with the assertion's body run as + # a CHILD task and the bound kept on the parent's wait. The bound cannot be raised into the + # assertion's own fiber: it would land inside a native read, where the adapter's token-first + # classifier turns any StandardError under a cancelled token into the very CancelledError the + # cancellation rows expect -- an adapter that never released a delivered body would PASS the + # mid-body row thirty seconds late instead of failing it (measured with the watcher's close + # deleted, review round 1's mutation 37: a `with_timeout` around `block.call` passed in 30 s). + # Cancelling the child instead raises the runtime's own Async::Cancel, which no classifier + # converts, Response#body_string's ensure releases the connection so the reactor can drain, + # and the assertion fails by name through the driver's own Failure path. `finished: false` + # is what `Sync` passes for its own root task: a child that fails before its first + # suspension would otherwise be logged by Console as an unhandled failure, and the parent's + # wait is what handles it. + def self.bounded(&) + Sync do |task| + child = task.async(finished: false, &) + begin + task.with_timeout(AROUND_BOUND, BoundExpired) { child.wait } + rescue BoundExpired + child.cancel + raise Dexpace::Conformance::Failure.new( + "the assertion's body did not finish within #{AROUND_BOUND} s: an adapter that never " \ + "releases what it holds would hang the run, and the driver's bound reports it instead", + expected: "completion within #{AROUND_BOUND} s", actual: "still blocked", + requirement_ids: [], + ) + end + end + end + + # The settle for a thread with no reactor of its own: a fresh reactor, closed before this + # returns, with the body read inside it (see the header) -- a bodyless response goes out as it + # is. + def self.settle_in_own_reactor(transport, request, options, cancellation) + Sync do + response = transport.call(request, options, cancellation).value(cancellation: cancellation) + body = response.body + next response if body.nil? + + buffer = Dexpace::IO::Buffer.new + buffer.write(response.body_bytes) + response.with(body: Dexpace::Body.buffer(buffer, media_type: body.media_type)) + end + end + + conformance( + Dexpace::Conformance::TransportSuite, + # Clause 4: a FACTORY, never an instance and never a constant; the two settings the contract + # allows, `timeout:` and `logger:`, are `.build`'s own keywords. + build: ->(**settings) { AsyncHTTP.build(**settings) }, + # Clause 4a: the factory takes the fixture's PORT and returns a BorrowedPair, so no assertion + # inside dexpace-conformance ever names Async::HTTP::Client. The client is built where the + # assertion runs -- inside `around:`'s reactor, the one it is then bound to -- with + # `retries: 0` set by the CALLER, never by `.using`, which refuses a client that lacks it. + # The probe is a real round trip through the client itself. + borrow: lambda do |port| + uri = ::URI::RFC3986_PARSER.parse("http://127.0.0.1:#{port}") + client = ::Async::HTTP::Client.new(::Async::HTTP::Endpoint.new(uri), retries: 0) + Dexpace::Conformance::BorrowedPair.build( + transport: AsyncHTTP.using(client), + probe: lambda do + in_reactor { client.get("/").read == "ok" } + rescue ::StandardError + false + end, + ) + end, + # Clause 8: `send` is ONE primitive and the async driver is what awaits the future; a + # cancellation surfaces from here as Dexpace::CancelledError because Completer#request_cancel + # settles the cancellation Future#value re-raises (clause 5). Inside `around:`'s reactor the + # response streams; from a thread of the assertion's own it is materialised (header). + settle: lambda do |transport, request, options, cancellation| + if ::Async::Task.current? + transport.call(request, options, cancellation).value(cancellation: cancellation) + else + settle_in_own_reactor(transport, request, options, cancellation) + end + end, + # Clause 9: the runner INVOKES each assertion, so this driver wraps it in the reactor the + # adapter needs (P8-39), bounded, and a streamed body is read inside. + around: method(:bounded), + waive: WAIVED, + ) + + # The driver's own contract, checked rather than assumed: every assertion became a test method, + # and the skips this run reports are exactly the ones the header accounts for. + test "one generated test per assertion, and the two waivers are the named ones" do + generated = public_methods(false).grep(/\Atest_/).reject do |name| + name.to_s.start_with?("test_: ") + end + ids = Dexpace::Conformance::TransportSuite.assertions.flat_map(&:ids).uniq + + assert_equal(Dexpace::Conformance::TransportSuite.assertions.size, generated.size) + assert_equal(34, generated.size) + assert_equal(%w[TRANSPORT-14 TRANSPORT-27], WAIVED) + assert_empty(WAIVED - ids, "a waiver must name an id the suite carries") + assert_includes(ids, "TRANSPORT-12") + assert_includes(ids, "TRANSPORT-13") + refute_includes(ids, "TRANSPORT-8") + end + + # The waiver's antecedent, measured rather than cited: the head 8a's WireServer script writes + # for TRANSPORT-14 makes protocol-http1 refuse the whole response, and the one for TRANSPORT-27 + # the same, so this adapter never sees a response object for either. + test "the two waived clauses are unreachable here: protocol-http1 refuses both heads out of " \ + "the read, as retryable transport failures" do + %i[malformed_headers malformed_content_length].each do |script| + server = Dexpace::Conformance::WireServer.start(Dexpace::Conformance::Scripts.public_send(script)) + adapter = AsyncHTTP.build + request = Dexpace::Request.build(method: "GET", url: "http://127.0.0.1:#{server.port}/", + headers: Dexpace::Headers::EMPTY, body: nil,) + begin + Sync do + error = assert_raises(Dexpace::TransportError) do + adapter.call(request, nil, nil).value(deadline: Dexpace::Clock.deadline_in(5)) + end + + assert_predicate(error, :retryable?, script.to_s) + assert_kind_of(::Protocol::HTTP1::Error, error.cause, script.to_s) + end + ensure + adapter.close + server.close + end + end + end +end diff --git a/gems/dexpace-transport-async_http/test/dexpace/transport/async_http/dispatch_conformance_test.rb b/gems/dexpace-transport-async_http/test/dexpace/transport/async_http/dispatch_conformance_test.rb new file mode 100644 index 0000000..44bd893 --- /dev/null +++ b/gems/dexpace-transport-async_http/test/dexpace/transport/async_http/dispatch_conformance_test.rb @@ -0,0 +1,231 @@ +# frozen_string_literal: true +# SPDX-License-Identifier: MIT + +require "stringio" +require_relative "../../../test_helper" +require_relative "../../../support/async_http_server_fixture" +require_relative "../../../support/async_http_holding_server" +require_relative "../../../support/async_http_recording_sink" +require_relative "../../../support/async_http_hermetic_configuration" +require_relative "../../../support/async_http_reactor" +require "dexpace/transport/async_http" + +# TRANSPORT-23 (never a null success), ASYNC-22 (many concurrent calls through one adapter, no +# cross-talk -- over HTTP/1.1's pool AND over one multiplexed HTTP/2 connection, the property the +# gem exists to prove), ASYNC-7's reactor-backed half of §3.3's contrast (a cancellation aborts a +# blocked read at the next scheduler checkpoint), the composed AsyncPipeline over the real +# adapter, and the stderr the adapter's whole lifecycle leaves clean: Console's default output +# would write JSON warnings there, and nothing in this SDK may. +# +# Three ways to HTTP/2 through the adapter, each what a caller would do: an OWNING adapter reaches +# it over TLS by ALPN (`AsyncHTTP.build(ssl_context:)` trusting the fixture's certificate -- +# plaintext prior-knowledge h2 is not something an adapter can know from a URL, and the plaintext +# client defaults to HTTP/1.1), and a BORROWING adapter over a caller's own prior-knowledge client. +# Two nested classes under Metrics/ClassLength: the deliveries, and the composition around them. +module DexpaceTransportAsyncHTTPDispatchConformanceTest + # The request builder, the bounded wait, the adapter per fixture flavour and the pool probe both + # classes share. + module DispatchTestSupport + include AsyncHTTPReactor + include AsyncHTTPHermeticConfiguration + + AsyncHTTP = Dexpace::Transport::AsyncHTTP + BOUND = 5.0 + + def request(url, headers: {}) + builder = Dexpace::Request.builder + builder.url = url + headers.each { |name, value| builder.header(name, value) } + builder.build + end + + def value_within(future) + future.value(deadline: Dexpace::Clock.deadline_in(BOUND)) + end + + # The adapter a caller builds for each fixture flavour. An owning one reads its connection + # limit off a hermetic chain, so the pool bound the ASYNC-22 case pins is the default and + # not the host's TRANSPORT_CONNECTION_LIMIT. + def adapter_for(server, variant) + case variant + when :http1 then AsyncHTTP.build(configuration: hermetic_configuration) + when :tls + AsyncHTTP.build(ssl_context: server.client_endpoint.ssl_context, + configuration: hermetic_configuration,) + else AsyncHTTP.using(::Async::HTTP::Client.new(server.client_endpoint, retries: 0)) + end + end + + def pool_size(adapter, server) + clients = adapter.instance_variable_get(:@clients) + by_key = clients.instance_variable_get(:@by_key) + key = by_key.keys.find { |k| k.origin[2] == server.client_endpoint.url.port } + by_key.fetch(key).pool.size + end + end + + # TRANSPORT-23 through a real reactor, and ASYNC-22 over three protocol shapes and two threads. + class DeliveryTest < DexpaceTestCase + include DispatchTestSupport + + test "TRANSPORT-23: a successful dispatch through a real reactor never settles with a nil " \ + "response -- over HTTP/1.1, TLS-negotiated HTTP/2 and prior-knowledge HTTP/2 alike" do + Sync do + { http1: "http/1.1", tls: "http/2", plaintext: "http/2" }.each do |variant, wire| + server = AsyncHTTPServerFixture.public_send(variant) { |_req| [200, [], ["ok"]] } + adapter = adapter_for(server, variant) + response = value_within(adapter.call(request(server.url), nil, nil)) + + refute_nil(response) + assert_instance_of(Dexpace::Response, response) + assert_equal("ok", response.body_string) + assert_equal(wire, response.protocol.wire, variant.to_s) + adapter.close + server.close + end + end + end + + # ASYNC-22: sixteen, because the requirement's own word is "many" and the design's probe ran + # eight. Over HTTP/1.1 the pool serves them within its bound; over HTTP/2 ONE connection + # multiplexes all sixteen streams, which is the property a thread pool cannot have. + { http1: AsyncHTTP::DEFAULT_CONNECTION_LIMIT, tls: 1 }.each do |variant, connections| + test "ASYNC-22 over #{variant}: sixteen concurrent calls through one owning adapter each " \ + "resolve to their own response with no cross-talk, over at most #{connections} " \ + "connection(s)" do + Sync do + server = AsyncHTTPServerFixture.public_send(variant) do |req| + [200, [], [req.headers["x-nonce"].to_s]] + end + adapter = adapter_for(server, variant) + futures = Array.new(16) do |i| + adapter.call(request(server.url, headers: { "X-Nonce" => i.to_s }), nil, nil) + end + + bodies = futures.map { |future| value_within(future).body_string } + + assert_equal((0...16).map(&:to_s), bodies) + assert_equal(16, server.received.size) + assert_operator(pool_size(adapter, server), :<=, connections) + assert_operator(pool_size(adapter, server), :>=, 1) + ensure + adapter&.close + server&.close + end + end + end + + # ASYNC-22's multi-thread clause, structurally: two OS threads, each its own reactor, each + # driving one shared adapter -- the shape that never completed through one shared client. + test "ASYNC-22 across threads: two reactors on two threads share one adapter and each gets " \ + "its own client" do + server = nil + adapter = AsyncHTTP.build + Sync do + server = AsyncHTTPServerFixture.http1 { |req| [200, [], [req.headers["x-nonce"].to_s]] } + url = server.url + results = Array.new(2) do |thread_index| + ::Thread.new do + Sync do + Array.new(5) do |i| + nonce = "#{thread_index}-#{i}" + [nonce, value_within(adapter.call(request(url, headers: { "X-Nonce" => nonce }), + nil, nil,)).body_string,] + end + end + end + end.map(&:value) + + results.flatten(1).each { |nonce, body| assert_equal(nonce, body) } + assert_equal(2, adapter.instance_variable_get(:@clients).size) + ensure + server&.close + end + ensure + adapter&.close + end + end + + # ASYNC-7's scheduler-checkpoint cancellation, the standard async pipeline over the adapter, and + # the clean stderr. + class CompositionTest < DexpaceTestCase + include DispatchTestSupport + + # ASYNC-7, this gem's own half of the cross-adapter contrast §3.3 fixes: "the reactor-backed + # ones abort at the next scheduler checkpoint." Measured as a bound, not compared against a + # README string: the server never releases, so a read left to finish would wait for the + # fixture's own teardown; the cancellation reaches it well before. + test "ASYNC-7: a cancellation aborts a blocked read at the next scheduler checkpoint, well " \ + "under the time the response would have taken to arrive" do + server = AsyncHTTPHoldingServer.new(hold: :head) + adapter = AsyncHTTP.build + source = Dexpace::Cancellation.source + + reactor_over(server) do + started = Process.clock_gettime(Process::CLOCK_MONOTONIC) + future = adapter.call(request("http://127.0.0.1:#{server.port}/"), nil, source.token) + server.wait_for_accept + source.cancel(:abort_now) + assert_raises(Dexpace::CancelledError) { value_within(future) } + elapsed = Process.clock_gettime(Process::CLOCK_MONOTONIC) - started + + assert_operator(elapsed, :<, 1.0) + end + ensure + adapter&.close + server&.close + end + + # The composed shape a generated client runs: phase 4c's AsyncPipeline.standard over the real + # adapter, driven from inside the caller's reactor. + test "AsyncPipeline.standard over the real adapter delivers a response through the future" do + Sync do + server = AsyncHTTPServerFixture.http1 { |_req| [200, [%w[x-served 1]], ["piped"]] } + adapter = AsyncHTTP.build + pipeline = Dexpace::AsyncPipeline.standard(adapter, redirect: :unsupported) + + response = value_within(pipeline.call(request(server.url), Dexpace::RequestOptions::EMPTY, + Dexpace::Cancellation.none,)) + + assert_equal("piped", response.body_string) + assert_equal(["1"], response.headers["x-served"]) + ensure + adapter&.close + server&.close + end + end + + # A whole lifecycle -- build, a request over HTTP/1.1 and over TLS HTTP/2, an unread close over + # HTTP/2, a cancellation, close -- writes nothing to stderr: Console's JSON output and the + # runtime's warnings alike. The runner's stderr scan reads only `warning:`; this reads all of + # it. + test "a full adapter lifecycle leaves stderr clean" do + original = $stderr + captured = StringIO.new + $stderr = captured + begin + server = AsyncHTTPHoldingServer.new(hold: :head) + source = Dexpace::Cancellation.source + reactor_over(server) do + h1 = AsyncHTTPServerFixture.http1 { |_req| [200, [], ["ok"]] } + h2 = AsyncHTTPServerFixture.tls { |_req| [200, [], ["ok"]] } + adapter = AsyncHTTP.build(ssl_context: h2.client_endpoint.ssl_context) + value_within(adapter.call(request(h1.url), nil, nil)).close + value_within(adapter.call(request(h2.url), nil, nil)).close + future = adapter.call(request("http://127.0.0.1:#{server.port}/"), nil, source.token) + server.wait_for_accept + source.cancel(:lifecycle) + assert_raises(Dexpace::CancelledError) { value_within(future) } + adapter.close + h1.close + h2.close + end + server.close + ensure + $stderr = original + end + + assert_empty(captured.string) + end + end +end diff --git a/gems/dexpace-transport-async_http/test/dexpace/transport/async_http/drop_policy_test.rb b/gems/dexpace-transport-async_http/test/dexpace/transport/async_http/drop_policy_test.rb new file mode 100644 index 0000000..5d4b765 --- /dev/null +++ b/gems/dexpace-transport-async_http/test/dexpace/transport/async_http/drop_policy_test.rb @@ -0,0 +1,117 @@ +# frozen_string_literal: true +# SPDX-License-Identifier: MIT + +require_relative "../../../test_helper" +require_relative "../../../support/async_http_recording_sink" +require "dexpace/transport/async_http" + +# TRANSPORT-13 (SHOULD): a configurable policy for how header drops are logged, with the +# per-name dedup mode case-insensitive and bounded, and the OBS-19 policy phase 5b postponed +# here. §17's own conformance clause: "under once-per-header assert the same name warns once +# then goes quiet, a different name warns once." Every emission is under the shared transport +# event, with String field keys, so one assertion reads a drop record from either adapter. +class DexpaceTransportAsyncHTTPDropPolicyTest < DexpaceTestCase + DropPolicy = Dexpace::Transport::AsyncHTTP::DropPolicy + EVENT = Dexpace::Instrumentation::Events::TRANSPORT_HEADER_DROPPED + + def logger_and_sink + sink = AsyncHTTPRecordingSink.new + [Dexpace::Instrumentation::Logger.build(sink: sink), sink] + end + + test "EVERY mode warns on every drop, same name or not" do + logger, sink = logger_and_sink + policy = DropPolicy.build(mode: DropPolicy::EVERY) + + policy.report(logger, "X-Bad:Name", "not a token") + policy.report(logger, "X-Bad:Name", "not a token") + + assert_equal(%i[warn warn], sink.severities(EVENT)) + assert_equal(DropPolicy::EVERY, policy.mode) + end + + test "QUIET never warns" do + logger, sink = logger_and_sink + policy = DropPolicy.build(mode: DropPolicy::QUIET) + + policy.report(logger, "X-Bad:Name", "not a token") + policy.report(logger, "Y-Bad:Name", "not a token") + + assert_equal(%i[debug debug], sink.severities(EVENT)) + end + + test "ONCE_PER_NAME (the default) warns once per distinct folded name, then goes quiet" do + logger, sink = logger_and_sink + policy = DropPolicy.build + + policy.report(logger, "X-Bad:Name", "not a token") + policy.report(logger, "X-Bad:Name", "not a token") + policy.report(logger, "Y-Bad:Name", "not a token") + + assert_equal(%i[warn debug warn], sink.severities(EVENT)) + assert_equal(DropPolicy::ONCE_PER_NAME, policy.mode) + end + + test "the per-name latch is case-insensitive on the folded name (HTTP-13)" do + logger, sink = logger_and_sink + policy = DropPolicy.build + + policy.report(logger, "X-Bad:Name", "not a token") + policy.report(logger, "x-bad:name", "not a token") + policy.report(logger, "X-BAD:NAME", "not a token") + + assert_equal(%i[warn debug debug], sink.severities(EVENT)) + end + + test "bounded at MAX_TRACKED_NAMES distinct names; the next degrades to quiet and is not " \ + "tracked, so a repeat of it is quiet too" do + logger, sink = logger_and_sink + policy = DropPolicy.build + bound = DropPolicy::MAX_TRACKED_NAMES + + bound.times { |i| policy.report(logger, "X-Bad:#{i}", "not a token") } + policy.report(logger, "X-Bad:#{bound}", "not a token") + policy.report(logger, "X-Bad:#{bound}", "not a token") + + assert_equal(64, bound) + assert_equal(bound, sink.severities(EVENT).count(:warn)) + assert_equal(%i[debug debug], sink.severities(EVENT).last(2)) + end + + test "a drop's record carries the header name and the reason under the shared event, with " \ + "String keys" do + logger, sink = logger_and_sink + DropPolicy.build.report(logger, "X-Bad:Name", "not an RFC 7230 token (TRANSPORT-12)") + + record = sink.entries.first.payload + + assert_equal(EVENT, record["event"]) + assert_equal("X-Bad:Name", record["header"]) + assert_equal("not an RFC 7230 token (TRANSPORT-12)", record["reason"]) + end + + test "a rejected mode raises Dexpace::InvalidArgumentError rather than degrading silently" do + error = assert_raises(Dexpace::InvalidArgumentError) { DropPolicy.build(mode: :bogus) } + + assert_match(/mode must be one of/, error.message) + assert_raises(NoMethodError) { DropPolicy.new(mode: DropPolicy::QUIET) } + end + + # OBS-20: every emission is contained -- a sink that raises never reaches the adapter. + test "a raising sink is contained, and the drop still returns nil" do + sink = Object.new + def sink.warn(*) = raise("sink exploded") + def sink.debug(*) = raise("sink exploded") + def sink.info(*) = nil + def sink.error(*) = nil + %i[debug? info? warn? error?].each { |query| sink.define_singleton_method(query) { true } } + logger = Dexpace::Instrumentation::Logger.build(sink: sink) + + assert_nil(DropPolicy.build.report(logger, "X-Bad:Name", "not a token")) + end + + test "MODES is the closed set of three, frozen" do + assert_equal(%i[every once_per_name quiet], DropPolicy::MODES) + assert_predicate(DropPolicy::MODES, :frozen?) + end +end diff --git a/gems/dexpace-transport-async_http/test/dexpace/transport/async_http/endpoints_test.rb b/gems/dexpace-transport-async_http/test/dexpace/transport/async_http/endpoints_test.rb new file mode 100644 index 0000000..f4db014 --- /dev/null +++ b/gems/dexpace-transport-async_http/test/dexpace/transport/async_http/endpoints_test.rb @@ -0,0 +1,75 @@ +# frozen_string_literal: true +# SPDX-License-Identifier: MIT + +require_relative "../../../test_helper" +require "dexpace/transport/async_http" + +# Dispatch step 9: Dexpace::Request#url arrives already parsed by URI::RFC3986_PARSER (phase 1's +# Dexpace::URL.parse!), and this module hands that object to Async::HTTP::Endpoint.new -- never +# to Endpoint.parse, which routes through URI::DEFAULT_PARSER (design §3.5, boundary 19). The +# TLS defaults are the design's: VERIFY_PEER for every host and ALPN offering h2, because +# async-http's own context leaves `localhost` unverified and a caller-supplied context gets no +# ALPN (the design's verified fact 3, re-measured on 0.105.0). +class DexpaceTransportAsyncHTTPEndpointsTest < DexpaceTestCase + Endpoints = Dexpace::Transport::AsyncHTTP.const_get(:Endpoints, false) + + def url(string) = Dexpace::URL.parse!(string) + + test "builds a plaintext endpoint straight from the already-parsed URI" do + endpoint = Endpoints.for(url("http://example.test:8080/a%20b?q=1")) + + assert_equal("example.test", endpoint.url.host) + assert_equal(8080, endpoint.url.port) + assert_equal("/a%20b?q=1", endpoint.path) + assert_nil(endpoint.instance_variable_get(:@options)[:ssl_context]) + end + + test "an https URL always gets an adapter-supplied ssl_context that verifies the peer" do + endpoint = Endpoints.for(url("https://localhost/")) + + assert_equal(::OpenSSL::SSL::VERIFY_PEER, endpoint.ssl_context.verify_mode) + end + + test "the adapter-supplied ssl_context offers h2 and http/1.1 by ALPN" do + endpoint = Endpoints.for(url("https://example.test/")) + + assert_equal(%w[h2 http/1.1], endpoint.ssl_context.alpn_protocols) + assert_equal(Dexpace::Transport::AsyncHTTP::ALPN_PROTOCOLS, endpoint.ssl_context.alpn_protocols) + end + + test "a caller-supplied ssl_context is used verbatim and not silently re-armed" do + context = ::OpenSSL::SSL::SSLContext.new + endpoint = Endpoints.for(url("https://example.test/"), ssl_context: context) + + assert_same(context, endpoint.ssl_context) + assert_nil(endpoint.ssl_context.alpn_protocols, "verbatim means no ALPN was added either") + end + + # Dexpace::URL.parse! admits ftp:// and Endpoint.new would dial port 21: the screen is the + # adapter's (TRANSPORT-21's "a URL the endpoint cannot build" delivered through the future). + test "a scheme other than http or https is refused as InvalidArgumentError, not dialled" do + error = assert_raises(Dexpace::InvalidArgumentError) { Endpoints.for(url("ftp://example.test/")) } + + assert_match(/http and https only/, error.message) + assert_match(/TRANSPORT-21/, error.message) + end + + test "the origin key is scheme, host and port, folded, and two URLs on one origin share it" do + a = Endpoints.origin_for(url("https://Example.test:443/one")) + b = Endpoints.origin_for(url("https://example.test/two")) + c = Endpoints.origin_for(url("https://example.test:8443/one")) + + assert_equal(a, b) + assert_equal(["https", "example.test", 443], a) + refute_equal(a, c) + assert_predicate(a, :frozen?) + end + + # Boundary 19, asserted over the source: the documented entry point is the wrong one here. + test "never calls Endpoint.parse or URI.parse anywhere under lib/" do + sources = Dir.glob(File.expand_path("../../../../lib/**/*.rb", __dir__)) + .map { |path| File.read(path).gsub(/^\s*#.*$/, "") }.join + + refute_match(/Endpoint\.parse|URI\.parse\b|DEFAULT_PARSER/, sources) + end +end diff --git a/gems/dexpace-transport-async_http/test/dexpace/transport/async_http/errors_test.rb b/gems/dexpace-transport-async_http/test/dexpace/transport/async_http/errors_test.rb new file mode 100644 index 0000000..7c9255c --- /dev/null +++ b/gems/dexpace-transport-async_http/test/dexpace/transport/async_http/errors_test.rb @@ -0,0 +1,129 @@ +# frozen_string_literal: true +# SPDX-License-Identifier: MIT + +require_relative "../../../test_helper" +require "dexpace/transport/async_http" + +# Dispatch step 17's classifier. P6-4: "wrap, and default to retryable" -- not one of the +# families async-http raises is an ::IOError but EOFError and IOError (the design's verified fact +# 11), so every one needs Dexpace::TransportError to answer #retryable? at all. TRANSPORT-3: the +# token is asked first, never the exception. P3-3: a body failure after the head is a +# StreamError, a sibling of TransportError and never its descendant. +class DexpaceTransportAsyncHTTPErrorsTest < DexpaceTestCase + Errors = Dexpace::Transport::AsyncHTTP.const_get(:Errors, false) + + NATIVE = [ + ::Async::TimeoutError.new("execution expired"), + ::Errno::ECONNREFUSED.new, + ::SocketError.new("getaddrinfo: Name or service not known"), + ::OpenSSL::SSL::SSLError.new("certificate verify failed"), + ::EOFError.new("end of file reached"), + ::IOError.new("stream closed in another thread"), + ::Protocol::HTTP::RefusedError.new("bad header"), + ::Protocol::HTTP::RemoteError.new("peer reset"), + ::Protocol::HTTP1::BadRequest.new("Invalid content length"), + ::NoMethodError.new("undefined method 'readpartial' for nil"), + ::RuntimeError.new("a family nobody thought of"), + ].freeze + + def none = Dexpace::Cancellation.none + + test "TRANSPORT-4/TRANSPORT-20: every native family wraps into a retryable TransportError " \ + "carrying the original as #cause and the phase given" do + NATIVE.each do |native| + wrapped = Errors.wrap(native, phase: :connect, cancellation: none) + + assert_instance_of(Dexpace::TransportError, wrapped, native.class.to_s) + assert_predicate(wrapped, :retryable?, native.class.to_s) + assert_same(native, wrapped.cause, native.class.to_s) + assert_equal(:connect, wrapped.phase) + assert_includes(wrapped.message, native.message) + end + end + + # The SDK's own errors are in the list too: a ClosedError out of a connection the watcher + # retired arrives under a cancelled token, and the token still wins -- the pass-through below is + # for an UNCANCELLED token only, so an implementation that consulted the class first survives + # every native family and fails exactly here. + test "TRANSPORT-3: asks the cancellation token first -- a cancelled token turns any error " \ + "into CancelledError with the token's reason, an IOError and a Dexpace:: error included" do + source = Dexpace::Cancellation.source + source.cancel(:caller_gave_up) + + ours = [Dexpace::ClosedError.new("retired"), Dexpace::StreamError.new("torn")] + (NATIVE + ours).each do |native| + wrapped = Errors.wrap(native, phase: :read, cancellation: source.token) + + assert_instance_of(Dexpace::CancelledError, wrapped, native.class.to_s) + assert_equal(:caller_gave_up, wrapped.reason) + refute_respond_to(wrapped, :retryable?) + end + end + + test "a Dexpace:: error is passed through unwrapped, unchanged, with no cause added" do + [Dexpace::StreamError.new("already ours"), Dexpace::InvalidArgumentError.new("HTTP-17"), + Dexpace::ClosedError.new("closed"),].each do |original| + assert_same(original, Errors.wrap(original, phase: :connect, cancellation: none)) + assert_nil(original.cause) + end + end + + test "is callable standalone, with no ambient rescue in flight" do + # No begin/rescue anywhere above this line: $! is nil here, and #wrap must still attach the + # argument as #cause rather than depending on an ambient in-flight exception. + wrapped = Errors.wrap(::EOFError.new("no ambient rescue"), phase: :connect, cancellation: none) + + assert_kind_of(::EOFError, wrapped.cause) + end + + test "TRANSPORT-4 never writes to the token: a deadline leaves the flag clear" do + source = Dexpace::Cancellation.source + Errors.wrap(::Async::TimeoutError.new("execution expired"), phase: :connect, + cancellation: source.token,) + + refute_predicate(source.token, :cancelled?) + end + + test "P3-3: a mid-stream body failure classifies as StreamError, the token first, never " \ + "TransportError" do + eof = ::EOFError.new("end of file reached") + classified = Errors.classify_read(eof, cancellation: none) + + assert_instance_of(Dexpace::StreamError, classified) + assert_same(eof, classified.cause) + refute_respond_to(classified, :retryable?) + refute_operator(Dexpace::StreamError, :<, Dexpace::TransportError) + + source = Dexpace::Cancellation.source + source.cancel(:mid_body) + + assert_instance_of(Dexpace::CancelledError, + Errors.classify_read(eof, cancellation: source.token),) + end + + # ASYNC-6: only Completer#request_cancel produces the settlement Future#cancelled? reads as + # true; a failure carrying a CancelledError instance would leave it false. + test "settle routes a cancelled token to #request_cancel and everything else to #fail" do + source = Dexpace::Cancellation.source + source.cancel(:gone) + cancelled = Dexpace::Async::Completer.new + Errors.settle(cancelled, ::EOFError.new, phase: :connect, cancellation: source.token) + + assert_predicate(cancelled.future, :cancelled?) + assert_equal(:gone, assert_raises(Dexpace::CancelledError) { cancelled.future.value }.reason) + + failed = Dexpace::Async::Completer.new + Errors.settle(failed, ::EOFError.new, phase: :connect, cancellation: none) + + refute_predicate(failed.future, :cancelled?) + assert_predicate(assert_raises(Dexpace::TransportError) { failed.future.value }, :retryable?) + end + + test "never sees Async::Cancel, by construction" do + # Errors.wrap is only ever called from a `rescue ::StandardError` arm, which Async::Cancel + # (< Exception) cannot reach; this documents the invariant the arms rest on. + refute_operator(::Async::Cancel, :<, ::StandardError) + assert_operator(::Async::TimeoutError, :<, ::StandardError) + assert_same(::Async::Stop, ::Async::Cancel) + end +end diff --git a/gems/dexpace-transport-async_http/test/dexpace/transport/async_http/matrix_facts_test.rb b/gems/dexpace-transport-async_http/test/dexpace/transport/async_http/matrix_facts_test.rb new file mode 100644 index 0000000..c646edf --- /dev/null +++ b/gems/dexpace-transport-async_http/test/dexpace/transport/async_http/matrix_facts_test.rb @@ -0,0 +1,130 @@ +# frozen_string_literal: true +# SPDX-License-Identifier: MIT + +require_relative "../../../test_helper" +require "dexpace/transport/async_http" +require "openssl" + +# Exercises: TRANSPORT-2, TRANSPORT-7, TRANSPORT-8, ASYNC-22 (the async-http facts +# they rest on) -- the phase-8c design's verified facts the adapter's shape depends on, re-run as +# a standing test on every CI row rather than once in a scratch script (8a's precedent). The +# gemspec pins `async-http ~> 0.104`, so what varies across the matrix is the interpreter (3.3, +# 3.4 and 4.0; the 3.2 row has no bundle for this gem, P8-36) and the openssl the bundle +# resolved for it. The first test prints the row's versions so a CI log records which ones each +# row proved. No lib/ mirror: it asserts the library, not a file. +class DexpaceTransportAsyncHTTPMatrixFactsTest < DexpaceTestCase + test "the active async-http is 0.104 or newer, and the row's versions are printed for the " \ + "record" do + openssl = Gem.loaded_specs["openssl"] + protocol = Gem.loaded_specs["protocol-http"]&.version + puts "\n[phase 8c matrix] ruby #{RUBY_VERSION}: async-http #{::Async::HTTP::VERSION}, " \ + "async #{::Async::VERSION}, protocol-http #{protocol}, openssl #{OpenSSL::VERSION} " \ + "(#{openssl&.default_gem? ? "default gem" : "installed gem"})" + + assert_operator(Gem::Version.new(::Async::HTTP::VERSION), :>=, Gem::Version.new("0.104")) + end + + test "TRANSPORT-8 / XCUT-2 fact: Async::Cancel is an Exception outside StandardError and " \ + "Async::TimeoutError is a StandardError, so the pair is told apart by class" do + refute_operator(::Async::Cancel, :<, ::StandardError) + assert_operator(::Async::Cancel, :<, ::Exception) + assert_operator(::Async::TimeoutError, :<, ::StandardError) + assert_same(::Async::Cancel, ::Async::Stop, "Stop is Cancel's older name") + end + + test "P8-39 fact: outside a reactor there is no current task and no scheduler, which is what " \ + "the adapter's SeamError reads" do + assert_nil(::Async::Task.current?) + assert_nil(Fiber.scheduler) + end + + test "watcher fact: Task#cancel from a foreign OS thread raises rather than cancelling, which " \ + "is why the cancellation crosses threads through a queue and never a cancel" do + error = nil + Sync do |task| + child = task.async { sleep(5) } + ::Thread.new do + child.cancel + rescue ::StandardError => error # the block closes over the outer local + error + end.join + + assert_equal(:running, child.status, "the foreign cancel did not land") + child.cancel + end + + assert_kind_of(::NoMethodError, error) + end + + test "watcher fact: Task#cancel(cause:) keeps an Exception cause and replaces anything else " \ + "with the runtime's own, which is why the bridge wraps the reason in CancelledError" do + seen = cancel_with(cause: ::RuntimeError.new("ours")) + + assert_kind_of(::RuntimeError, seen.cause) + assert_equal("ours", seen.cause.message) + assert_kind_of(::Async::Cancel::Cause, cancel_with(cause: :ours).cause) + end + + test "P8-39 fact: Task#async runs the child eagerly to its first suspension and hands control " \ + "back to the caller's fiber, so a future assigned inside the child is not yet readable" do + order = [] + Sync do |task| + child = task.async do + order << :child_started + sleep(0.01) + order << :child_resumed + end + order << :caller_continued + child.wait + end + + assert_equal(%i[child_started caller_continued child_resumed], order) + end + + test "ASYNC-22 fact: Fiber.scheduler is one object across every task of one reactor and a " \ + "different one on another thread, which is what keys the client map" do + inner = nil + other = nil + outer = Sync do |task| + inner = task.async { Fiber.scheduler }.wait + other = ::Thread.new { Sync { Fiber.scheduler } }.value + Fiber.scheduler + end + + assert_same(outer, inner) + refute_same(outer, other) + assert_predicate(other, :closed?, "the other thread's reactor closed with its Sync") + end + + test "TRANSPORT-2 fact: Async::HTTP::Client.new takes retries: and limit:, opens no socket, " \ + "and a caller's ssl_context reaches the endpoint verbatim" do + context = OpenSSL::SSL::SSLContext.new + uri = ::URI::RFC3986_PARSER.parse("https://127.0.0.1:1") + endpoint = ::Async::HTTP::Endpoint.new(uri, ssl_context: context) + client = ::Async::HTTP::Client.new(endpoint, retries: 0, limit: 3) + + assert_equal(0, client.retries) + assert_equal(3, client.pool.limit) + assert_equal(0, client.pool.size) + assert_same(context, endpoint.ssl_context) + end + + private + + # The cancellation a child sees under `cancel(cause:)`, as the rescued Async::Cancel. + def cancel_with(cause:) + seen = nil + Sync do |task| + child = task.async do + sleep(5) + rescue ::Exception => error # rubocop:disable Lint/RescueException -- Async::Cancel is an Exception, and the fact under test is what it carries + seen = error + raise + end + sleep(0.01) + child.cancel(cause: cause) + child.wait + end + seen + end +end diff --git a/gems/dexpace-transport-async_http/test/dexpace/transport/async_http/parent_cancellation_test.rb b/gems/dexpace-transport-async_http/test/dexpace/transport/async_http/parent_cancellation_test.rb new file mode 100644 index 0000000..aa13ea1 --- /dev/null +++ b/gems/dexpace-transport-async_http/test/dexpace/transport/async_http/parent_cancellation_test.rb @@ -0,0 +1,211 @@ +# frozen_string_literal: true +# SPDX-License-Identifier: MIT + +require_relative "../../../test_helper" +require_relative "../../../support/async_http_silent_server" +require_relative "../../../support/async_http_recording_body" +require_relative "../../../support/async_http_reactor" +require "dexpace/transport/async_http" + +# R14: TRANSPORT-8's antecedent, measured live -- a cancellation "originating inside" the native +# client, from the host runtime's own structured-concurrency scope, while the SDK future is still +# live -- paired with a genuine timeout on the SAME withheld-head path, because the pair IS the +# requirement: the first settles a terminal, non-retryable cancellation, the second a retryable +# transport failure, discriminated by class (Async::Cancel < Exception against +# Async::TimeoutError < StandardError, XCUT-2) and never by message. This is the row §12 records +# as vacuous and this adapter satisfies; the §12 correction is on phase 10's inbound list. Two +# nested classes under Metrics/ClassLength: the pair, and the close discipline under both. +module DexpaceTransportAsyncHTTPParentCancellationTest + # The request builder, the bounded wait and the supervisor both classes share. + module ParentCancellationTestSupport + include AsyncHTTPReactor + + AsyncHTTP = Dexpace::Transport::AsyncHTTP + BOUND = 5.0 + + def request(url) + builder = Dexpace::Request.builder + builder.url = url + builder.build + end + + def value_within(future) + future.value(deadline: Dexpace::Clock.deadline_in(BOUND)) + end + + # A supervisor task that makes the call and then parks on a queue -- never a sleep -- until the + # test closes it, so the exchange stays its live child. `Task#async` runs a child eagerly only + # to its first suspension and hands control back to the CALLER's fiber then, so the future the + # supervisor assigns is read only after it says it has it. + def supervise(root, adapter, request, options: nil) + ready = ::Thread::Queue.new + park = ::Thread::Queue.new + future = nil + supervisor = root.async do + future = adapter.call(request, options, nil) + ready.push(true) + park.pop + end + ready.pop + [supervisor, future, park] + end + end + + # TRANSPORT-8 and its timeout pair over one withheld-head path, discriminated by class. + class PairTest < DexpaceTestCase + include ParentCancellationTestSupport + + # The exchange Adapter#call spawns is a CHILD of `supervisor`, because `task.async` inside + # #call reads Async::Task.current at the moment #call runs. Cancelling `supervisor` from `root` + # -- a sibling relationship, not self-cancellation -- cascades into the exchange exactly as an + # "internal cancel-all" would (TRANSPORT-8's own example phrase). No Dexpace::Cancellation is + # involved and no Future#cancel: nothing in the SDK asked for it. + test "TRANSPORT-8: cancelling a PARENT task delivers Async::Cancel into the still-live " \ + "exchange and settles a terminal, non-retryable CancelledError" do + server = AsyncHTTPSilentServer.new + adapter = AsyncHTTP.build + + reactor_over(server) do |root| + supervisor, future, park = supervise(root, adapter, request("http://127.0.0.1:#{server.port}/")) + server.wait_for_accept + supervisor.cancel + + error = assert_raises(Dexpace::CancelledError) { value_within(future) } + assert_equal(:async_cancelled, error.reason) + assert_predicate(future, :cancelled?) + refute_respond_to(error, :retryable?) + assert_equal(:cancelled, supervisor.status) + park.close + end + ensure + adapter&.close + server&.close + end + + test "TRANSPORT-8's pair: a with_timeout expiry on the SAME withheld-head path settles a " \ + "RETRYABLE Dexpace::TransportError, never CancelledError" do + server = AsyncHTTPSilentServer.new + adapter = AsyncHTTP.build + + reactor_over(server) do |task| + options = Dexpace::RequestOptions.builder.tap { |b| b.timeout = 0.1 }.build + future = adapter.call(request("http://127.0.0.1:#{server.port}/"), options, nil) + server.wait_for_accept + + error = assert_raises(Dexpace::TransportError) { value_within(future) } + assert_predicate(error, :retryable?) + assert_kind_of(::Async::TimeoutError, error.cause) + refute_predicate(future, :cancelled?) + assert_exchange_released(task) + end + ensure + adapter&.close + server&.close + end + end + + # R13 and TRANSPORT-22: what a runtime cancellation, a deadline or an adaptation failure closes, + # and exactly once. + class CloseDisciplineTest < DexpaceTestCase + include ParentCancellationTestSupport + + # R13, measured rather than assumed: on this adapter the native response exists only once + # `Client#call` has returned, and no checkpoint lies between that return and delivery, so a + # runtime cancellation or a deadline landing INSIDE the native call finds nothing of the + # exchange's to close -- the library's own ensure releases the connection -- and the + # undelivered-response close is reached on two paths: check-after-resume + # (cancellation_test.rb's TRANSPORT-9 case, close_count 1) and an adaptation failure after + # the head (TRANSPORT-22, below). + test "R13: a runtime cancellation or a deadline landing inside the native call leaves no " \ + "undelivered response, and the pivot still settles" do + %i[cancel timeout].each do |path| + native = AsyncHTTPRecordingBody.new(["late".b], length: 4) + gate = ::Thread::Queue.new + client = Object.new + client.define_singleton_method(:retries) { 0 } + client.define_singleton_method(:pool) { Object.new.tap { |pool| def pool.close = nil } } + client.define_singleton_method(:call) do |_native| + gate.pop # suspends the exchange at a checkpoint until the test releases it + ::Protocol::HTTP::Response.new("HTTP/1.1", 200, ::Protocol::HTTP::Headers.new, native) + end + adapter = AsyncHTTP.using(client) + + Sync do |root| + options = Dexpace::RequestOptions.builder.tap { |b| b.timeout = 0.1 }.build + supervisor, future, park = supervise(root, adapter, request("http://example.test/"), + options: options,) + if path == :cancel + supervisor.cancel + assert_raises(Dexpace::CancelledError) { value_within(future) } + else + assert_raises(Dexpace::TransportError) { value_within(future) } + park.close + end + gate.close + end + + assert_equal(0, native.close_count, "#{path}: the response never existed to be closed") + end + end + + # TRANSPORT-22's adaptation-failure half: a head phase 1's model refuses -- a 999 status -- + # raises InvalidArgumentError through the future, and the native body the exchange was holding + # is closed exactly once before the future settles. + test "TRANSPORT-22: an adaptation failure after the head closes the native body exactly once " \ + "and settles InvalidArgumentError through the future" do + native = AsyncHTTPRecordingBody.new(["late".b], length: 4) + client = Object.new + client.define_singleton_method(:retries) { 0 } + client.define_singleton_method(:pool) { Object.new.tap { |pool| def pool.close = nil } } + client.define_singleton_method(:call) do |_native| + ::Protocol::HTTP::Response.new("HTTP/1.1", 999, ::Protocol::HTTP::Headers.new, native) + end + adapter = AsyncHTTP.using(client) + + Sync do + future = adapter.call(request("http://example.test/"), nil, nil) + + assert_raises(Dexpace::InvalidArgumentError) { value_within(future) } + end + + assert_equal(1, native.close_count) + end + + # The net under #run's ensure, reached only when a runtime cancellation lands INSIDE an exit + # arm: the adaptation failure above, with the undelivered native body's close suspending at a + # checkpoint (a native close that waits on the peer's stream reset would) and the parent + # cancelled there. Async::Cancel leaves the rescue arm before Errors.settle ran, so the + # ensure's net is the only thing left to settle the pivot -- cancelled, never left pending + # (review round 0's R0-4, a surviving mutant). No sleep: the body parks on a queue the test + # never pushes to, and the cancellation is what wakes it. + test "a runtime cancellation landing inside an exit arm's native close still settles the " \ + "pivot cancelled, through the ensure's net" do + gate = ::Thread::Queue.new + native = AsyncHTTPRecordingBody.new(["late".b], length: 4) + native.define_singleton_method(:close) { |error = nil| gate.pop && super(error) } + client = Object.new + client.define_singleton_method(:retries) { 0 } + client.define_singleton_method(:pool) { Object.new.tap { |pool| def pool.close = nil } } + client.define_singleton_method(:call) do |_native| + ::Protocol::HTTP::Response.new("HTTP/1.1", 999, ::Protocol::HTTP::Headers.new, native) + end + adapter = AsyncHTTP.using(client) + + Sync do |root| + supervisor, future, park = supervise(root, adapter, request("http://example.test/")) + + refute_predicate(future, :settled?, "the exit arm is parked inside the native close") + supervisor.cancel + + error = assert_raises(Dexpace::CancelledError) { value_within(future) } + assert_equal(:async_cancelled, error.reason) + assert_predicate(future, :cancelled?) + assert_exchange_released(root) + park.close + gate.close + end + + assert_equal(0, native.close_count, "the native close was interrupted, never completed") + end + end +end diff --git a/gems/dexpace-transport-async_http/test/dexpace/transport/async_http/request_body_test.rb b/gems/dexpace-transport-async_http/test/dexpace/transport/async_http/request_body_test.rb new file mode 100644 index 0000000..605e810 --- /dev/null +++ b/gems/dexpace-transport-async_http/test/dexpace/transport/async_http/request_body_test.rb @@ -0,0 +1,90 @@ +# frozen_string_literal: true +# SPDX-License-Identifier: MIT + +require_relative "../../../test_helper" +require "dexpace/transport/async_http" + +# A Protocol::HTTP::Body::Readable over a Dexpace::Body, pulled through phase 3a's +# BufferedSource.over, so an outbound body is never materialised -- the design's verified fact 7 +# measured exactly one #read per chunk plus one for end of stream over the reference library, +# and chunked framing on the wire for an unknown length. TRANSPORT-17's second guarantee: a +# single-use body refuses to rewind, so the library's own Request#retry! never re-reads it. +class DexpaceTransportAsyncHTTPRequestBodyTest < DexpaceTestCase + RequestBody = Dexpace::Transport::AsyncHTTP.const_get(:RequestBody, false) + + def two_chunk_body + Class.new do + include Dexpace::Body + + def write_to(sink) + sink.write("ab".b) + sink.write("cd".b) + 4 + end + end.new + end + + test "is a Protocol::HTTP::Body::Readable, so the library pulls it and never materialises it" do + assert_kind_of(::Protocol::HTTP::Body::Readable, RequestBody.new(Dexpace::Body.bytes("x".b))) + end + + test "reads chunks on demand, BINARY, then nil at end of stream" do + body = RequestBody.new(two_chunk_body) + chunks = [] + while (chunk = body.read) + chunks << chunk + end + + assert_equal("abcd", chunks.join) + assert(chunks.all? { |chunk| chunk.encoding == ::Encoding::BINARY }) + assert_nil(body.read) + end + + test "#length reports the wrapped body's own content_length, or nil for the -1 sentinel" do + assert_equal(3, RequestBody.new(Dexpace::Body.bytes("abc".b)).length) + assert_nil(RequestBody.new(two_chunk_body).length) + end + + test "#rewindable? mirrors the wrapped body's #replayable?, and #rewind resets the read" do + replayable = RequestBody.new(Dexpace::Body.bytes("abc".b)) + + assert_predicate(replayable, :rewindable?) + assert_equal("abc", replayable.read) + assert(replayable.rewind) + assert_equal("abc", replayable.read) + end + + test "a single-use (non-replayable) body refuses to rewind (TRANSPORT-17)" do + single_use = RequestBody.new(two_chunk_body) + single_use.read + + refute_predicate(single_use, :rewindable?) + refute(single_use.rewind) + end + + test "the library's own retry gate refuses a POST and a non-rewindable body alike" do + request = ::Protocol::HTTP::Request.new("http", "h", "POST", "/", nil, + ::Protocol::HTTP::Headers.new, + RequestBody.new(Dexpace::Body.bytes("abc".b)),) + get = ::Protocol::HTTP::Request.new("http", "h", "GET", "/", nil, ::Protocol::HTTP::Headers.new, + RequestBody.new(two_chunk_body),) + + refute(request.retry!) + refute(get.retry!) + end + + test "#close releases the pull source and never closes the caller's Dexpace::Body" do + closes = 0 + source = Class.new do + include Dexpace::Body + + define_method(:write_to) { |sink| sink.write("x".b) } + define_method(:close) { closes += 1 } + end.new + body = RequestBody.new(source) + body.read + + assert_nil(body.close) + assert_equal(0, closes) + end +end diff --git a/gems/dexpace-transport-async_http/test/dexpace/transport/async_http/request_mapper_test.rb b/gems/dexpace-transport-async_http/test/dexpace/transport/async_http/request_mapper_test.rb new file mode 100644 index 0000000..9792c4a --- /dev/null +++ b/gems/dexpace-transport-async_http/test/dexpace/transport/async_http/request_mapper_test.rb @@ -0,0 +1,226 @@ +# frozen_string_literal: true +# SPDX-License-Identifier: MIT + +require_relative "../../../test_helper" +require_relative "../../../support/async_http_recording_sink" +require "dexpace/transport/async_http" + +# Dispatch steps 4 to 9. The wire-boundary re-validation phase 1 postponed to the adapters +# (HTTP-17, HTTP-18, XCUT-18): HeaderSyntax re-run immediately before dispatch, on every name +# and outbound value. TRANSPORT-11: the framing-header drop set, ten folded names, each a +# smuggling vector on this adapter (a caller-set host or content-length is APPENDED beside the +# library's own). TRANSPORT-12/13, P8-40: the RFC 7230 token predicate applied before dispatch, +# on both protocols, over the seventeen bytes HTTP-17 admits and the token set refuses. Two +# nested classes under Metrics/ClassLength: the header gates, and the rest of the mapping. +module DexpaceTransportAsyncHTTPRequestMapperTest + # The request builder, the recording logger, the mapper call and the forged requests both + # classes share. + module RequestMapperTestSupport + RequestMapper = Dexpace::Transport::AsyncHTTP.const_get(:RequestMapper, false) + RequestBody = Dexpace::Transport::AsyncHTTP.const_get(:RequestBody, false) + DropPolicy = Dexpace::Transport::AsyncHTTP::DropPolicy + EVENT = Dexpace::Instrumentation::Events::TRANSPORT_HEADER_DROPPED + + # Request::Builder exposes a writer per member plus #header(name, value) and no readers, so + # headers are accumulated here rather than read back. + def request(headers: {}, body: nil, method: "GET", url: "https://example.test/p?q=1") + builder = Dexpace::Request.builder + builder.method = method + builder.url = url + headers.each { |name, value| builder.header(name, value) } + builder.body = body + builder.build + end + + def logger_and_sink + sink = AsyncHTTPRecordingSink.new + [Dexpace::Instrumentation::Logger.build(sink: sink), sink] + end + + def map(request, logger: Dexpace::Instrumentation::Logger::NULL, policy: DropPolicy.build) + RequestMapper.call(request, logger: logger, drop_policy: policy) + end + + def names(native) = native.headers.to_a.map { |name, _| name.downcase } + + # A request-shaped object answering the four readers the seam contract types nothing about, + # carrying headers no Dexpace validation has seen -- design §10.10's admitted hole -- and, + # with `body:`, a body on a method the builder's HTTP-7 would have refused it on. + def forged_request(name, value) + forged_request_with([name, value]) + end + + def forged_request_with(*pairs, method: "GET", body: nil) + template = request(method: method) + headers = Object.new + headers.define_singleton_method(:each_entry) do |&block| + pairs.each do |pair| + block.call(*pair) + end + end + forged = Object.new + forged.define_singleton_method(:method) { template.method } + forged.define_singleton_method(:url) { template.url } + forged.define_singleton_method(:headers) { headers } + forged.define_singleton_method(:body) { body } + forged + end + end + + # The re-validation, the framing drop set and the token predicate: what never reaches the wire. + class HeaderGatesTest < DexpaceTestCase + include RequestMapperTestSupport + + test "HTTP-17: a header name HeaderSyntax rejects raises before anything is mapped" do + forged = forged_request("X-Evil\r\nInjected", "v") + + error = assert_raises(Dexpace::InvalidArgumentError) { map(forged) } + + assert_match(/HTTP-17/, error.message) + end + + test "HTTP-18: an outbound value HeaderSyntax rejects raises before anything is mapped" do + forged = forged_request("X-Evil", "a\r\nInjected: 1") + + error = assert_raises(Dexpace::InvalidArgumentError) { map(forged) } + + assert_match(/HTTP-18/, error.message) + end + + test "TRANSPORT-11: the ten framing headers are dropped and logged verbose, " \ + "case-insensitively" do + logger, sink = logger_and_sink + framing = Dexpace::Transport::AsyncHTTP::FRAMING_HEADERS.map.with_index do |name, index| + [index.even? ? name.upcase : name.capitalize, "v"] + end.to_h + native = map(request(headers: framing.merge("X-Keep" => "yes")), logger: logger) + + assert_equal(["x-keep"], names(native)) + assert_equal(%i[debug] * 10, sink.severities(EVENT)) + assert_equal("transport framing header (TRANSPORT-11)", + sink.events(EVENT).first.payload["reason"],) + end + + test "TRANSPORT-11: the drop set is exactly the ten folded names the charter fixes, and " \ + "proxy-authorization is not among them" do + expected = %w[host content-length transfer-encoding connection keep-alive proxy-connection te + trailer upgrade expect] + + assert_equal(expected, Dexpace::Transport::AsyncHTTP::FRAMING_HEADERS) + assert_predicate(Dexpace::Transport::AsyncHTTP::FRAMING_HEADERS, :frozen?) + native = map(request(headers: { "Proxy-Authorization" => "Basic abc" })) + + assert_includes(names(native), "proxy-authorization") + end + + test "TRANSPORT-12/13, P8-40: a model-valid non-token name is dropped, reported through the " \ + "policy, and every other header still maps" do + logger, sink = logger_and_sink + req = request(headers: { "X-Bad:Name" => "v", "X-Normal" => "n" }) + + assert(Dexpace::HeaderSyntax.valid_name?("X-Bad:Name"), "the model admits it (HTTP-17)") + refute(Dexpace::HeaderSyntax.token?("X-Bad:Name"), "the wire grammar refuses it") + native = map(req, logger: logger) + + assert_equal(["x-normal"], names(native)) + assert_equal(%i[warn], sink.severities(EVENT)) + assert_equal("X-Bad:Name", sink.events(EVENT).first.payload["header"]) + assert_match(/TRANSPORT-12/, sink.events(EVENT).first.payload["reason"]) + end + + # The antecedent, measured: exactly the bytes HTTP-17 admits and RFC 7230's tchar refuses. + test "TRANSPORT-12: the seventeen bytes the model admits and the token grammar refuses" do + admitted = (0x21..0x7E).map(&:chr).select do |byte| + Dexpace::HeaderSyntax.valid_name?("x#{byte}") && !Dexpace::HeaderSyntax.token?("x#{byte}") + end + + assert_equal('"(),/:;<=>?@[\]{}'.chars, admitted) + admitted.each do |byte| + native = map(request(headers: { "x#{byte}" => "v", "X-Ok" => "1" })) + + assert_equal(["x-ok"], names(native), byte.inspect) + end + end + end + + # Content-Type's three cases, the body, the target and authority, and a name's spelling. + class MappingTest < DexpaceTestCase + include RequestMapperTestSupport + + test "TRANSPORT-10: an explicit Content-Type wins over the body's own media type" do + body = Dexpace::Body.string("{}", media_type: Dexpace::MediaType.parse("application/json")) + native = map(request(headers: { "Content-Type" => "text/plain" }, body: body, method: "POST")) + + assert_equal([["Content-Type", "text/plain"]], + native.headers.to_a.select { |name, _| name.downcase == "content-type" },) + end + + test "TRANSPORT-10: with no explicit header, the body's own media type is emitted" do + body = Dexpace::Body.string("{}", media_type: Dexpace::MediaType.parse("application/json")) + native = map(request(body: body, method: "POST")) + + assert_equal([["content-type", "application/json"]], + native.headers.to_a.select { |name, _| name.downcase == "content-type" },) + end + + test "TRANSPORT-10: no header and no media type invents nothing -- async-http stamps no " \ + "default of its own, unlike Net::HTTP" do + native = map(request(body: Dexpace::Body.bytes("raw".b), method: "POST")) + + refute_includes(names(native), "content-type") + end + + test "a body-less request maps to a nil native body" do + assert_nil(map(request(method: "POST")).body) + assert_nil(map(request(method: "GET")).body) + end + + # Dispatch step 8: a body-forbidden method gets no body attached. Through the model the guard + # is unreachable -- HTTP-7 already makes a GET's body nil at the builder -- so it is asserted + # over the forged shape that met no builder (design §10.10's hole), where a body on a GET or + # a HEAD would otherwise go out chunked; the forged POST is the control that shows the + # fixture carries its body through (review round 2's R2-2). + test "a body-forbidden method attaches none, even on a forged request carrying one" do + smuggled = Dexpace::Body.bytes("smuggled".b) + + %w[GET HEAD].each do |method| + forged = forged_request_with(%w[X-Ok 1], method: method, body: smuggled) + + assert_predicate(forged.method, :body_forbidden?, method) + assert_nil(map(forged).body, method) + end + control = map(forged_request_with(%w[X-Ok 1], method: "POST", body: smuggled)) + + assert_kind_of(RequestBody, control.body) + end + + test "a body maps to a RequestBody the library pulls; framing is never copied from a header" do + native = map(request(headers: { "Content-Length" => "999" }, + body: Dexpace::Body.bytes("abc".b), method: "POST",)) + + assert_kind_of(RequestBody, native.body) + assert_equal(3, native.body.length) + refute_includes(names(native), "content-length") + end + + test "the native request carries the scheme, the authority with a non-default port, the " \ + "method token and the request target with its query" do + native = map(request(url: "https://example.test:8443/a%20b?q=1", method: "POST")) + default = map(request(url: "http://example.test/")) + + assert_equal("https", native.scheme) + assert_equal("example.test:8443", native.authority) + assert_equal("POST", native.method) + assert_equal("/a%20b?q=1", native.path) + assert_equal("example.test", default.authority) + end + + test "a caller's spelling of a name survives as spelled, and a duplicate name twice" do + native = map(request(headers: { "X-MiXeD-CaSe" => "1" })) + twice = map(forged_request_with(%w[Set-Cookie a], %w[Set-Cookie b])) + + assert_includes(native.headers.to_a, %w[X-MiXeD-CaSe 1]) + assert_equal(2, twice.headers.to_a.count { |name, _| name == "Set-Cookie" }) + end + end +end diff --git a/gems/dexpace-transport-async_http/test/dexpace/transport/async_http/response_body_test.rb b/gems/dexpace-transport-async_http/test/dexpace/transport/async_http/response_body_test.rb new file mode 100644 index 0000000..2f4fccb --- /dev/null +++ b/gems/dexpace-transport-async_http/test/dexpace/transport/async_http/response_body_test.rb @@ -0,0 +1,190 @@ +# frozen_string_literal: true +# SPDX-License-Identifier: MIT + +require_relative "../../../test_helper" +require_relative "../../../support/async_http_recording_body" +require "dexpace/transport/async_http" + +# The design's verified fact 6: lazy, pull-shaped, BINARY, unfrozen; close does not drain. +# Protocol::HTTP::Body::Readable's own #each closes in its own ensure, which is why this class +# drives the native #read directly and closes through its OWN latch -- one close path, never two +# racing ones. P3-3: a native failure mid-stream is a Dexpace::StreamError, the token asked first. +# ASYNC-21's property, asserted on 7b's precedent although the ID is N/A: one native read per +# yield, nothing read ahead. Two nested classes under Metrics/ClassLength: the pull and the close, +# and the read surface with its failures. +module DexpaceTransportAsyncHTTPResponseBodyTest + # The one constructor both classes share. + module ResponseBodyTestSupport + ResponseBody = Dexpace::Transport::AsyncHTTP.const_get(:ResponseBody, false) + + def body(native, cancellation: Dexpace::Cancellation.none, on_release: nil, length: -1) + ResponseBody.new(native: native, media_type: nil, content_length: length, + cancellation: cancellation, on_release: on_release,) + end + end + + # One native read per yield, and one close through the body's own latch whichever path took it. + class PullAndCloseTest < DexpaceTestCase + include ResponseBodyTestSupport + + # The first chunk arrives tagged UTF-8, as a library that trusted a charset would hand it, + # and the second BINARY: the retag is asserted on the one the native body did NOT already + # tag, because a double handing over BINARY chunks alone cannot see `String#b` go missing + # (review round 0's R0-3, a surviving mutant). + test "#each yields one native #read per chunk, retagged BINARY, and stops at nil" do + utf8 = +"a" # a source literal: UTF-8, and unfrozen as a native chunk is + native = AsyncHTTPRecordingBody.new([utf8, "b".b]) + chunks = body(native).each.map { |chunk| chunk } + + assert_equal(::Encoding::UTF_8, utf8.encoding, "the fixture hands over a UTF-8 chunk") + assert_equal(%w[a b], chunks) + assert_equal([::Encoding::BINARY, ::Encoding::BINARY], chunks.map(&:encoding)) + assert_equal(3, native.reads, "two chunks plus one read for end of stream") + end + + test "ASYNC-21's property: exactly one native read per yield, nothing read ahead of demand" do + native = AsyncHTTPRecordingBody.new(["a".b, "b".b, "c".b]) + remaining = body(native).each.map { |_chunk| native.remaining } + + assert_equal([2, 1, 0], remaining) + end + + test "#each closes the native body exactly once, on natural exhaustion, and an explicit " \ + "#close afterwards is a no-op" do + native = AsyncHTTPRecordingBody.new(["a".b]) + subject = body(native) + + subject.each { |_chunk| nil } + subject.close + + assert_equal(1, native.close_count) + assert_predicate(subject, :closed?) + end + + test "#close before consumption releases the native body exactly once, idempotently " \ + "(TRANSPORT-16)" do + native = AsyncHTTPRecordingBody.new(["a".b, "b".b]) + subject = body(native) + + subject.close + subject.close + + assert_equal(1, native.close_count) + assert_equal(0, native.reads, "close does not drain") + end + + test "a read after #close raises ClosedError rather than re-opening anything" do + subject = body(AsyncHTTPRecordingBody.new(["a".b])) + subject.close + + assert_raises(Dexpace::ClosedError) { subject.each { |_chunk| nil } } + end + + test "the release hook runs once, after the native close, whichever path closed it" do + calls = [] + native = AsyncHTTPRecordingBody.new(["a".b]) + subject = body(native, on_release: -> { calls << native.close_count }) + + subject.each { |_chunk| nil } + subject.close + + assert_equal([1], calls) + end + end + + # #source, #content_length and #write_to, then the failures: mid-stream, under a cancelled + # token, and a cancel between chunks. + class ReadSurfaceTest < DexpaceTestCase + include ResponseBodyTestSupport + + test "#source is built with BufferedSource.over and is the same handle every call (BODY-14)" do + subject = body(AsyncHTTPRecordingBody.new(["ab".b, "cd".b])) + + assert_instance_of(Dexpace::IO::BufferedSource, subject.source) + assert_same(subject.source, subject.source) + source = subject.source + refute_predicate(source, :owns_upstream?) if source.respond_to?(:owns_upstream?) + end + + # A fresh view per call would strand the bytes the first view had buffered. + test "reading through #source twice continues where the first read stopped" do + subject = body(AsyncHTTPRecordingBody.new(["ab".b, "cd".b])) + + assert_equal("a", subject.source.read(1)) + assert_equal("bcd", subject.source.read) + end + + test "#content_length is the -1 sentinel for an unknown length and exact otherwise (BODY-35)" do + assert_equal(-1, body(AsyncHTTPRecordingBody.new([], length: nil)).content_length) + exact = body(AsyncHTTPRecordingBody.new(["hi".b], length: 2), length: 2) + + assert_equal(2, exact.content_length) + end + + test "#write_to copies every chunk once and is single-use (BODY-6)" do + subject = body(AsyncHTTPRecordingBody.new(["ab".b, "cd".b])) + buffer = Dexpace::IO::Buffer.new + + assert_equal(4, subject.write_to(buffer)) + assert_equal("abcd", buffer.read) + assert_raises(Dexpace::StreamError) { subject.write_to(Dexpace::IO::Buffer.new) } + end + + # A body shorter than its Content-Length raises a bare EOFError from the library (measured): + # the consumer sees phase 3a's contract, with the original as the cause, and the body is closed. + test "P3-3: a native failure mid-stream surfaces as StreamError with the cause, and closes" do + native = AsyncHTTPRecordingBody.new(["ab".b, "cd".b], raise_after: 1) + subject = body(native) + chunks = [] + + error = assert_raises(Dexpace::StreamError) { subject.each { |chunk| chunks << chunk } } + + assert_equal(["ab"], chunks) + assert_kind_of(::EOFError, error.cause) + assert_equal(1, native.close_count) + refute_respond_to(error, :retryable?) + end + + test "TRANSPORT-3 on the body path: a native failure under a cancelled token is the " \ + "cancellation, not a stream failure" do + source = Dexpace::Cancellation.source + native = AsyncHTTPRecordingBody.new(["ab".b, "cd".b], raise_after: 1, + error: ::IOError.new("closed stream"),) + subject = body(native, cancellation: source.token) + source.cancel(:reader_gave_up) + + error = assert_raises(Dexpace::CancelledError) { subject.each { |_chunk| nil } } + + assert_equal(:reader_gave_up, error.reason) + assert_equal(1, native.close_count) + end + + # Check-after-resume on the body path: a chunk that arrived after the token was cancelled is + # not yielded. + test "a token cancelled between chunks is honoured before the next chunk is yielded" do + source = Dexpace::Cancellation.source + native = AsyncHTTPRecordingBody.new(["ab".b, "cd".b]) + subject = body(native, cancellation: source.token) + chunks = [] + + assert_raises(Dexpace::CancelledError) do + subject.each do |chunk| + chunks << chunk + source.cancel(:between_chunks) + end + end + + assert_equal(["ab"], chunks) + assert_equal(1, native.close_count) + end + + test "is a Dexpace::Body that answers #source and #close, as anything in Response#body must" do + subject = body(AsyncHTTPRecordingBody.new(["x".b])) + + assert_kind_of(Dexpace::Body, subject) + assert_respond_to(subject, :source) + assert_respond_to(subject, :close) + refute_predicate(subject, :replayable?) + end + end +end diff --git a/gems/dexpace-transport-async_http/test/dexpace/transport/async_http/response_mapper_test.rb b/gems/dexpace-transport-async_http/test/dexpace/transport/async_http/response_mapper_test.rb new file mode 100644 index 0000000..5aee477 --- /dev/null +++ b/gems/dexpace-transport-async_http/test/dexpace/transport/async_http/response_mapper_test.rb @@ -0,0 +1,172 @@ +# frozen_string_literal: true +# SPDX-License-Identifier: MIT + +require_relative "../../../test_helper" +require_relative "../../../support/async_http_recording_body" +require_relative "../../../support/async_http_recording_sink" +require "dexpace/transport/async_http" + +# Dispatch step 15. TRANSPORT-14: obs-text in a value preserved, a control byte in a value +# dropped (that header only, logged verbose); the malformed-NAME half is unreachable on this +# adapter -- protocol-http1 raises BadHeader out of the read before a response exists to adapt +# (P8-38) -- so it is a NAMED WAIVER in the conformance driver, not a test here. TRANSPORT-24: +# the status mapping over the model's range. TRANSPORT-27: a malformed Content-Type downgrades to +# no media type; an absent native length maps to the -1 sentinel. The body handed on is the +# native BODY, never the response, whose #read is the whole body joined. Two nested classes under +# Metrics/ClassLength: the head, and the body. +module DexpaceTransportAsyncHTTPResponseMapperTest + # The native response, the request and the mapper call both classes share. + module ResponseMapperTestSupport + ResponseMapper = Dexpace::Transport::AsyncHTTP.const_get(:ResponseMapper, false) + ResponseBody = Dexpace::Transport::AsyncHTTP.const_get(:ResponseBody, false) + EVENT = Dexpace::Instrumentation::Events::TRANSPORT_HEADER_DROPPED + + def native_response(status: 200, headers: [], body: AsyncHTTPRecordingBody.new([], length: nil), + version: "HTTP/1.1") + ::Protocol::HTTP::Response.new(version, status, ::Protocol::HTTP::Headers.new(headers), body) + end + + def request(method: "GET") + builder = Dexpace::Request.builder + builder.method = method + builder.url = "https://example.test/" + builder.build + end + + def map(native, request: self.request, logger: Dexpace::Instrumentation::Logger::NULL, + head: false, on_release: nil) + ResponseMapper.call(native, request: request, logger: logger, head: head, + cancellation: Dexpace::Cancellation.none, on_release: on_release,) + end + end + + # The status and protocol (TRANSPORT-24) and the inbound header leniency (TRANSPORT-14). + class HeadTest < DexpaceTestCase + include ResponseMapperTestSupport + + test "TRANSPORT-24: a non-standard status inside the model's range maps, with a readable " \ + "body" do + native = native_response(status: 520, body: AsyncHTTPRecordingBody.new(["hi".b], length: 2)) + response = map(native) + + assert_equal(520, response.status.code) + assert_equal("http/1.1", response.protocol.wire) + assert_equal("hi", response.body_string) + end + + test "TRANSPORT-24: an HTTP/2 head maps to the model's http/2 protocol" do + response = map(native_response(version: "HTTP/2")) + + assert_equal("http/2", response.protocol.wire) + end + + # Phase 1's Status guards 100-599 and Protocol admits HTTP/1.1 and HTTP/2: a head outside either + # is InvalidArgumentError after the head, the disposition 8a records and phase 10 decides. + test "a status outside 100-599 or an HTTP/1.0 head raises InvalidArgumentError from the " \ + "model" do + assert_raises(Dexpace::InvalidArgumentError) { map(native_response(status: 999)) } + assert_raises(Dexpace::InvalidArgumentError) { map(native_response(version: "HTTP/1.0")) } + end + + test "TRANSPORT-14: a control byte in an inbound value is dropped, that header only, and " \ + "logged verbose by name" do + sink = AsyncHTTPRecordingSink.new + native = native_response(headers: [["x-ctl", "a\x01b"], ["x-normal", "n"]]) + response = map(native, logger: Dexpace::Instrumentation::Logger.build(sink: sink)) + + assert_nil(response.headers["x-ctl"]) + assert_equal(["n"], response.headers["x-normal"]) + assert_equal(%i[debug], sink.severities(EVENT)) + assert_equal("x-ctl", sink.events(EVENT).first.payload["header"]) + assert_equal("malformed inbound header (TRANSPORT-14)", + sink.events(EVENT).first.payload["reason"],) + end + + test "TRANSPORT-14: obs-text in an inbound value is preserved, and a repeated Set-Cookie " \ + "survives as two values" do + native = native_response(headers: [["x-obs", "caf\xE9".b], ["set-cookie", "a=1"], + ["set-cookie", "b=2"],]) + response = map(native) + + assert_equal(["caf\xE9".b], response.headers["x-obs"]) + assert_equal(["a=1", "b=2"], response.headers["set-cookie"]) + end + + test "TRANSPORT-14: an inbound name HeaderSyntax refuses is dropped rather than failing " \ + "the response, should one ever arrive" do + native = native_response(headers: [["x-b\xE9d".b, "y"], %w[x-ok 1]]) + response = map(native) + + assert_equal(["1"], response.headers["x-ok"]) + assert_equal(1, response.headers.size) + end + end + + # The media type and length (TRANSPORT-27), which native body is handed on, and the three shapes + # that carry none. + class BodyTest < DexpaceTestCase + include ResponseMapperTestSupport + + test "TRANSPORT-27: a malformed Content-Type downgrades to no media type rather than " \ + "failing the response, and the raw value still reaches the caller" do + native = native_response(headers: [["content-type", "not a/;;media type"]]) + response = map(native) + + assert_nil(response.body.media_type) + assert_equal(["not a/;;media type"], response.headers["content-type"]) + end + + test "a well-formed Content-Type reaches the body as its media type" do + native = native_response(headers: [["content-type", "text/plain; charset=utf-8"]]) + + assert_equal("text/plain; charset=utf-8", map(native).body.media_type.render) + end + + test "TRANSPORT-27: an absent native length maps to the -1 sentinel, a known one exactly" do + assert_equal(-1, map(native_response).body.content_length) + known = native_response(body: AsyncHTTPRecordingBody.new(["hi".b], length: 2)) + + assert_equal(2, map(known).body.content_length) + end + + test "the body handed on is a ResponseBody over the native BODY, never the response" do + native = native_response(body: AsyncHTTPRecordingBody.new(["hi".b], length: 2)) + response = map(native) + + assert_kind_of(ResponseBody, response.body) + assert_same(native.body, response.body.instance_variable_get(:@native)) + end + + test "a response the library delivers with no body -- a 204 -- gets body nil and the release " \ + "hook runs at once" do + released = 0 + response = map(native_response(status: 204, body: nil), on_release: -> { released += 1 }) + + assert_nil(response.body) + assert_equal(1, released) + end + + test "a HEAD response carries no body whatever the head says, and its native body is closed" do + native = AsyncHTTPRecordingBody.new([], length: 2) + response = map(native_response(body: native), request: request(method: "HEAD"), head: true) + + assert_nil(response.body) + assert_equal(1, native.close_count) + end + + test "a Protocol::HTTP::Body::Head native body is treated as no body" do + head = ::Protocol::HTTP::Body::Head.new(5) + response = map(native_response(body: head)) + + assert_nil(response.body) + end + + test "the HTTP/1.1 reason phrase is carried, and its absence over HTTP/2 is nil" do + with_reason = native_response + with_reason.define_singleton_method(:reason) { "Custom Reason" } + + assert_equal("Custom Reason", map(with_reason).reason) + assert_nil(map(native_response(version: "HTTP/2")).reason) + end + end +end diff --git a/gems/dexpace-transport-async_http/test/dexpace/transport/async_http/server_fixture_test.rb b/gems/dexpace-transport-async_http/test/dexpace/transport/async_http/server_fixture_test.rb new file mode 100644 index 0000000..fdcba19 --- /dev/null +++ b/gems/dexpace-transport-async_http/test/dexpace/transport/async_http/server_fixture_test.rb @@ -0,0 +1,89 @@ +# frozen_string_literal: true +# SPDX-License-Identifier: MIT + +require_relative "../../../test_helper" +require_relative "../../../support/async_http_server_fixture" +require "dexpace/transport/async_http" + +# The design's verified facts 2 and 3, as the fixture's own proof of concept before the wire +# grammar and dispatch suites build on it: a plaintext prior-knowledge h2 pair, a TLS pair +# negotiating h2 by real ALPN against a per-run self-signed certificate, and the HTTP/1.1 +# constructor over the same interface. No lib/ mirror: it proves a test double. The fixture +# writes nothing to stderr through Console -- no readiness probe of the wrong protocol -- which +# the adapter's own no-stderr assertion in dispatch_conformance_test.rb reads. +class DexpaceTransportAsyncHTTPServerFixtureTest < DexpaceTestCase + test "plaintext prior-knowledge h2 negotiates HTTP/2 with no client-side ALPN" do + Sync do + server = AsyncHTTPServerFixture.plaintext { |_request| [200, [], ["hi"]] } + client = ::Async::HTTP::Client.new(server.client_endpoint, retries: 0) + + response = client.get("/") + + assert_equal("HTTP/2", response.version) + assert_equal("hi", response.read) + ensure + client&.close + server&.close + end + end + + test "TLS negotiates h2 by ALPN against the fixture's self-signed certificate, generated " \ + "fresh for this run" do + Sync do + server = AsyncHTTPServerFixture.tls { |_request| [200, [], ["hi"]] } + client = ::Async::HTTP::Client.new(server.client_endpoint, retries: 0) + + response = client.get("/") + + assert_equal("HTTP/2", response.version) + assert_equal("hi", response.read) + assert_equal("https", server.client_endpoint.url.scheme) + ensure + client&.close + server&.close + end + end + + test "the http1 constructor serves HTTP/1.1 over the same interface, and records the headers" do + Sync do + server = AsyncHTTPServerFixture.http1 { |_request| [200, [], ["hi"]] } + client = ::Async::HTTP::Client.new(server.client_endpoint, retries: 0) + + response = client.get("/", { "x-probe" => "1" }) + + assert_equal("HTTP/1.1", response.version) + assert_equal("hi", response.read) + assert_includes(server.received.last, %w[x-probe 1]) + ensure + client&.close + server&.close + end + end + + test "each run generates its own certificate rather than a cached one" do + Sync do + a = AsyncHTTPServerFixture.tls { |_r| [200, [], []] } + b = AsyncHTTPServerFixture.tls { |_r| [200, [], []] } + + refute_equal(a.certificate.to_der, b.certificate.to_der) + ensure + a&.close + b&.close + end + end + + # Without the close, the accept task would keep the enclosing Sync block from returning. + test "#url names the fixture's own origin, and #close finishes the accept loop" do + Sync do |task| + server = AsyncHTTPServerFixture.http1 { |_r| [200, [], []] } + port = server.client_endpoint.url.port + + assert_equal("http://127.0.0.1:#{port}/x", server.url("/x")) + refute_predicate(server, :closed?) + server.close + task.yield + + assert_predicate(server, :closed?) + end + end +end diff --git a/gems/dexpace-transport-async_http/test/dexpace/transport/async_http/wire_grammar_test.rb b/gems/dexpace-transport-async_http/test/dexpace/transport/async_http/wire_grammar_test.rb new file mode 100644 index 0000000..1cd2f2c --- /dev/null +++ b/gems/dexpace-transport-async_http/test/dexpace/transport/async_http/wire_grammar_test.rb @@ -0,0 +1,173 @@ +# frozen_string_literal: true +# SPDX-License-Identifier: MIT + +require_relative "../../../test_helper" +require_relative "../../../support/async_http_server_fixture" +require_relative "../../../support/async_http_recording_sink" +require "dexpace/transport/async_http" + +# TRANSPORT-12/13 dispatched over BOTH protocols, and P8-40's reason: protocol-http1 refuses a +# non-token name AFTER the request line and host: are on the wire (a RefusedError, no 200) while +# protocol-http2 transmits it lowercased and unvalidated -- so a run over HTTP/1.1 alone could +# pass with the drop deleted (the library refusing instead), and only the h2 half proves the +# adapter's own predicate is doing anything on that protocol. The same fixture serves both, so +# "what did the peer receive" is read off the same kind of object on each. The wire-boundary +# re-validation's h2 half is here too: on HTTP/2 nothing below the model validates, and a CRLF +# value reaches the peer verbatim without it (the design's verified fact 4). Two nested classes +# under Metrics/ClassLength: the token predicate, and the re-validation with the spelling. +module DexpaceTransportAsyncHTTPWireGrammarTest + # The request builder and the borrowing adapter both classes share. + module WireGrammarTestSupport + AsyncHTTP = Dexpace::Transport::AsyncHTTP + DropPolicy = Dexpace::Transport::AsyncHTTP::DropPolicy + EVENT = Dexpace::Instrumentation::Events::TRANSPORT_HEADER_DROPPED + + def request(url, headers: {}) + builder = Dexpace::Request.builder + builder.url = url + headers.each { |name, value| builder.header(name, value) } + builder.build + end + + def borrowed(server, **) + AsyncHTTP.using(::Async::HTTP::Client.new(server.client_endpoint, retries: 0), **) + end + end + + # TRANSPORT-12/13 over three protocol shapes, the default policy's once-per-name warning, and + # the antecedent measured on protocol-http1. + class TokenPredicateTest < DexpaceTestCase + include WireGrammarTestSupport + + %i[http1 plaintext tls].each do |variant| + test "TRANSPORT-12/13, P8-40 over #{variant}: a non-token name is dropped, the normal " \ + "header and body still dispatch, and the future completes normally" do + Sync do + server = AsyncHTTPServerFixture.public_send(variant) { |_req| [200, [], ["ok"]] } + adapter = borrowed(server) + req = request(server.url, headers: { "X-Bad:Name" => "v", "X-Normal" => "n" }) + + response = adapter.call(req, nil, nil).value + + assert_equal(200, response.status.code) + assert_equal("ok", response.body_string) + names = server.received.last.map { |name, _| name.downcase } + + refute_includes(names, "x-bad:name") + assert_includes(names, "x-normal") + ensure + server&.close + end + end + end + + test "TRANSPORT-13: a real dispatch through the default policy warns once per distinct bad " \ + "name, then goes quiet" do + sink = AsyncHTTPRecordingSink.new + logger = Dexpace::Instrumentation::Logger.build(sink: sink) + + Sync do + server = AsyncHTTPServerFixture.plaintext { |_req| [200, [], []] } + adapter = borrowed(server, drop_policy: DropPolicy.build, logger: logger) + req = request(server.url, headers: { "X-Bad:Name" => "v" }) + + adapter.call(req, nil, nil).value.close + adapter.call(req, nil, nil).value.close + adapter.call(request(server.url, headers: { "y{bad}" => "v" }), nil, nil).value.close + + assert_equal(%i[warn debug warn], sink.severities(EVENT)) + ensure + server&.close + end + end + + # The antecedent on HTTP/1.1, measured through the real library so the drop's reason is a + # fact and not a citation: without the predicate, a model-valid name reaches protocol-http1, + # which refuses it after the request line is on the wire. + test "the antecedent: protocol-http1 refuses a model-valid non-token name that " \ + "protocol-http2 transmits" do + Sync do + h1 = AsyncHTTPServerFixture.http1 { |_req| [200, [], []] } + h2 = AsyncHTTPServerFixture.plaintext { |_req| [200, [], []] } + headers = ::Protocol::HTTP::Headers.new([["X-Bad:Name", "v"]]) + + refused = assert_raises(::Protocol::HTTP::RefusedError) do + ::Async::HTTP::Client.new(h1.client_endpoint, retries: 0).get("/", headers) + end + ::Async::HTTP::Client.new(h2.client_endpoint, retries: 0).get("/", headers).finish + + assert_kind_of(::Protocol::HTTP1::BadHeader, refused.cause) + assert_includes(h2.received.last.map(&:first), "x-bad:name") + ensure + h1&.close + h2&.close + end + end + end + + # The wire-boundary re-validation over HTTP/2, its antecedent, and a name's spelling on each + # protocol. + class RevalidationTest < DexpaceTestCase + include WireGrammarTestSupport + + # HTTP-17/HTTP-18/XCUT-18 over HTTP/2, where the re-validation is the ONLY defence: a forged + # request carrying a CRLF value is refused before any byte reaches the peer, and the fixture + # records nothing. + test "wire-boundary re-validation over HTTP/2: a forged CRLF header value never reaches the " \ + "peer" do + Sync do + server = AsyncHTTPServerFixture.plaintext { |_req| [200, [], []] } + adapter = borrowed(server) + template = request(server.url) + headers = Object.new + headers.define_singleton_method(:each_entry) do |&block| + block.call("x-inject", "a\r\nEvil: 1") + end + forged = Object.new + forged.define_singleton_method(:method) { template.method } + forged.define_singleton_method(:url) { template.url } + forged.define_singleton_method(:headers) { headers } + forged.define_singleton_method(:body) { nil } + + future = adapter.call(forged, nil, nil) + + assert_raises(Dexpace::InvalidArgumentError) { future.value } + assert_empty(server.received) + ensure + server&.close + end + end + + # The counter-measurement for the test above: protocol-http2 itself transmits the CRLF value. + test "the antecedent: protocol-http2 transmits a CRLF-bearing value verbatim" do + Sync do + server = AsyncHTTPServerFixture.plaintext { |_req| [200, [], []] } + headers = ::Protocol::HTTP::Headers.new([["x-inject", "a\r\nEvil: 1"]]) + + ::Async::HTTP::Client.new(server.client_endpoint, retries: 0).get("/", headers).finish + + assert_includes(server.received.last, ["x-inject", "a\r\nEvil: 1"]) + ensure + server&.close + end + end + + test "a caller's mixed-case name reaches an HTTP/1.1 peer as spelled and an HTTP/2 peer " \ + "lowercased -- the intra-adapter folding hazard the suite compares folded for" do + Sync do + h1 = AsyncHTTPServerFixture.http1 { |_req| [200, [], []] } + h2 = AsyncHTTPServerFixture.plaintext { |_req| [200, [], []] } + [h1, h2].each do |server| + borrowed(server).call(request(server.url, headers: { "X-MiXeD-CaSe" => "v" }), nil, nil) + .value.close + end + + assert_includes(h1.received.last.map(&:first), "X-MiXeD-CaSe") + assert_includes(h2.received.last.map(&:first), "x-mixed-case") + ensure + h1&.close + h2&.close + end + end + end +end diff --git a/gems/dexpace-transport-async_http/test/dexpace/transport/async_http_test.rb b/gems/dexpace-transport-async_http/test/dexpace/transport/async_http_test.rb index 145782a..bf9ebfb 100644 --- a/gems/dexpace-transport-async_http/test/dexpace/transport/async_http_test.rb +++ b/gems/dexpace-transport-async_http/test/dexpace/transport/async_http_test.rb @@ -1,17 +1,23 @@ # frozen_string_literal: true # SPDX-License-Identifier: MIT +require "open3" +require "prism" +require "rbconfig" require_relative "../../test_helper" # NFR-15: the version a published artifact reports at runtime is the real one. Styleguide 12.7: -# no constant outside Dexpace::. +# no constant outside Dexpace::. NFR-2: dexpace-core plus async-http and nothing else. P8-36: this +# gem's own Ruby floor. SEAM-5/SEAM-6: the require-time registration, proved in a bare child. class AsyncHTTPTest < DexpaceTestCase # The top-level namespace is snapshotted around the require, so "defines nothing outside # Dexpace" holds whether this file loads alone or after the other five gems in one - # `rake test:gems` process, where Dexpace already exists. The intermediate namespace is - # snapshotted the same way: a sibling this entry file placed beside its own constant -- - # `Dexpace::Transport::Shared` -- is outside the gem's namespace and inside nothing this suite - # would otherwise look at. + # `rake test:gems` process, where Dexpace already exists. `async/http` -- the one library this + # gem's lib/ requires beyond core's own, declared in the gemspec -- and `dexpace` itself are + # loaded first, the way core's smoke suite preloads `digest`: `Async`, `Protocol`, `Console`, + # `IO::Endpoint` and core's whole tree are theirs, not the entry file's. + require "async/http" + require "dexpace" TOP_LEVEL_BEFORE = Object.constants NAMESPACE_BEFORE = defined?(Dexpace) ? Dexpace.constants(false) : [] SIBLINGS_BEFORE = defined?(Dexpace::Transport) ? Dexpace::Transport.constants(false) : [] @@ -20,21 +26,127 @@ class AsyncHTTPTest < DexpaceTestCase NAMESPACE_ADDED = (Dexpace.constants(false) - NAMESPACE_BEFORE).freeze SIBLINGS_ADDED = (Dexpace::Transport.constants(false) - SIBLINGS_BEFORE).freeze + # The public surface phase 8c built into the skeleton: the transport, the policy, the six named + # constants and phase 0's VERSION. The eight private_constants are absent from constants(false) + # by definition. + PUBLIC = %i[ + VERSION Adapter DropPolicy DEFAULT_TIMEOUT_SECONDS DEFAULT_CONNECTION_LIMIT MAX_ORIGINS + REGISTRY_KEY FRAMING_HEADERS ALPN_PROTOCOLS + ].freeze + + GEM_ROOT = File.expand_path("../../..", __dir__) + private_constant :GEM_ROOT + test "defines a semver VERSION string" do assert_match(/\A\d+\.\d+\.\d+\z/, Dexpace::Transport::AsyncHTTP::VERSION) end test "the VERSION matches the gemspec this gem is built from" do - gemspec = File.expand_path("../../../dexpace-transport-async_http.gemspec", __dir__) - spec = Gem::Specification.load(gemspec) + spec = Gem::Specification.load(File.join(GEM_ROOT, "dexpace-transport-async_http.gemspec")) assert_equal(spec.version.to_s, Dexpace::Transport::AsyncHTTP::VERSION) end - test "defines nothing outside the Dexpace namespace" do + test "defines nothing outside the Dexpace namespace, and exactly its own constants inside it" do assert_empty(TOP_LEVEL_ADDED - [:Dexpace], "top-level constants added by the entry file") assert_empty(NAMESPACE_ADDED - %i[Transport], "constants added directly under Dexpace") assert_empty(SIBLINGS_ADDED - %i[AsyncHTTP], "constants added beside this gem's namespace") - assert_equal(%i[VERSION], Dexpace::Transport::AsyncHTTP.constants(false).sort) + assert_equal(PUBLIC.sort, Dexpace::Transport::AsyncHTTP.constants(false).sort) + end + + test "declares dexpace-core and async-http, and nothing else (NFR-2)" do + spec = Gem::Specification.load(File.join(GEM_ROOT, "dexpace-transport-async_http.gemspec")) + + assert_equal(%w[async-http dexpace-core], spec.runtime_dependencies.map(&:name).sort) + assert_equal(["~> 0.104"], spec.runtime_dependencies.find do |d| + d.name == "async-http" + end.requirement.as_list,) + end + + # P8-36: narrower than the repository's 3.2 floor, read from VERSIONS' own per-gem row rather + # than written twice (NFR-14). + test "P8-36: this gem's required_ruby_version is >= 3.3, VERSIONS' per-gem floor" do + spec = Gem::Specification.load(File.join(GEM_ROOT, "dexpace-transport-async_http.gemspec")) + + assert_equal(">= 3.3", spec.required_ruby_version.to_s) + assert_equal("3.3", DexpaceVersions.ruby_floor("dexpace-transport-async_http")) + assert_equal("3.2", DexpaceVersions.ruby_floor) + end + + # The registration is proved in a bare subprocess, where nothing else has required the gem: + # in one `rake test:gems` process the adapter's own suites load first and the key is already + # there, so an in-process before/after snapshot would prove nothing. The child clears RUBYOPT + # (bundler's -rbundler/setup) and inherits the parent's Gem.path instead, because async-http + # lives in a scoped BUNDLE_PATH on this machine and in the default GEM_HOME in CI. + test "registers itself under REGISTRY_KEY at require time; the registry resolves an adapter" do + key = Dexpace::Transport::AsyncHTTP::REGISTRY_KEY + + assert_includes(Dexpace::AsyncTransport.registered_keys, key) + assert_kind_of(Dexpace::Transport::AsyncHTTP::Adapter, Dexpace::AsyncTransport.resolve) + assert_equal("[:async_http]", bare_require("p Dexpace::AsyncTransport.registered_keys").strip) + end + + test "the registered factory builds a fresh owning adapter on every resolution (SEAM-5)" do + first = Dexpace::Transport::AsyncHTTP.default + second = Dexpace::Transport::AsyncHTTP.default + + refute_same(first, second) + assert_predicate(first, :owned?) + ensure + first&.close + second&.close + end + + # Inside `module Dexpace` a bare `Async` is core's own `Dexpace::Async` (the pivot's namespace) + # and a bare `Protocol` is phase 1's `Dexpace::Protocol`, so every reference to the socketry + # gems under lib/ is `::`-qualified. The phase-0 cop Dexpace/QualifiedCoreConstant names + # Thread, Queue, Mutex, SizedQueue, ConditionVariable, JSON and IO and not these -- listing + # `Async` there would flag core's legitimate uses -- so this scan is the guard, in the shape + # 5a's uuid_test.rb scans for SecureRandom: parsed, so a name in a comment or a message string + # is not a reference, and a bare constant read IS whatever the lexical scope resolves it to. + test "every Async, Protocol, OpenSSL and Console reference under lib/ is ::-qualified" do + offenders = Dir.glob(File.join(GEM_ROOT, "lib/**/*.rb")).flat_map do |path| + bare_constant_reads(Prism.parse_file(path).value).map do |node| + "#{File.basename(path)}:#{node.location.start_line}: #{node.name}" + end + end + + assert_empty(offenders) + end + + test "lib/ requires dexpace, async/http and openssl, and no transitive gem by name" do + requires = Dir.glob(File.join(GEM_ROOT, "lib/**/*.rb")).flat_map do |path| + File.read(path).scan(/^\s*require "([^"]+)"/).flatten + end + + assert_equal(%w[async/http dexpace openssl], requires.uniq.sort) + end + + LIBS = [File.expand_path("../../../lib", __dir__), + File.expand_path("../../../../dexpace-core/lib", __dir__),].freeze + private_constant :LIBS + + private + + FOREIGN = %i[Async Protocol OpenSSL Console].freeze + private_constant :FOREIGN + + # Every ConstantReadNode -- a constant resolved through the lexical scope, never through `::` + # -- whose name is one of the socketry gems' roots, anywhere in the tree. + def bare_constant_reads(node, found = []) + found << node if node.is_a?(Prism::ConstantReadNode) && FOREIGN.include?(node.name) + node.compact_child_nodes.each { |child| bare_constant_reads(child, found) } + found + end + + def bare_require(program) + command = [RbConfig.ruby, "-w", "-W:deprecated", *LIBS.flat_map { |lib| ["-I", lib] }, "-e", + "require \"dexpace/transport/async_http\"; #{program}",] + env = { "RUBYOPT" => nil, "GEM_PATH" => Gem.path.join(File::PATH_SEPARATOR) } + stdout, stderr, status = Open3.capture3(env, *command) + + assert_predicate(status, :success?, stderr) + assert_empty(stderr, "a bare require must be silent under -w") + stdout end end diff --git a/gems/dexpace-transport-async_http/test/support/async_http_hermetic_configuration.rb b/gems/dexpace-transport-async_http/test/support/async_http_hermetic_configuration.rb new file mode 100644 index 0000000..30b65e7 --- /dev/null +++ b/gems/dexpace-transport-async_http/test/support/async_http_hermetic_configuration.rb @@ -0,0 +1,21 @@ +# frozen_string_literal: true +# SPDX-License-Identifier: MIT + +require "dexpace" + +# A configuration chain that reads nothing off the host: `Dexpace::Configuration.build` defaults +# `env_source:` to `Sources::ENVIRONMENT`, the real process environment, so a test that asserts a +# DEFAULT through it -- the connection limit's 8, the timeout's 60 seconds -- would move under a +# developer's exported `TRANSPORT_CONNECTION_LIMIT` or `REQUEST_TIMEOUT` (review round 0's R0-1, +# measured moving under both). Every adapter test that asserts a value the chain resolves builds +# its configuration here, with the environment tier answering nothing, so the only tiers left are +# the overrides the test itself wrote and the default. Named for its gem (phase 8a's rule 35): +# `test:gems` loads every gem's test/support/ into one process. +module AsyncHTTPHermeticConfiguration + # @param overrides [Hash] the exact-name override map, the one tier above the default + # @return [Dexpace::Configuration] + def hermetic_configuration(overrides = {}) + Dexpace::Configuration.build(overrides: overrides, + env_source: Dexpace::Configuration::Sources::NONE,) + end +end diff --git a/gems/dexpace-transport-async_http/test/support/async_http_holding_server.rb b/gems/dexpace-transport-async_http/test/support/async_http_holding_server.rb new file mode 100644 index 0000000..59a6d6e --- /dev/null +++ b/gems/dexpace-transport-async_http/test/support/async_http_holding_server.rb @@ -0,0 +1,84 @@ +# frozen_string_literal: true +# SPDX-License-Identifier: MIT + +require "socket" + +# A raw TCPServer that decides when the client leaves a blocked state, so a cancellation test +# needs no sleep to "get into" it: it provokes the cancellation only after this server confirms +# (through `#wait_for_accept`) that it is holding the connection open. Two modes, one class: +# `hold: :head` writes a partial status line and headers and holds before the blank line, so the +# client is blocked waiting for the HEAD (the exchange task is in flight); `hold: :body` writes a +# complete chunked head and one chunk and holds before the next, so the head has been delivered +# and the CONSUMER is blocked in a body read. +# +# No Thread#kill anywhere: phase 0's Dexpace/NoThreadInterrupt cop is enabled repository-wide +# and a gem's own test/support/ file is scanned like any other. The handler thread is retired by +# closing what it is blocked on -- Thread::Queue#close wakes a blocked #pop with nil and +# TCPServer#close wakes a blocked #accept with IOError -- and joined with a bound, so +# DexpaceTestCase's per-test thread count is satisfied. Every response it writes is +# `Connection: close`, because the handler closes the socket after one exchange and a keep-alive +# connection the client's pool would reuse is the stale-connection race 8c's checklist records. +class AsyncHTTPHoldingServer + def initialize(hold: :head) + @hold = hold + @server = ::TCPServer.new("127.0.0.1", 0) + @gate = ::Thread::Queue.new + @accepted = ::Thread::Queue.new + @thread = ::Thread.new { serve } + end + + def port = @server.addr[1] + + # Blocks the calling thread or fiber until this server has accepted a connection and written + # what it writes before holding. Safe to call from inside a reactor: the pop is scheduler-aware + # there, and this server's thread is not the reactor's. + def wait_for_accept + @accepted.pop + end + + # Lets the held response finish with the given body (the second chunk, in body mode). + def release(body = "released") + @gate.push(body) + end + + def close + @gate.close + @server.close + @thread.join(2) + nil + end + + private + + def serve + socket = @server.accept + read_request(socket) + @hold == :body ? hold_body(socket) : hold_head(socket) + rescue ::IOError, ::Errno::EBADF, ::Errno::ECONNRESET, ::Errno::EPIPE, ::ClosedQueueError + nil + ensure + socket&.close + end + + def read_request(socket) + request = +"" + request << socket.readpartial(4096) until request.include?("\r\n\r\n") + end + + def hold_head(socket) + socket.write("HTTP/1.1 200 OK\r\nContent-Type: text/plain\r\nConnection: close\r\n") + socket.flush + @accepted.push(true) + body = @gate.pop # nil once #close has closed the queue: the fixture is shutting down + socket.write("Content-Length: #{body.bytesize}\r\n\r\n#{body}") if body + end + + def hold_body(socket) + socket.write("HTTP/1.1 200 OK\r\nContent-Type: text/plain\r\nTransfer-Encoding: chunked\r\n" \ + "Connection: close\r\n\r\n5\r\nfirst\r\n") + socket.flush + @accepted.push(true) + body = @gate.pop + socket.write("#{body.bytesize.to_s(16)}\r\n#{body}\r\n0\r\n\r\n") if body + end +end diff --git a/gems/dexpace-transport-async_http/test/support/async_http_reactor.rb b/gems/dexpace-transport-async_http/test/support/async_http_reactor.rb new file mode 100644 index 0000000..a0bfff0 --- /dev/null +++ b/gems/dexpace-transport-async_http/test/support/async_http_reactor.rb @@ -0,0 +1,58 @@ +# frozen_string_literal: true +# SPDX-License-Identifier: MIT + +require "async" + +# A reactor for a test that drives an exchange against a holding fixture (AsyncHTTPHoldingServer, +# AsyncHTTPSilentServer). The fixture is closed INSIDE the block's ensure -- before `Sync` waits +# for the reactor's children -- so an exchange a defective adapter left blocked on the fixture is +# released by the fixture's close and the failed assertion surfaces as a failure, where an ensure +# outside the `Sync` leaves the reactor waiting on that exchange forever. Two of the reviewer's +# mutations (the token hook cancelling the exchange directly; the per-call timeout removed) hung +# the suite rather than failing it until this existed (2026-09-21); the outer ensure a test keeps +# is still the fixture's release on every other path, and both closes are idempotent. +module AsyncHTTPReactor + # How many reactor turns a finished exchange and its watcher are given to be consumed from the + # task tree: the cancel a watcher delivers hands control to the exchange first and the watcher + # is resumed a turn later (measured: two turns on a cancellation, one on a timeout), while a + # watcher a defect never released stays for good. + RELEASE_TURNS = 10 + + # @param server [#close] the fixture to close before the reactor drains + # @yield [Async::Task] the reactor's root task + def reactor_over(server) + Sync do |task| + yield task + ensure + server.close + end + end + + # The watcher tasks alive under `task`, found by the adapter's own annotation: one per exchange + # whose queue is still open -- in flight, or delivered with its body not yet released. + # + # @param task [Async::Task] the task the exchange was spawned under + # @return [Array] + def watcher_tasks(task) + watcher = Dexpace::Transport::AsyncHTTP.const_get(:Exchange, false)::WATCHER_ANNOTATION + Array(task.children).select { |child| child.annotation == watcher } + end + + # After a settlement that delivered no response, or after a delivered body's release, the + # exchange task and its transient watcher are both gone from the reactor: a turn or two for + # them to finish, then no non-transient child remains and no child is a watcher. The pool's own + # transient gardener is the one child that legitimately stays for the client's life, which is + # why the watcher is found by its annotation rather than by counting. The watcher's release is + # what keeps a long-lived reactor from accumulating one parked task per failed exchange, and + # this is what turns its absence red. + # + # @param task [Async::Task] the task the exchange was spawned under + def assert_exchange_released(task) + RELEASE_TURNS.times do + task.yield + return if Array(task.children).all?(&:transient?) && watcher_tasks(task).empty? + end + + flunk("the exchange task or its watcher outlived the settlement by #{RELEASE_TURNS} turns") + end +end diff --git a/gems/dexpace-transport-async_http/test/support/async_http_recording_body.rb b/gems/dexpace-transport-async_http/test/support/async_http_recording_body.rb new file mode 100644 index 0000000..705a7c5 --- /dev/null +++ b/gems/dexpace-transport-async_http/test/support/async_http_recording_body.rb @@ -0,0 +1,45 @@ +# frozen_string_literal: true +# SPDX-License-Identifier: MIT + +require "async/http" + +# R13's counting double over Protocol::HTTP::Body::Readable, the native body the adapter's +# ResponseBody holds. Counting the close beats watching the socket: "the connection looked +# released" is not an assertion, and `close_count == 1` is. `#reads` counts every native `#read`, +# which is what ASYNC-21's one-read-per-demand property is asserted over. +# +# A top-level constant named for its gem (phase 8a's rule 35): `test:gems` loads every gem's +# suite into one process, and core's test/support/ already owns a bare `RecordingBody`. +class AsyncHTTPRecordingBody < Protocol::HTTP::Body::Readable + attr_reader :close_count, :close_errors, :reads, :length + + # @param chunks [Array] what successive `#read`s return before nil + # @param length [Integer, nil] the native length, nil for unknown + # @param raise_after [Integer, nil] a native failure raised on the read after this many chunks + def initialize(chunks, length: nil, raise_after: nil, error: ::EOFError.new("end of file")) + super() + @chunks = chunks.dup + @length = length + @raise_after = raise_after + @error = error + @close_count = 0 + @close_errors = [] + @reads = 0 + end + + # @return [Integer] how many chunks are still unread + def remaining = @chunks.size + + def read + @reads += 1 + raise @error if @raise_after && @reads > @raise_after + + @chunks.shift + end + + def close(error = nil) + @close_count += 1 + @close_errors << error + super + end +end diff --git a/gems/dexpace-transport-async_http/test/support/async_http_recording_sink.rb b/gems/dexpace-transport-async_http/test/support/async_http_recording_sink.rb new file mode 100644 index 0000000..7997ac0 --- /dev/null +++ b/gems/dexpace-transport-async_http/test/support/async_http_recording_sink.rb @@ -0,0 +1,53 @@ +# frozen_string_literal: true +# SPDX-License-Identifier: MIT + +# A logging sink that records every write, for the tests that assert what the adapter logs. The +# gem's own double: core's test support is not reachable from an adapter gem's suite, and a +# double this small is not worth a shared home. Every method the facade's duck-typed `_Sink` +# names, and nothing else -- the shape phase 8a's NetHTTPRecordingSink fixed. +# +# Named for its gem, never a bare `RecordingSink`: `test:gems` loads every gem's suite into ONE +# process (tools/suite_runner.rb, so SimpleCov reports one aggregate figure), and core's +# test/support/recording_sink.rb already defines a top-level `RecordingSink` with an `Entry` of its +# own -- a second definition re-assigns the constant, and that "already initialized constant" +# warning is fatal under NFR-6 at load time, before a single test runs. +class AsyncHTTPRecordingSink + Entry = ::Data.define(:severity, :payload) + + attr_reader :entries + + def initialize + @entries = [] + @mutex = ::Thread::Mutex.new + end + + def debug(message = nil, &) = record(:debug, message, &) + def info(message = nil, &) = record(:info, message, &) + def warn(message = nil, &) = record(:warn, message, &) + def error(message = nil, &) = record(:error, message, &) + + def debug? = true + def info? = true + def warn? = true + def error? = true + + # The records whose event field is the given name. + def events(name) + @mutex.synchronize do + @entries.select { |entry| entry.payload.is_a?(::Hash) && entry.payload["event"] == name } + end + end + + # The severities, in order, of every record under the given event name. + def severities(name) + events(name).map(&:severity) + end + + private + + def record(severity, message) + payload = block_given? ? yield : message + @mutex.synchronize { @entries << Entry.new(severity: severity, payload: payload) } + nil + end +end diff --git a/gems/dexpace-transport-async_http/test/support/async_http_server_fixture.rb b/gems/dexpace-transport-async_http/test/support/async_http_server_fixture.rb new file mode 100644 index 0000000..177771e --- /dev/null +++ b/gems/dexpace-transport-async_http/test/support/async_http_server_fixture.rb @@ -0,0 +1,155 @@ +# frozen_string_literal: true +# SPDX-License-Identifier: MIT + +require "async/http" +require "openssl" +require "socket" + +# An in-process async-http server over plaintext HTTP/1.1, plaintext prior-knowledge HTTP/2, and +# a self-signed TLS endpoint negotiating HTTP/2 by real ALPN -- the HTTP/2 driver the design's +# R16 says only this gem can supply, because 8a's TCPServer fixture speaks HTTP/1.1 bytes only. +# Must be constructed inside a running reactor (Sync/Async): it spawns its accept loop on +# Async::Task.current, exactly as the adapter's own exchange does. It lives in this gem's +# test/support/ and never in dexpace-conformance, which declares dexpace-core and nothing else. +# +# Readiness needs no probe: `Task#async` runs the child eagerly to its first suspension point, +# and Async::HTTP::Server#run binds the listener before it first suspends in `accept`, so the +# port is accepting when `#initialize` returns. A bare TCPSocket probe would have made the h2 +# server log a JSON "Invalid connection preface" warning to stderr through Console on every +# construction. The reactor starts no OS thread, so DexpaceTestCase's thread count is unmoved; +# `#close` cancels the accept task, without which the enclosing `Sync` block never returns. +# +# Generates its own certificate per run (the design's open question 6): an RSA-2048 keygen costs +# about 100 ms, cheap enough to pay every run and avoiding a cached artefact's own .gitignore +# and invalidation story. +class AsyncHTTPServerFixture + attr_reader :client_endpoint, :cert_store, :certificate, :received + + # async-http's server lets an EOFError out of a request read escape its per-connection task, + # which Console reports to stderr as a task failure. A peer that closes between the request + # line and the end of the headers -- an exchange cancelled mid-send, which several suites here + # do on purpose -- is exactly that, and it is the peer's doing and not the fixture's; seen + # once in a hundred-odd runs on 4.0.6 under load (2026-09-21), so it is swallowed rather than + # left as a rare line on stderr. A BadRequest is already ignored by the superclass. + class QuietServer < ::Async::HTTP::Server + def accept(...) + super + rescue ::EOFError + nil + end + end + + private_constant :QuietServer + + # Three constructors, one fixture; each takes the app as a block from the native request to + # `[status, headers, body]`, and records every request's header pairs in `#received`. + def self.http1(&) = new(tls: false, http2: false, &) + def self.plaintext(&) = new(tls: false, http2: true, &) + def self.tls(&) = new(tls: true, http2: true, &) + + def initialize(tls:, http2:, &handler) + port = free_port + @received = [] + @handler = handler + server_endpoint, @client_endpoint = endpoints(tls, http2, port) + @task = ::Async::Task.current.async do + QuietServer.new(method(:app), server_endpoint).run + end + end + + # @return [String] the fixture's origin, for a Dexpace::Request + def url(path = "/") + "#{@client_endpoint.url.scheme}://127.0.0.1:#{@client_endpoint.url.port}#{path}" + end + + # `#cancel`, never the deprecated `#stop`. The accept loop runs in a CHILD of the task the + # constructor spawned -- Async::HTTP::Server#run returns once its listeners are bound -- and a + # cancel on that completed parent still reaches the running child (measured on 2.46.0). + def close + @task&.cancel + nil + end + + # Whether the accept loop and every per-listener child have finished, which is when the + # listening socket is closed: a connect probe would race the kernel's backlog and see a reset. + def closed? + return true if @task.nil? + + @task.finished? && Array(@task.children).all?(&:finished?) + end + + private + + def app(request) + @received << request.headers.to_a + status, headers, body = @handler.call(request) + ::Protocol::HTTP::Response[status, headers, body] + end + + # Endpoint.new over a URI::RFC3986_PARSER-parsed URI, never Endpoint.parse (design §3.5). + def endpoints(tls, http2, port) + if tls + generate_certificate! + uri = ::URI::RFC3986_PARSER.parse("https://127.0.0.1:#{port}") + [::Async::HTTP::Endpoint.new(uri, ssl_context: server_ssl_context), + ::Async::HTTP::Endpoint.new(uri, ssl_context: client_ssl_context),] + else + uri = ::URI::RFC3986_PARSER.parse("http://127.0.0.1:#{port}") + options = http2 ? { protocol: ::Async::HTTP::Protocol::HTTP2 } : {} + endpoint = ::Async::HTTP::Endpoint.new(uri, **options) + [endpoint, endpoint] + end + end + + def free_port + probe = ::TCPServer.new("127.0.0.1", 0) + probe.addr[1] + ensure + probe&.close + end + + def generate_certificate! + key = ::OpenSSL::PKey::RSA.new(2048) + cert = certificate_for(key) + @key = key + @certificate = cert + @cert_store = ::OpenSSL::X509::Store.new + @cert_store.add_cert(cert) + end + + # A self-signed certificate for 127.0.0.1, valid from a minute ago for an hour. + def certificate_for(key) + name = ::OpenSSL::X509::Name.parse("/CN=127.0.0.1") + cert = ::OpenSSL::X509::Certificate.new + { version: 2, serial: ::OpenSSL::BN.rand(64), subject: name, issuer: name, + public_key: key.public_key, not_before: ::Time.now - 60, not_after: ::Time.now + 3600, } + .each { |attribute, value| cert.public_send(:"#{attribute}=", value) } + cert.add_extension(subject_alt_name(cert)) + cert.sign(key, ::OpenSSL::Digest.new("SHA256")) + cert + end + + def subject_alt_name(cert) + factory = ::OpenSSL::X509::ExtensionFactory.new + factory.subject_certificate = cert + factory.issuer_certificate = cert + factory.create_extension("subjectAltName", "DNS:localhost,IP:127.0.0.1") + end + + def server_ssl_context + context = ::OpenSSL::SSL::SSLContext.new + context.cert = @certificate + context.key = @key + context.alpn_select_cb = ->(protocols) { protocols.include?("h2") ? "h2" : protocols.first } + context + end + + # The client's context trusts the fixture's own certificate and offers h2 by ALPN, because a + # caller-supplied context is used verbatim by the adapter and async-http alike. + def client_ssl_context + context = ::OpenSSL::SSL::SSLContext.new + context.set_params(verify_mode: ::OpenSSL::SSL::VERIFY_PEER, cert_store: @cert_store) + context.alpn_protocols = %w[h2 http/1.1] + context + end +end diff --git a/gems/dexpace-transport-async_http/test/support/async_http_silent_server.rb b/gems/dexpace-transport-async_http/test/support/async_http_silent_server.rb new file mode 100644 index 0000000..51dd3b5 --- /dev/null +++ b/gems/dexpace-transport-async_http/test/support/async_http_silent_server.rb @@ -0,0 +1,43 @@ +# frozen_string_literal: true +# SPDX-License-Identifier: MIT + +require "socket" + +# Accepts, reads the request and writes NOTHING: the client blocks waiting for the status line, +# which is the phase TRANSPORT-8's pair needs to hold -- before `Client#call` has returned +# anything at all -- for both a parent task's cancellation and a deadline. Retired the way +# AsyncHTTPHoldingServer is: by closing the queue and the listener, never by an interrupt. +class AsyncHTTPSilentServer + def initialize + @server = ::TCPServer.new("127.0.0.1", 0) + @accepted = ::Thread::Queue.new + @gate = ::Thread::Queue.new + @thread = ::Thread.new { serve } + end + + def port = @server.addr[1] + + # Blocks until a connection was accepted and its request read -- the client is now waiting. + def wait_for_accept = @accepted.pop + + def close + @gate.close + @server.close + @thread.join(2) + nil + end + + private + + def serve + socket = @server.accept + request = +"" + request << socket.readpartial(4096) until request.include?("\r\n\r\n") + @accepted.push(true) + @gate.pop # parks until #close closes the queue + rescue ::IOError, ::Errno::EBADF, ::Errno::ECONNRESET, ::Errno::EPIPE, ::ClosedQueueError + nil + ensure + socket&.close + end +end diff --git a/gems/dexpace-transport-async_http/test/test_helper.rb b/gems/dexpace-transport-async_http/test/test_helper.rb index f67df15..75a0db8 100644 --- a/gems/dexpace-transport-async_http/test/test_helper.rb +++ b/gems/dexpace-transport-async_http/test/test_helper.rb @@ -7,3 +7,7 @@ # own test support, which is not the cross-gem require_relative styleguide 12.6 forbids -- that # rule is about reaching into another *gem's* internals. require_relative "../../../test/support/dexpace_test_case" +# The one IO::Buffer that spends Ruby 4.0's once-per-process experimental warning outside every +# test, before the test base's fatal-warning hook can see it (see the file). Not net_http_warmup: +# this gem starts no Net::HTTP, and the reactor starts no thread. +require_relative "../../../test/support/async_http_warmup" diff --git a/gems/dexpace-transport-net_http/test/dexpace/transport/net_http/conformance_test.rb b/gems/dexpace-transport-net_http/test/dexpace/transport/net_http/conformance_test.rb index a46f6d1..51c874b 100644 --- a/gems/dexpace-transport-net_http/test/dexpace/transport/net_http/conformance_test.rb +++ b/gems/dexpace-transport-net_http/test/dexpace/transport/net_http/conformance_test.rb @@ -9,8 +9,12 @@ # The shared conformance suite run against the real adapter (8a's R16, the suite contract): # one generated test per assertion, driven through MinitestDriver with no `settle:`, `around:` # or `wire:` -- the defaults ARE this adapter's shape -- and `waive: []`, because TRANSPORT-28's -# zero-copy clause has no assertion to suppress. TRANSPORT-18 reports vacuous (a skip in -# Minitest's vocabulary): with max_retries = 0 no native re-subscription exists to measure. +# zero-copy clause has no assertion to suppress. Three report vacuous BY MEASUREMENT (a skip in +# Minitest's vocabulary): TRANSPORT-18, because with max_retries = 0 no native re-subscription +# exists; and, since phase 8c appended its six portable assertions, TRANSPORT-12 and TRANSPORT-13, +# because net-http sends a model-valid non-token name rather than rejecting it -- the same +# antecedent this file's own cross-reference test measures absent. 8c's other four (TRANSPORT-7, +# 9, 21, 23) pass against this adapter as real properties of it. # The class inherits DexpaceTestCase so a fixture thread an assertion leaked fails the test that # leaked it, and includes NetHTTPHermeticProxy so the adapter every assertion builds resolves no # proxy from the host's environment. @@ -46,16 +50,18 @@ class DexpaceTransportNetHttpConformanceTest < DexpaceTestCase ) # The driver's own contract, checked rather than assumed: every assertion in the suite became - # a test method here, and the suite still carries no TRANSPORT-12 or TRANSPORT-13 row. - test "one generated test per assertion, and none for the two IDs the suite does not carry" do + # a test method here. The suite carries TRANSPORT-12 and TRANSPORT-13 since phase 8c, whose + # rows they are; against this adapter each resolves vacuous by measurement (the skips above). + test "one generated test per assertion, 8c's six included" do generated = public_methods(false).grep(/\Atest_/).reject do |name| name.to_s.start_with?("test_: ") end ids = Dexpace::Conformance::TransportSuite.assertions.flat_map(&:ids).uniq assert_equal(Dexpace::Conformance::TransportSuite.assertions.size, generated.size) - refute_includes(ids, "TRANSPORT-12") - refute_includes(ids, "TRANSPORT-13") + assert_equal(34, generated.size) + assert_includes(ids, "TRANSPORT-12") + assert_includes(ids, "TRANSPORT-13") end # What a real third-party adapter author would write: the borrow lambda sets max_retries = 0 diff --git a/gems/dexpace-transport-net_http/test/support/adapter_fixtures.rb b/gems/dexpace-transport-net_http/test/support/adapter_fixtures.rb index 43b5930..34f4a76 100644 --- a/gems/dexpace-transport-net_http/test/support/adapter_fixtures.rb +++ b/gems/dexpace-transport-net_http/test/support/adapter_fixtures.rb @@ -66,9 +66,12 @@ def trickle(count, delay) # Two responses on ONE connection: the first request was read by the fixture, the second is # read here, so a client that keeps its connection alive is told apart from one that does not. + # The first response says so (`close: false`): since phase 8c every scripted head announces + # `Connection: close` by default, because the fixture closes after one exchange and a pooling + # client re-used the closed connection otherwise; this is the one script that serves two. def keep_alive_twice(first, second) lambda do |conn, _head| - Dexpace::Conformance::Scripts.write_response(conn, body: first) + Dexpace::Conformance::Scripts.write_response(conn, body: first, close: false) head = (+"").b head << conn.readpartial(4096) until head.include?("\r\n\r\n") Dexpace::Conformance::Scripts.write_response(conn, body: second) diff --git a/rbs_collection.yaml b/rbs_collection.yaml index 63529e6..7812de5 100644 --- a/rbs_collection.yaml +++ b/rbs_collection.yaml @@ -29,3 +29,12 @@ gems: ignore: true - name: dexpace-conformance ignore: true + # Phase 8c. The collection's one entry for dexpace-transport-async_http's closure is + # `async/2.12`, which predates the primitives the adapter calls -- its Task declares `#stop` and + # neither `#cancel` nor `.current?` -- so installing it would type those calls as NoMethod and + # turn `steep` red. The walk does not reach it today only because the workspace gem above is + # ignored (rbs cuts the dependency walk at an ignored gem); this row keeps the stale signatures + # out however the walk is reached. async-http, protocol-*, async-pool, io-* and console have no + # collection entry and ship no sig/, so nothing else in the closure needs a row. + - name: async + ignore: true diff --git a/tasks/gates.rake b/tasks/gates.rake index 8cc5ebc..8b0cabf 100644 --- a/tasks/gates.rake +++ b/tasks/gates.rake @@ -180,6 +180,8 @@ namespace :gates do require "open3" require "tmpdir" + require_relative "../tools/versions" + root = gate_root override = ENV.fetch("DEXPACE_CLEAN_BUNDLE_GEM", nil) targets = @@ -189,6 +191,12 @@ namespace :gates do else CLEAN_BUNDLE_ENTRIES end + # A gem whose own floor this interpreter does not meet is skipped, not failed: Bundler refuses + # a path gem's required_ruby_version at install time, and that refusal would be a red row for + # a gem the row is not meant to build (phase 8c's P8-36). The closing count says how many ran. + targets = targets.select do |name, _| + DexpaceVersions.gem_supported?(name, RUBY_VERSION, File.join(root, "VERSIONS")) + end targets.each do |name, (entry, constant)| path = override || File.join(root, "gems", name) diff --git a/tasks/quality.rake b/tasks/quality.rake index 62af9ce..b6ca086 100644 --- a/tasks/quality.rake +++ b/tasks/quality.rake @@ -79,13 +79,20 @@ task :bundler_audit do end require_relative "../tools/suite_runner" +require_relative "../tools/versions" + +# The gems whose own VERSIONS floor this interpreter meets (phase 8c's P8-36): on the 3.2 row +# dexpace-transport-async_http is absent from the bundle, so its suite -- whose smoke test +# requires the gem in its class body -- cannot load into the one test:gems process there. +def supported_gem_dirs + Dir.glob("gems/*").select { |dir| DexpaceVersions.gem_supported?(File.basename(dir)) } +end namespace :test do desc "NFR-5/NFR-6/NFR-10: every gem's suite, warnings fatal, coverage floor enforced" task :gems do - SuiteRunner.run( - FileList["gems/*/test/**/*_test.rb"], Dir.glob("gems/*/lib") + %w[test], coverage: true, - ) + files = FileList[supported_gem_dirs.map { |dir| "#{dir}/test/**/*_test.rb" }] + SuiteRunner.run(files, Dir.glob("gems/*/lib") + %w[test], coverage: true) rescue SuiteRunner::Failure => error abort(error.message) end diff --git a/test/fixtures/gates/gemspec_audit/per_gem_floor_ahead/VERSIONS b/test/fixtures/gates/gemspec_audit/per_gem_floor_ahead/VERSIONS new file mode 100644 index 0000000..e9974ca --- /dev/null +++ b/test/fixtures/gates/gemspec_audit/per_gem_floor_ahead/VERSIONS @@ -0,0 +1,5 @@ +# Fixture: a miniature VERSIONS with one per-gem floor row (phase 8c's P8-36). +gem dexpace-core 0.0.0 +gem dexpace-transport-async_http 0.0.0 +ruby floor 3.2 +ruby floor:dexpace-transport-async_http 3.3 diff --git a/test/fixtures/gates/gemspec_audit/per_gem_floor_ahead/gems/dexpace-transport-async_http/dexpace-transport-async_http.gemspec b/test/fixtures/gates/gemspec_audit/per_gem_floor_ahead/gems/dexpace-transport-async_http/dexpace-transport-async_http.gemspec new file mode 100644 index 0000000..1196aac --- /dev/null +++ b/test/fixtures/gates/gemspec_audit/per_gem_floor_ahead/gems/dexpace-transport-async_http/dexpace-transport-async_http.gemspec @@ -0,0 +1,14 @@ +# frozen_string_literal: true +# SPDX-License-Identifier: MIT + +# Gate fixture: a gemspec that declares the global floor where VERSIONS gives the gem its own, +# narrower one -- the audit must read the per-gem row (phase 8c's P8-36). +Gem::Specification.new do |spec| + spec.name = "dexpace-transport-async_http" + spec.version = "0.0.0" + spec.authors = ["dexpace"] + spec.summary = "Gate fixture: a gemspec below its own per-gem Ruby floor." + spec.required_ruby_version = ">= 3.2" + spec.files = [] + spec.add_dependency "dexpace-core", "~> 0.0" +end diff --git a/test/fixtures/gates/versions/per_gem_floor_ahead/.github/workflows/ci.yml b/test/fixtures/gates/versions/per_gem_floor_ahead/.github/workflows/ci.yml new file mode 100644 index 0000000..83fde78 --- /dev/null +++ b/test/fixtures/gates/versions/per_gem_floor_ahead/.github/workflows/ci.yml @@ -0,0 +1,11 @@ +# Gate fixture: only the matrix is read. +name: CI +on: [push] +jobs: + test: + runs-on: ubuntu-latest + strategy: + matrix: + ruby: ["3.2", "3.3", "3.4", "4.0"] + steps: + - run: bundle exec rake test:gems diff --git a/test/fixtures/gates/versions/per_gem_floor_ahead/.ruby-version b/test/fixtures/gates/versions/per_gem_floor_ahead/.ruby-version new file mode 100644 index 0000000..d13e837 --- /dev/null +++ b/test/fixtures/gates/versions/per_gem_floor_ahead/.ruby-version @@ -0,0 +1 @@ +4.0.6 diff --git a/test/fixtures/gates/versions/per_gem_floor_ahead/VERSIONS b/test/fixtures/gates/versions/per_gem_floor_ahead/VERSIONS new file mode 100644 index 0000000..86a28ca --- /dev/null +++ b/test/fixtures/gates/versions/per_gem_floor_ahead/VERSIONS @@ -0,0 +1,6 @@ +# Gate fixture: a miniature VERSIONS carrying one per-gem floor row (phase 8c's P8-36). +gem dexpace-transport-async_http 0.0.0 +ruby floor 3.2 +ruby floor:dexpace-transport-async_http 3.3 +ruby matrix 3.2 3.3 3.4 4.0 +ruby dev 4.0.6 diff --git a/test/fixtures/gates/versions/per_gem_floor_ahead/gems/dexpace-transport-async_http/dexpace-transport-async_http.gemspec b/test/fixtures/gates/versions/per_gem_floor_ahead/gems/dexpace-transport-async_http/dexpace-transport-async_http.gemspec new file mode 100644 index 0000000..547f0c8 --- /dev/null +++ b/test/fixtures/gates/versions/per_gem_floor_ahead/gems/dexpace-transport-async_http/dexpace-transport-async_http.gemspec @@ -0,0 +1,13 @@ +# frozen_string_literal: true +# SPDX-License-Identifier: MIT + +# Gate fixture: the gem VERSIONS gives its own 3.3 floor, whose gemspec still declares the global +# 3.2 -- the one mismatch a gate reading only the global row cannot see (phase 8c's P8-36). +Gem::Specification.new do |spec| + spec.name = "dexpace-transport-async_http" + spec.version = "0.0.0" + spec.authors = ["dexpace"] + spec.summary = "Gate fixture: a gemspec below its own per-gem Ruby floor." + spec.required_ruby_version = ">= 3.2" + spec.files = [] +end diff --git a/test/fixtures/gates/versions/per_gem_floor_ahead/gems/dexpace-transport-async_http/lib/dexpace/transport/async_http/version.rb b/test/fixtures/gates/versions/per_gem_floor_ahead/gems/dexpace-transport-async_http/lib/dexpace/transport/async_http/version.rb new file mode 100644 index 0000000..e651196 --- /dev/null +++ b/test/fixtures/gates/versions/per_gem_floor_ahead/gems/dexpace-transport-async_http/lib/dexpace/transport/async_http/version.rb @@ -0,0 +1,11 @@ +# frozen_string_literal: true +# SPDX-License-Identifier: MIT + +module Dexpace + module Transport + # Gate fixture. + module AsyncHTTP + VERSION = "0.0.0" + end + end +end diff --git a/test/fixtures/surface/dexpace-core.txt b/test/fixtures/surface/dexpace-core.txt index 5ccc0f0..0cb841f 100644 --- a/test/fixtures/surface/dexpace-core.txt +++ b/test/fixtures/surface/dexpace-core.txt @@ -278,6 +278,7 @@ Dexpace::Configuration::Keys::MAX_RETRY_ATTEMPTS : String Dexpace::Configuration::Keys::MAX_TRACKED_CONTEXTS : String Dexpace::Configuration::Keys::NO_PROXY : String Dexpace::Configuration::Keys::REQUEST_TIMEOUT : String +Dexpace::Configuration::Keys::TRANSPORT_CONNECTION_LIMIT : String Dexpace::Configuration::Sources Dexpace::Configuration::Sources.from_hash Dexpace::Configuration::Sources::ENVIRONMENT : Proc diff --git a/test/fixtures/surface/dexpace-transport-async_http.txt b/test/fixtures/surface/dexpace-transport-async_http.txt index 01b720e..4dc49b9 100644 --- a/test/fixtures/surface/dexpace-transport-async_http.txt +++ b/test/fixtures/surface/dexpace-transport-async_http.txt @@ -1,2 +1,25 @@ Dexpace::Transport::AsyncHTTP +Dexpace::Transport::AsyncHTTP#build +Dexpace::Transport::AsyncHTTP#default +Dexpace::Transport::AsyncHTTP#using +Dexpace::Transport::AsyncHTTP::ALPN_PROTOCOLS : Array +Dexpace::Transport::AsyncHTTP::Adapter +Dexpace::Transport::AsyncHTTP::Adapter#call +Dexpace::Transport::AsyncHTTP::Adapter.borrowing +Dexpace::Transport::AsyncHTTP::Adapter.owning +Dexpace::Transport::AsyncHTTP::Adapter::REACTOR_MESSAGE : String +Dexpace::Transport::AsyncHTTP::DEFAULT_CONNECTION_LIMIT : Integer +Dexpace::Transport::AsyncHTTP::DEFAULT_TIMEOUT_SECONDS : Float +Dexpace::Transport::AsyncHTTP::DropPolicy +Dexpace::Transport::AsyncHTTP::DropPolicy#mode +Dexpace::Transport::AsyncHTTP::DropPolicy#report +Dexpace::Transport::AsyncHTTP::DropPolicy.build +Dexpace::Transport::AsyncHTTP::DropPolicy::EVERY : Symbol +Dexpace::Transport::AsyncHTTP::DropPolicy::MAX_TRACKED_NAMES : Integer +Dexpace::Transport::AsyncHTTP::DropPolicy::MODES : Array +Dexpace::Transport::AsyncHTTP::DropPolicy::ONCE_PER_NAME : Symbol +Dexpace::Transport::AsyncHTTP::DropPolicy::QUIET : Symbol +Dexpace::Transport::AsyncHTTP::FRAMING_HEADERS : Array +Dexpace::Transport::AsyncHTTP::MAX_ORIGINS : Integer +Dexpace::Transport::AsyncHTTP::REGISTRY_KEY : Symbol Dexpace::Transport::AsyncHTTP::VERSION : String diff --git a/test/gates/gemspec_audit_test.rb b/test/gates/gemspec_audit_test.rb index 832059a..82e4cea 100644 --- a/test/gates/gemspec_audit_test.rb +++ b/test/gates/gemspec_audit_test.rb @@ -37,6 +37,16 @@ class GemspecAuditTest < GateCase assert_includes(found.join("\n"), "expected >= 3.2 (NFR-10)") end + # Phase 8c's P8-36: the floor is per gem when VERSIONS carries a `floor:` row, so a + # gemspec on the global 3.2 where its own row says 3.3 is refused against 3.3, by name. + test "rejects a gemspec below its own per-gem Ruby floor" do + found = GemspecAudit.violations(File.join(FIXTURES, "per_gem_floor_ahead")) + + assert_equal(1, found.length, found.inspect) + assert_includes(found.first, "dexpace-transport-async_http") + assert_includes(found.first, "expected >= 3.3 (NFR-10)") + end + # NFR-12's ordering half, which the design lists among the audit's assertions. RubyGems sorts # spec.files in its own reader, so the unsorted fixture is the positive control: what the audit # refuses is the list that depends on git. diff --git a/test/gates/versions_gate_test.rb b/test/gates/versions_gate_test.rb index e5badd5..2571a35 100644 --- a/test/gates/versions_gate_test.rb +++ b/test/gates/versions_gate_test.rb @@ -41,8 +41,19 @@ class VersionsGateTest < GateCase assert_equal(["dexpace-core.gemspec did not load."], found) end + # Phase 8c's P8-36: a `floor:` row in VERSIONS is the floor THAT gem must declare, so a + # gemspec left on the global floor is a violation naming the gem -- which a gate reading only + # the global row would report as agreeing. + test "rejects a per-gem floor ahead of what the gemspec declares" do + found = VersionsGate.violations(File.join(FIXTURES, "per_gem_floor_ahead")) + + assert_includes(found.join("\n"), "dexpace-transport-async_http") + assert_includes(found.join("\n"), "expected >= 3.3") + end + test "each fixture is wrong in exactly one place" do - %w[stale_pin dropped_matrix_row ahead_literal gemspec_does_not_load].each do |fixture| + %w[stale_pin dropped_matrix_row ahead_literal gemspec_does_not_load per_gem_floor_ahead] + .each do |fixture| found = nil capture_io { found = VersionsGate.violations(File.join(FIXTURES, fixture)) } diff --git a/test/support/async_http_warmup.rb b/test/support/async_http_warmup.rb new file mode 100644 index 0000000..3253de1 --- /dev/null +++ b/test/support/async_http_warmup.rb @@ -0,0 +1,31 @@ +# frozen_string_literal: true +# SPDX-License-Identifier: MIT + +# Ruby 4.0's fiber-scheduler `io_read` hook hands io-event an `IO::Buffer`, and CRuby emits the +# once-per-process `warning: IO::Buffer is experimental and both the Ruby and C interface may +# change in the future!` the first time one is allocated -- with or without `-w`, because the +# experimental category is on by default -- so on the 4.0 row the FIRST `IO#read_nonblock` under +# an Async reactor warns, from inside the exchange's own fiber. Measured on 4.0.6 (any read_nonblock +# inside `Sync`, `:63`); 3.3.12 and 3.4.10 take a path that allocates none. Nothing +# in the SDK allocates one: it is the interpreter's own scheduler machinery. +# +# DexpaceTestCase turns every Warning.warn into an error, so the first test to send a request on +# 4.0.6 would fail with a warning it did not cause, wrapped as a transport failure. One buffer +# here, at this gem's test-helper load, with the experimental category off for exactly that +# allocation, spends the once-only warning silently and leaves `Warning[:experimental]` as it +# was -- the same category of arrangement as net_http_warmup.rb's Timeout thread (P8-62), and +# not the SDK's: the gem's lib/ sets no `Warning[]` and a consumer on 4.0 sees the interpreter's +# one warning line on their first request, which the gem's README says. +module AsyncHTTPWarmup + # @return [nil] + def self.run + was = Warning[:experimental] + Warning[:experimental] = false + ::IO::Buffer.new(1).free + nil + ensure + Warning[:experimental] = was + end +end + +AsyncHTTPWarmup.run diff --git a/tools/gemspec_audit.rb b/tools/gemspec_audit.rb index 4739b89..704c649 100644 --- a/tools/gemspec_audit.rb +++ b/tools/gemspec_audit.rb @@ -18,7 +18,6 @@ module GemspecAudit def violations(root) versions_path = File.join(root, "VERSIONS") expected_constraint = constraint_for(versions_path) - expected_floor = ">= #{DexpaceVersions.value("ruby", "floor", versions_path)}" Dir.glob(File.join(root, "gems/*/*.gemspec")).flat_map do |path| spec = Gem::Specification.load(path) @@ -26,6 +25,10 @@ def violations(root) # the finding should name the file rather than surface as a NoMethodError on nil. next ["#{path}: gemspec did not load."] if spec.nil? + # NFR-10's floor is per gem since phase 8c (P8-36): a gem with a `floor:` row in + # VERSIONS declares that one, every other gem the global one. + gem_name = File.basename(path, ".gemspec") + expected_floor = ">= #{DexpaceVersions.ruby_floor(gem_name, versions_path)}" check(spec, expected_constraint, expected_floor) end end diff --git a/tools/versions.rb b/tools/versions.rb index 37b0097..a02b83c 100644 --- a/tools/versions.rb +++ b/tools/versions.rb @@ -39,7 +39,25 @@ def value(kind, name, path = PATH) def gem_version(name) = value("gem", name) def gem_names = records.select { |kind, _, _| kind == "gem" }.map { |_, name, _| name } - def ruby_floor = value("ruby", "floor") + + # A gem's own floor when VERSIONS carries a `floor:` row, else the global floor (phase + # 8c's R15, P8-36). The per-gem row is colon-joined into the three-token `name` column so + # `records` and every existing `value` call site read it unchanged; a gem with no row is on the + # global floor, which is every gem but dexpace-transport-async_http. + def ruby_floor(gem_name = nil, path = PATH) + return value("ruby", "floor", path) if gem_name.nil? + + value("ruby", "floor:#{gem_name}", path) + rescue KeyError + value("ruby", "floor", path) + end + + # Whether the running interpreter satisfies a gem's own floor: the one question the root + # Gemfile, test:gems and gates:clean_bundle each ask before touching a gem on a matrix row. + def gem_supported?(gem_name, ruby_version = RUBY_VERSION, path = PATH) + Gem::Version.new(ruby_version) >= Gem::Version.new(ruby_floor(gem_name, path)) + end + def ruby_dev = value("ruby", "dev") def ruby_matrix = value("ruby", "matrix").split diff --git a/tools/versions_gate.rb b/tools/versions_gate.rb index 449e006..fabc1e9 100644 --- a/tools/versions_gate.rb +++ b/tools/versions_gate.rb @@ -36,12 +36,14 @@ def matrix_violations(root, versions) "#{expected.join(" ")}` (NFR-14, NFR-10)."] end + # The floor each gemspec must declare is the gem's OWN when VERSIONS carries a `floor:` + # row and the global one otherwise (phase 8c's P8-36); the gemspec reads the same row, so a + # disagreement is a gemspec that stopped reading VERSIONS. def gem_violations(root, versions) - floor = ">= #{DexpaceVersions.value("ruby", "floor", versions)}" - Dir.glob(File.join(root, "gems/*")).flat_map do |dir| name = File.basename(dir) declared = DexpaceVersions.value("gem", name, versions) + floor = ">= #{DexpaceVersions.ruby_floor(name, versions)}" spec = Gem::Specification.load(File.join(dir, "#{name}.gemspec")) next ["#{name}.gemspec did not load."] if spec.nil?