Repository navigation
Phase 8b: async-runtime adapter — documentation and phase record - #95
Merged
Merged
Conversation
Build the async-runtime adapter: Dexpace::Async::Thread::Pool, a fixed-size ::Thread pool over a bounded ::Thread::SizedQueue that is the first real implementation of SEAM-18's caller-supplied executor duck type and of Dexpace::Page::_Executor, settles phase 2's core-owned pivot from a worker thread, carries ASYNC-15..ASYNC-17's lifecycle over Dexpace::Closeable with SEAM-25's shutdown event, performs the ASYNC-8..ASYNC-12 diagnostic hop over Fiber[], and ships ASYNC-18's scheduled delay on one timer thread. Pool.build(size:, ...) creates exactly size workers and never grows; #post never blocks the caller (a full queue is RejectedError, a closed pool Dexpace::ClosedError, both translated from the queue's own errors at the one call site and routed to the failure channel by phase 2's bridge, P8-23); #delay answers a Dexpace::Async::Future settled with true, as 5a's Async.delay does, because Completer#fulfil(nil) raises (P8-71); #close closes the queue, stops the private Timer and fails its outstanding delays, drains the worker exit sentinels and joins the workers within one shutdown_timeout budget, then emits Events::INSTRUMENTATION_SHUTDOWN once (P8-24). A worker clears its inherited fiber storage at thread start and again after every task so Diagnostics.with installs the caller's snapshot rather than merging it onto the pool builder's or the previous task's (P8-20); both threads the pool owns rescue ::Exception and report the defect as a contained diagnostic instead of dying (P8-22, extended to the timer, whose #schedule after #stop refuses the entry, P8-75). The entry file asserts Dexpace::VERSION against REQUIRED_CORE at require time, because SEAM-18 leaves no executor registry to register into (P8-21, P8-72). Timer is a private_constant with a hooks.rbs-style sig mirror. The smoke suite preloads core before its snapshot and pins the four public constants; the surface manifest gains the object model's twelve rows and no private one. The gemspec is unchanged: dexpace-core and nothing else.
…ining Phase 8b, review round 0's R0-2 and R0-8. A Pool#close issued from a #delay future's on_settle handler -- which runs on the timer thread, as the README documents -- reached Timer#stop's thread.join on the current thread: ThreadError escaped #close with the latch already flipped, no shutdown event was emitted and every other outstanding delay stayed unsettled forever (measured on 4.0.6 and 3.2.11). Timer#stop now skips the join when the thread is the current one; the wake queue is closed regardless, so the next pop answers nil and #run exits as soon as the handler returns, and the leftovers are failed on the closing thread as on any other. The same self-wait, one thread over: a #close issued from inside a posted task waited for its own worker's exit sentinel, which cannot be pushed until #release returns, so every such close burned the whole shutdown budget and reported drained: false. drain_workers now counts the calling worker as exited and never joins it; that worker exits by itself once the task returns, draining whatever the closed queue still holds first. Recorded as ledger row P8-76 on the docs branch; the YARD on #delay and #release states both outcomes.
Phase 8b, review round 1's R1-1. Pool#delay validated a duration as Numeric and not negative, and a NaN answers false to both negative? and zero?: it reached the timer, whose list is ordered by deadline. The NaN deadline made next_wait's max raise inside the timer thread's net -- one diagnostic, the thread gone for good with @thread still set, the future never settled -- and every later #delay on that pool raised a bare ArgumentError synchronously from sort_by! (measured on 4.0.6 and 3.2.11). A Complex is a Numeric with no order and no #negative?, a bare NoMethodError from the same method. validate_duration! now asks real? before negative? and finite? after it, so a NaN, an infinite or a Complex duration raises Dexpace::InvalidArgumentError before anything is scheduled (ASYNC-18's "MUST reject", P8-77). finite? is Numeric's own protocol and covers a Float or BigDecimal NaN or infinity with one call, which is why Infinity is refused with NaN: an entry that never fires is not a delay. The same shape guarded shutdown_timeout: a NaN budget made #release's deadline arithmetic raise a bare ArgumentError out of #close with the latch already flipped, and an infinite one is the unbounded close XCUT-13 forbids for this gem by name, so non_negative_numeric! becomes finite_non_negative! (real, non-negative and finite), renamed in the sig.
Phase 8b, review round 2's R2-1. The timer thread is the gem's second ::Thread.new carrier, and it inherited the FIRST #delay caller's fiber storage at creation and was never cleared and never handed a per-delay snapshot: every later delay's #on_settle and #then callback ran under a stale, foreign context -- caller B's handler and a contextless caller C's both read caller A's trace id, and a key one handler wrote was visible to every later one (measured on 4.0.6 and 3.2.11). ASYNC-8 names callbacks explicitly, and the design's own finding 1 names this mechanism as ASYNC-10's stale assembly-time snapshot; only the workers had the floor P8-20 states. Timer#schedule now captures Diagnostics.capture per entry, on the scheduling caller's thread inside Pool#delay (ASYNC-10's per-submission point); the Entry carries the snapshot; the timer thread clears its inherited storage once at start and again after every fired callback, which runs under Diagnostics.with over the entry's snapshot (ASYNC-8, ASYNC-9) -- the shape Pool#run already has, applied to the second thread the gem owns (P8-78, extending P8-20). The shutdown callback runs on whichever thread stopped the timer under the same install-and-restore, so the closer's own context is put back and never cleared: that thread is not the timer's to empty. The sig declares the new member and the three private methods; no public surface changes.
The gem's one log event emitted on a caller's behalf after the hop -- P8-22's ERROR http.instrumentation.hook for a posted block or a delay handler that raised -- was emitted after Diagnostics.with had restored the thread, so on the worker and on the timer alike it carried no trace id (payload keys exactly [cause, event] on 4.0.6 and 3.2.11), while a line the block itself logged inside the hop carried the id. ASYNC-8's stated purpose is that events emitted after the hop retain correlation, and this was the one event the gem itself emits there (review round 3's R3-1). Pool#run now rescues inside the Diagnostics.with block, so report_failure runs with the task's snapshot still installed and the ensure clear stays at method level; Timer#fire and Timer#shut_down nest guarded inside Diagnostics.with rather than around it, so the timer's on_error runs under the delay caller's snapshot on the timer thread and on the closing thread. Nothing outside either net can raise -- .with's install and restore are Fiber[]= over the Symbol keys .capture read off a fiber's storage -- so both threads' survival stays structural. The sig files are unchanged.
Nine suites under test/dexpace/async/thread/ and five doubles under test/support/, every one a top-level constant unique across the six gems (PoolFakeTransport, CountingResponse, PoolRecordingSink, PoolStubClock, PoolProbeScheduler; PoolMatrixFactsTest, because core owns the bare name and rake test:gems loads every gem's suite into one process). pool_test.rb proves construction, the three source scans (two bounded spawn sites, none of the forbidden calls, every Ruby constant ::-qualified), #post's worker hop and non-blocking push, the worker net that survives exit, Interrupt, NoMemoryError and a raising sink, and #close's idempotent latch, drain, one event and spent budget off a stub clock; pool_diagnostics_test.rb the ASYNC-8..ASYNC-12 hop with the pool built in a different context from the one it is driven from, every worker-side read through Diagnostics.capture (the 3.2 floor retains nil-valued keys) and an ensure-clear proof over a key never held; pool_delay_test.rb ASYNC-18's four clauses, the entry-list observable a cancel must empty, the timer under close and its containment, and a two-fiber liveness check under a probe scheduler; bridge_test.rb ASYNC-1, 2, 5, 7, 13, 14, 15(b), 16, 17, 19, 20 and PIPE-33's four met clauses over a real worker, every window gated on the double's entered queue because phase 2's bridge checks the token before dispatch; page_executor_test.rb the first real Dexpace::Page::_Executor, bridged over an inline executor so only the paginator's executor: can move the consumer; pool_concurrency_test.rb the XCUT-11 evidence; and composed_transport_test.rb the charter's convergence point 2 — 8a's Dexpace::Transport::NetHTTP through Transport.async_over(adapter, executor: pool) against dexpace-conformance's WireServer, the one file in the gem that opens a socket and the one that requires test/support/net_http_warmup (P8-62). Twenty-eight guards run red on 4.0.6 and 3.2.11; every wait is bounded and no test sleeps.
Phase 8b, review round 0. Four guards the round found missing or wrong, each run red against its mutation on 4.0.6 and 3.2.11: - R0-2: "close from a delay's on_settle handler returns nil, emits once, fails the rest" -- close issued on the timer thread; with the self-join check removed the handler reports ThreadError. - R0-8: "close from inside a task returns before the budget, drained; the worker exits" -- with the calling worker counted and joined the outcome pop times out at a quarter of the budget. - R0-3: "close joins the workers after their sentinels, so none is alive when it returns" -- the exit queue is replaced by one whose push parks the worker after its sentinel is visible, so deleting the join leaves both workers provably alive when close returns (the GVL hid that gap from every plain thread count); the construction and 20-cycle tests gain the same postcondition. - R0-1: LockScopeTest, a cancel and a close issued from helper threads while the timer is provably parked, both with bounded joins -- the plan's Task 7 Step 8 mutation (the mutex held across the wait) is red by two reported failures here, where every other test in the file is red by a hang through teardown's close; the SchedulerTest comment that called that mutation green is corrected. pool_test.rb's three drain tests move into a DrainTest class under the class-length limit; shutdown_events moves to the base.
Phase 8b, review round 1. Three P8-77 cases in the delay suite: a NaN duration raises InvalidArgumentError before any timer thread exists and a later delay on the same pool still fires (under the previous validation nothing was raised, the timer thread died and every later delay raised a bare ArgumentError); an infinite duration is refused as non-finite and a negative infinity as negative; a Complex raises InvalidArgumentError where the previous check raised NoMethodError. One construction case pins shutdown_timeout refusing NaN, either infinity and a Complex by name. All four are red against the previous pool.rb on 4.0.6 and 3.2.11. The diagnostics suite's teardown now asserts the restore it performs -- the main fiber's compacted storage reads exactly as it did before setup -- which is the teardown-order assertion the design's testing strategy (group 4) names and the suite did not carry (review round 1's R1-3).
…pops Phase 8b, review round 2. R2-1: five cases in the diagnostics suite drive the timer thread as the gem's second carrier -- a first caller spawns it, later callers' #on_settle handlers read their own context and a contextless caller's reads empty; the first handler to fire does not see the spawning caller's keys (the construction floor, isolated from the reuse floor); a key a handler writes is invisible to the next handler; a #then continuation on a timer another request spawned reads the delay caller's context; and the entry #close fails settles under its caller's context on the closing thread, whose own context is put back after it. All five are red against the previous timer.rb on 4.0.6 and 3.2.11, and one each turns red under the four mutations of the fixed one (the thread-start clear removed, the after-callback clear removed, the callback run bare, the snapshot captured at Timer.new). The suite is reshaped into a test-less base holding setup, teardown and the readers, with WorkerContextTest and TimerContextTest nested beside each other -- the delay suite's shape -- so the timer cases inherit no worker case; one worker test's name is shortened to fit the column. R2-3: every queue pop in the file is bounded, so a mutant that never runs a task fails an assertion instead of tripping Ruby's deadlock detector. R2-2: the composed suite's one reason-less disable goes, the call it covered split across two statements.
Review round 3's R3-1: the P8-22 defect diagnostic is emitted under the
caller's context. pool_test.rb's ScriptError case posts under
{trace.id: DEFECT-7} and asserts the id on the hook payload;
pool_delay_test.rb's ContainmentTest schedules under DEFECT-8 (the
timer thread) and DEFECT-9 with the close issued under CLOSER (the
closing thread) and asserts each. Against the round-3 lib on 4.0.6 and
3.2.11 the first two read nil and the third reads "CLOSER" -- the
shutdown diagnostic carried the closer's correlation, not the delay
caller's; green after the nesting fix.
R3-2: every main-thread gate wait in bridge_test.rb, pool_test.rb,
composed_transport_test.rb, pool_concurrency_test.rb and
page_executor_test.rb is pop(timeout: 5) with refute_nil, the helper
threads' pops are bounded and their joins are join(5) with refute_nil,
as pool_diagnostics_test.rb has done since R2-3; the gate pops inside
task blocks are the tests' own controls and stay. A mutant that never
runs the worker now fails an assertion instead of ending in Ruby's
deadlock detector or hanging under the WireServer's accept thread.
… note The phase record for the async-runtime adapter, written from what was built: the checklist with one row per ID (seventeen ✅, ASYNC-3 ⏳ citing docs/first-release.md's unsatisfied-MUST entry, ASYNC-4 N/A on §10.5, the PIPE-33 and ASYNC-6 cross-reference rows and the rows execution added), the twenty-eight guards run red with their messages and rows, the audit groups, thirty-five deviations from the plan, the findings routed and the postponed-work mark; the design's As-built addendum, P8-71..P8-75 with P8-22 and P8-25 extended in place; docs/sdk-documentation/async-thread.md, the twentieth as-built page, every example run on 4.0.6 and 3.2.11 as one script; the gem README rewritten for a built gem with ASYNC-7's section; architecture.md, README.md and docs/README.md pointing at the page; CLAUDE.md's built-phases paragraph, skeleton clause, checklist count and constraints list re-derived from the tree; the roadmap's status note and two dated inbound bullets for phase 10; one dated status sentence in docs/first-release.md's unsatisfied-MUST entry; and the observability note's 8b entry amended in place under its own marker. docs/deviations.md is untouched, for phase 10 to flip; docs/sdk-design-ruby/ is frozen.
Phase 8b, review round 0. R0-1 (blocking): the design addendum, the checklist (deviation 13, "Findings routed"), the roadmap's inbound bullet and status note all recorded the plan's Task 7 Step 8 mutation -- the timer's mutex held across its queue wait -- as measured green. It is red: Timer#stop parks on the mutex the parked timer thread holds, #close never returns and the delay suite hangs in teardown, three runs in three with a thread dump; the plan's mechanism (a per-fiber ThreadError) is what does not occur, because the wait runs on the timer thread. The addendum paragraph is rewritten to say so, the checklist's guards table gains row 29 with the dump, the phase-10 bullet routed on the false premise is removed before it reaches main, and the status note says why. R0-2 and R0-8: ledger row P8-76 -- a close issued from the pool's own threads completes, neither the timer's stop nor the drain joining the thread it runs on -- with guards 30 and 31, the ASYNC-15, ASYNC-16, ASYNC-18, SEAM-25 and XCUT-11 rows updated, deviation 36, the README's lifecycle table and "Where callbacks run", the as-built page's #close section (a grace-period example, run on 4.0.6 and 3.2.11) and a CLAUDE.md constraint. R0-3: guard 32 and deviation 29 say the workers join is now pinned. R0-4: 115 runs in the gem, 3,809 in test:gems. R0-5: P8-75's identity-removal clause reworded as intent over an equivalent mutant. R0-6: the page declares e, sink, fake and strategy. R0-7: the README's ASYNC-7 contrast no longer states the reactor adapter's behaviour as shipped.
Record review round 1's repairs where each lives. The design's As-built addendum gains P8-77: #delay refuses a NaN, an infinite or a Complex duration beside a negative or non-Numeric one, and .build refuses the same shapes for shutdown_timeout, where the design's clause table and object model named only the first two checks -- a NaN answers false to both negative? and zero?, reached the timer's deadline-ordered list, killed its thread and turned every later #delay into a bare ArgumentError, and a NaN budget raised one out of #close with the latch flipped. The checklist gains the round's paragraph, the ASYNC-18 and XCUT-13 rows' clauses, guards 33-35 (the two validation mutations and the teardown-restore mutation, red on 4.0.6 and 3.2.11), deviations 37 and 38, the corrected ASYNC-2 test citation (R1-2), the renamed helper and the run counts (119 in the gem, 3,813 under test:gems). The teardown-order assertion the design's testing strategy names (R1-3) is recorded as built. Core's Async.validate_delay and Clock::Guard.duration share the hole and go to phase 10's inbound list by date and content. The gem README, the as-built page (the two new examples run on 4.0.6 and 3.2.11 as one script with the rest, 55 checks) and CLAUDE.md's constraint bullet state the widened contract.
The design's As-built addendum gains P8-78 -- the timer thread carries a delay's diagnostic context the way a worker carries a task's, where the timer sketch, R11 and the class comment gave the gem's second ::Thread.new carrier no context discipline and P8-20's floor was stated for the workers alone -- and P8-25's extension names the context a delay handler runs under. The checklist gains the round-2 paragraph, the timer half on the ASYNC-8, ASYNC-9, ASYNC-10 and ASYNC-12 rows, guards 36-40 (the four mechanisms each removed alone, and the round-2 timer.rb as a whole, all red on 4.0.6 and 3.2.11), deviations 39 and 40, the reshaped diagnostics suite and the run counts (124 in the gem, 3,818 under test:gems). The gem README's context and callbacks sections and the as-built page state where a delay callback's context comes from, the page with a three-delay example run on both rows (58 checks as one script); CLAUDE.md's two-boundary bullet names the lazily spawned carrier; the observability note's 8b entry is amended in place under its own sha with the rule restated for every ::Thread.new a library keeps; the roadmap's status note carries the round.
…41–43 Review round 3's should-fix (R3-1): the P8-22 defect diagnostic, the one log event the gem emits on a caller's behalf after the hop, was emitted after Diagnostics.with had restored the thread and so carried no trace id on the worker and the timer, and the closer's id on the closing thread. The checklist gains the round-3 paragraph, the correlation clause on its ASYNC-8 row, guards 41-43 (each nesting put back alone, each caught by the one assertion that guards it) plus the reviewer's mutation 6 re-run against the bounded suites, deviation 41 and the bounded-wait sentence (R3-2); the design's addendum extends P8-22 a second time rather than numbering a row; the gem README's "Where callbacks run", the as-built page, CLAUDE.md's P8-22 bullet and the roadmap's status note each state the diagnostic's correlation; the observability note's 8b entry is amended in place under its own marker with the corollary for a carrier's own emissions. No run count changes (124 in the gem, 3,818 under test:gems); no requirement row's mark moves.
Wahbeh-Mohammad
added
type:feature
Contributor
Author
Review record for the phase 8b stack (#93 → #94 → #95)5 independent reviews, each by a fresh agent with no memory of the previous one, each re-running every gate itself on every tip (4.0.6 every gate individually at the code tip and the full
Nothing was skipped. Round 0 → fixed in round 1
Round 1 → fixed in round 2
Round 2 → fixed in round 3
Round 3 → fixed in round 4
Round 4 (final) — approveWhat the final reviewer verified by experiment, both interpreters
Tips reviewed at the final round: code |
Wahbeh-Mohammad
changed the base branch from
31-phase-8b-async-runtime-adapter-tests
to
main
September 22, 2026 10:37
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #31. Third PR of phase 8b's stack — the phase record and the documentation — on top of #94. Umbrella #29 stays open: 8c remains.
What lands
11 files, +1,177 / −35.
docs/work/mvp/phase8/phase8b/2026-09-11-phase8b-async-runtime-adapter-checklist.md— nineteen own rows (seventeen ✅;ASYNC-3⏳ citingdocs/first-release.md's unsatisfied-MUST entry;ASYNC-4N/A citing §10.5) plus fifteen cross-reference rows (PIPE-33re-asserted ⏳ on its phase-4 row,ASYNC-6,SEAM-12,SEAM-18,SEAM-25,XCUT-11,XCUT-13,XCUT-22,OBS-20, …); the roadmap's legend verbatim; sections: requirement rows, what was built, matrix facts re-run on every interpreter, guards run red (43), audit groups run, deviations from the plan (41), findings routed, postponed work.docs/sdk-design-ruby/is frozen.docs/sdk-documentation/async-thread.md(new; every block executed on 4.0.6 and 3.2.11 — eleven blocks, 56 checks, extracted mechanically and run as one script by the final reviewer),architecture.md, the gem's README (with "Where callbacks run" — a#delayfuture'son_settleruns on the timer thread, or on the closing thread when#closefails it),README.md,docs/README.md(the twentieth page written).docs/knowledge/notes/observability.md— the 8b Reference entry filed on 2026-09-12 amended in place under its own manual sha with the as-built facts (both boundaries cleared, thecaptureread, the floor's retained-nil keys, the reserveddexpace.slots travelling with the snapshot); nothing duplicated.docs/first-release.md— the existing unsatisfied-MUST entry gains one dated status sentence (ASYNC-3⏳ andPIPE-33's interrupt clause re-asserted, as forecast;ASYNC-4N/A on §10.5 alone). Nothing else needed 8b.CLAUDE.md— the built-phases sentence gains 8b ("the second, built off the tree that holds 8a concurrently with 8c"); the opening paragraph gains the fifth real gem; the skeleton clause losesdexpace-async-thread(one skeleton remains); a "Constraints that will bite" line for the one-processtest:gemsrule extended to test class names (PoolMatrixFactsTest); nineteen checklists; the phase8/ sentence (8a and 8b hold checklists); counts unchanged where 8b adds no core file (two hundred and twenty lib files, nineteenprivate_constantexceptions).SEAM-25landed, its harness half phase 9's Task 11), and two dated phase-10 inbound bullets (the main fiber's storage residue under the one-process runner; the plan's Task 7 Step 8 premise).docs/product-spec/,docs/sdk-design-ruby/,docs/knowledge/harvested/,docs/deviations.mdand every other phase's documents — 8a's, 8c's and phase 7's included — are untouched. No file undergems/dexpace-core,gems/dexpace-transport-net_http,gems/dexpace-conformance,gems/dexpace-transport-async_http,tools/,tasks/,Steepfile,rbs_collection.yaml,VERSIONS,Gemfileor.githubchanged anywhere in the stack.Verification
Docs tip:
bundle exec rake(eighteen gates) green on 4.0.6;ruby .claude/skills/housekeeping/probe.rbexit 0 (all eight checks);ruby scripts/verify_knowledge_structure.rbOK; the housekeeping and knowledge tooling suites green;surface:regeneratewith no diff; the new page's blocks run on 4.0.6 and 3.2.11.Known follow-ups
The two routed inbound bullets and the recorded equivalent mutants — see the code PR's list.