Skip to content

fix: wake a parked async_http body reader instead of closing under it - #107

Merged
Wahbeh-Mohammad merged 1 commit into
mainfrom
fix-ci-async-http-cancellation
Sep 25, 2026
Merged

Wahbeh-Mohammad merged 1 commit into
mainfrom
fix-ci-async-http-cancellation

Conversation

@Wahbeh-Mohammad

Copy link
Copy Markdown
Contributor

Root cause

main has been red on test (3.3), test (3.4) and test (4.0) since phase 8c landed, and green locally, because the local machines and the hosted runners run different io-event selectors, and they differ on exactly the property the async transport's TRANSPORT-7 body path relied on.

When a token is cancelled after the response is delivered, the exchange's watcher (on the reactor thread) closed the delivered response and relied on that close to wake a consumer parked in a body read. That works only by accident of the selector:

  • URing (io-event's choice wherever liburing was present at build time: the developer machines) notices a descriptor closed under a parked fiber, and the reader wakes with IOError: closed stream.
  • EPoll (every hosted runner; its io-event has no liburing, and the CI trace shows IO::Event::Selector::EPoll#transfer) silently drops a closed descriptor from its interest set, so the reader stays parked:
    • On 3.3 and 3.4 it stays parked until the test's own 5 s with_timeout fires. That is the Expected #<Async::TimeoutError> to not be a kind of Async::TimeoutError failure at cancellation_test.rb:274.
    • On the conformance row it stays parked until the driver's 30 s bound fires (minitest_driver.rb:72).
    • On Ruby 4.0, IO#close's own interrupt of the parked fiber is deferred through Fiber::Scheduler#fiber_interrupt. It lands later, at whatever suspension point that fiber reaches next: here AsyncHTTPHoldingServer#close's Thread#join, which gives IOError: stream closed in another thread.

The defect is in the adapter, not in the test: a caller's reader on EPoll hangs until its own deadline.

Evidence

  • Reproduced locally with IO_EVENT_SELECTOR=EPoll:
    • 3.4.10: failed 5 of 5 (timeout at :274).
    • 4.0.6: failed 3 of 3 (IOError: stream closed in another thread from the fixture's join). These are the exact CI signatures.
  • With the old watcher and the default selector (URing), the new EPoll-named test fails the same way on a developer box.
  • The conformance mid-body cancel failed 3 of 6 on 3.4.10 under EPoll before the fix. The race is inherent to that row, as its own comment says.
  • Minimal probe on 3.4.10 with a plain socket pair: a fiber parked in readpartial is not woken by another fiber's close under EPoll (it times out), but is woken under URing.

Fix

ResponseBody now records the fiber parked in its native read, and the scheduler that fiber is parked under. The watcher calls ResponseBody#cancel_read, which does one of two things:

  • A reader is parked on this scheduler: it raises an IOError into that fiber through Fiber.scheduler.raise. This is a checkpoint interrupt, the mechanism Async::Task#cancel and #with_timeout use. The fiber is provably suspended inside #pull's own read at that point. The reader unwinds its wait, closes the body on its own fiber, and surfaces CancelledError through the existing token-first classifier.
  • No reader is parked, or it is on another thread or reactor: the body is closed as before.

Nothing is ever closed under a parked fiber, so neither EPoll's silent drop nor 4.0's deferred interrupt can occur. No Timeout.timeout, Thread#raise or Thread#kill is involved.

Tests

  • response_body_test.rb CancelReadTest uses a native body whose close never wakes its reader, which models EPoll independent of the host. It pins the wake on both the #each path and the #source enumerator-fiber path (Response#body_string's route). It also pins that the close runs after the read unwound, not under it. Mutation check: with #cancel_read reduced to a plain close, both tests fail on their bound instead of hanging.
  • cancellation_test.rb runs the TRANSPORT-7 body-path test once per selector the host has (AsyncHTTPReactor::SELECTORS, via a new reactor_on(selector)), so the EPoll case runs on an io_uring machine too.

Verification

  • 200 runs of the whole cancellation_test.rb file, each under IO_EVENT_SELECTOR=EPoll, taskset -c 0 and a CPU hog on the same core, with varied seeds: 0 failures on each of 3.3.12, 3.4.10 and 4.0.6.
  • Full async_http gem suite under EPoll and under the default selector: 198 runs green on each of 3.3.12, 3.4.10 and 4.0.6.
  • The async_http conformance suite under EPoll, 30 runs per Ruby: 0 failures.
  • bundle exec rake (all 24 gates) on 4.0.6: exit 0. Honest RuboCop is clean.
  • Hosted Ruby 4.0.7 was not installed locally; CI's 4.0 row is the proof there.

TRANSPORT-7's body path: a token cancelled after delivery reaches a
consumer blocked in a body read through the exchange's watcher. The
watcher closed the delivered response and relied on that close to
wake the parked reader. Whether it does depends on io-event's
selector: io_uring's poll sees a descriptor closed under a parked
fiber, epoll silently drops it from its interest set. Developer
machines with liburing run URing and stayed green; hosted CI runners
have no liburing, run EPoll, and the reader stayed parked until the
test's own 5 s bound fired (3.3, 3.4) or, on Ruby 4.0, IO#close's
deferred fiber interrupt landed later in the fixture's Thread#join.
The conformance mid-body cancel hung for the same reason.

ResponseBody now records the fiber parked in its native read, and
the watcher's #cancel_read raises an IOError into that fiber through
the scheduler -- a checkpoint interrupt, as Task#cancel and
with_timeout deliver -- so the reader unwinds its wait, closes the
body on its own fiber and surfaces CancelledError through the
token-first classifier. With no reader parked, or one on another
thread or reactor, the body is closed as before.

Pinned by CancelReadTest over a native body whose close never wakes
its reader (the #each and #source paths), and by the body-path
cancellation test run on every selector the host has, so the EPoll
case is exercised on an io_uring machine too.
@Wahbeh-Mohammad
Wahbeh-Mohammad merged commit 8501687 into main Sep 25, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant