Skip to content

Report read errors correctly - #479

Merged
mattjala merged 1 commit into
masterfrom
fix/streamed-read-errors
Oct 2, 2026
Merged

mattjala merged 1 commit into
masterfrom
fix/streamed-read-errors

Conversation

@mattjala

@mattjala mattjala commented Oct 2, 2026

Copy link
Copy Markdown
Collaborator

GET_Value and POST_Value sent the success response headers before fetching any data, then caught every error from the fetch so they could still finish the response. A read that failed on the data node therefore came back as:

  • a JSON read: 200 with an empty body
  • a binary read with a Content-Length: 200, then fewer bytes than promised, which clients report as a broken connection
  • a streamed read without a Content-Length (queries, variable-length types): 200 and a normally terminated body containing whatever pages were written before the failure. In testing, a query over a dataset with a bad chunk returned 2000 of its 4000 bytes as a success.

Both handlers now fetch before sending headers, and any errors there propagate. A failure on a later page can't change the status, so the handler raises instead of ending the stream, and aiohttp drops the connection so that the client sees an incomplete transfer instead of a complete-looking short one. JSON serialization errors also happen before the headers now, where they used to be swallowed the same way.

GET_Value and POST_Value sent the success response headers before
fetching any data, then caught every error from the fetch so they could
still finish the response. A read that failed on the data node therefore
came back as:

- a JSON read: 200 with an empty body
- a binary read with a Content-Length: 200, then fewer bytes than
  promised, which clients report as a broken connection
- a streamed read without a Content-Length (queries, variable-length
  types): 200 and a normally terminated body containing whatever pages
  were written before the failure. In testing, a query over a dataset
  with a bad chunk returned 2000 of its 4000 bytes as a success.

Both handlers now fetch before sending headers, and any errors there
propagate. A failure on a later page can't change the status, so the
handler raises instead of ending the stream, and aiohttp drops the
connection so that the client sees an incomplete transfer instead of a
complete-looking short one. JSON serialization errors also happen
before the headers now, where they used to be swallowed the same way.
@mattjala mattjala added this to the HSDS 1.1.0 milestone Oct 2, 2026
@mattjala mattjala added the bug label Oct 2, 2026
@mattjala mattjala changed the title Report read errors instead of returning 200 with a missing or short body Report read errors correctly Oct 2, 2026
@mattjala
mattjala merged commit 43627e2 into master Oct 2, 2026
49 checks passed
@mattjala
mattjala deleted the fix/streamed-read-errors branch October 2, 2026 17:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

Development

Successfully merging this pull request may close these issues.

1 participant