fix: ensure response_buffer cleanup in RegularRequestHandler on cancellation

[algojogacor]

Summary

Fixes #452

When a regular (non-streaming) request is cancelled before receiving a response — for example, due to client disconnect or timeout — the corresponding uid entry in response_buffer persists indefinitely, causing unbounded memory growth.

Root Cause

RegularRequestHandler.handle_request (server.py, line 328–361) stores the uid in self.server.response_buffer[uid] at line 340, but only removes it on the success path at line 345 (response_buffer.pop(uid)). If await event.wait() at line 342 raises a CancelledError (or any other exception), the entry is never cleaned up because the handler has no finally block.

The StreamingRequestHandler (line 382–415) already avoids this issue via stream_with_cleanup() at line 404–411, which runs response_buffer.pop(uid, None) inside a finally block.

Fix

Added a finally block that conditionally removes the orphaned entry with pop(uid, None) (safe no-op if already popped). This matches the existing cleanup pattern used by StreamingRequestHandler.stream_with_cleanup() at lines 408–409.

The uid = None initialization ensures the finally block is a no-op when _submit_request itself fails before assigning uid.

Changes

• src/litserve/server.py: added finally block with safe buffer cleanup (+5 -0 lines)

Testing

• Happy path (client receives response normally): response_buffer is cleaned up via pop(uid) on success, then finally is a no-op ✅
• Client disconnects before response: CancelledError raised during event.wait(), finally block removes orphaned entry ✅
• Existing tests pass: tests/unit/test_request_handlers.py — all 3 tests pass unchanged ✅

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 85%. Comparing base (a69b635) to head (60de491).
⚠️ Report is 4 commits behind head on main.
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@         Coverage Diff         @@
##           main   #683   +/-   ##
===================================
  Coverage    85%    85%           
===================================
  Files        39     39           
  Lines      3282   3285    +3     
===================================
+ Hits       2778   2781    +3     
  Misses      504    504           

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.