fix: handle channel removal gracefully in processCompletedReceives - #3580
Open
VimalN2005 wants to merge 1 commit into
Open
fix: handle channel removal gracefully in processCompletedReceives#3580VimalN2005 wants to merge 1 commit into
VimalN2005 wants to merge 1 commit into
Conversation
…ceives (AutoMQ#2094) - Downgrade IllegalStateException to debug logging when channel is removed before processing completed receive - Safely close NetworkReceive to release buffers back to MemoryPool - Clean up channelContexts for disconnected connections - Add unit test testCompletedReceiveWithRemovedChannel
VimalN2005
requested review from
1sonofqiu,
Gezi-lzq and
superhx
as code owners
September 5, 2026 07:01
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #2094
Problem
When a client disconnects abruptly, times out, or closes during request processing:
Selector.processCompletedReceives.SocketServer.scala,openOrClosingChannel(receive.source)returnsNone, which previously threwIllegalStateException(s"Channel ${receive.source} removed from selector before processing completed receive").ERRORstack trace byprocessChannelException, flooding broker logs in production.receive.close()was never invoked on the orphaned receive, causing any allocated buffer inMemoryPoolto leak until GC.Solution
SocketServer.scala:processCompletedReceives(), whenopenOrClosingChannelreturnsNone, log atdebuglevel instead of throwingIllegalStateException.CoreUtils.swallow(receive.close(), this, Level.WARN)to safely releasereceive.payloadbuffer back toMemoryPool.channelContextsviaremoveChannelContext(receive.source).SocketServerTest.scala:testCompletedReceiveWithRemovedChannel()verifying that an orphaned completed receive whose channel was removed from selector is processed cleanly without throwing anIllegalStateExceptionor recording uncaught exceptions, keeping the processor healthy.Verification
./gradlew :core:test --tests "kafka.network.SocketServerTest.testCompletedReceiveWithRemovedChannel" -x checkstyleTest -x checkstyleMain -x spotbugsMain -x spotbugsTest