feat: merge apache kafka 3.9.2 - #3584
Merged
Merged
Conversation
…ing. (#19631) ### Motivation While investigating “events skipped in group rebalancing” ([spring‑projects/spring‑kafka#3703](spring-projects/spring-kafka#3703)) I discovered a race condition between - the main poll/commit thread, and - the consumer‑coordinator heartbeat thread. If the main thread enters `ConsumerCoordinator.sendOffsetCommitRequest()` while the heartbeat thread is finishing a rebalance (`SyncGroupResponseHandler.handle()`), the group state transitions in the following order: ``` COMPLETING_REBALANCE → (race window) → STABLE ``` Because we read the state twice without a lock: 1. `generationIfStable()` returns `null` (state still `COMPLETING_REBALANCE`), 2. the heartbeat thread flips the state to `STABLE`, 3. the main thread re‑checks with `rebalanceInProgress()` and wrongly decides that a rebalance is still active, 4. a spurious `CommitFailedException` is returned even though the commit could succeed. For more details, please refer to sequence diagram below. <img width="1494" alt="image" src="https://github.com/user-attachments/assets/90f19af5-5e2d-4566-aece-ef764df2d89c" /> ### Impact - The exception is semantically wrong: the consumer is in a stable group, but reports failure. - Frameworks and applications that rely on the semantics of `CommitFailedException` and `RetryableCommitException` (for example `Spring Kafka`) take the wrong code path, which can ultimately skip the events and break “at‑most‑once” guarantees. ### Fix We enlarge the synchronized block in `ConsumerCoordinator.sendOffsetCommitRequest()` so that the consumer group state is examined atomically with respect to the heartbeat thread: ### Jira https://issues.apache.org/jira/browse/KAFKA-19242 https: //github.com/spring-projects/spring-kafka/issues/3703 Signed-off-by: chickenchickenlove <ojt90902@naver.com> Reviewers: David Jacot <david.jacot@gmail.com>
Following release process guide: https://cwiki.apache.org/confluence/display/KAFKA/Release+Process#ReleaseProcess-Afterthevotepasses ``` Update version on the branch to 0.10.0.1-SNAPSHOT in the following places: docs/js/templateData.js gradle.properties kafka-merge-pr.py streams/quickstart/java/pom.xml streams/quickstart/java/src/main/resources/archetype-resources/pom.xml streams/quickstart/pom.xml tests/kafkatest/__init__.py (note: this version name can't follow the -SNAPSHOT convention due to python version naming restrictions, instead update it to 0.10.0.1.dev0) tests/kafkatest/version.py ``` Reviewers: Chia-Ping Tsai <chia7712@gmail.com>, TengYao Chi <frankvicky@apache.org>
At the retry limit binaryExponentialElectionBackoffMs it becomes statistically likely that the exponential backoff returned electionBackoffMaxMs. This is an issue as multiple replicas can get stuck starting elections at the same cadence. This change fixes that by added a random jitter to the max election backoff. Reviewers: José Armando García Sancio <jsancio@apache.org>, TaiJuWu <tjwu1217@gmail.com>, Yung <yungyung7654321@gmail.com>
Reviewers: Colin P. McCabe <cmccabe@apache.org>
```
Conflicts:
clients/src/main/java/org/apache/kafka/common/requests/AlterUserScramCredentialsRequest.java - import statement
clients/src/main/java/org/apache/kafka/common/requests/IncrementalAlterConfigsRequest.java - import statement
core/src/test/scala/unit/kafka/network/RequestChannelTest.scala - difference in unrelated parameter
core/src/test/scala/unit/kafka/server/KafkaApisTest.scala - different logging and metadatacache instantiation
```
Cherry-Picked-From: 042be5b9ac3f4be83069b18f17fda382e07d6539
Cherry-Picked-By: Alyssa Huang <ahuang@confluent.io>
Cherry-Picked-At: Mon May 12 11:11:19 2025 -0700
… (#19817) ## Summary - MetadataShell may deletes lock file unintentionally when it exists or fails to acquire lock. If there's running server, this causes unexpected result as below: * MetadataShell succeeds on 2nd run unexpectedly * Even worse, LogManager/RaftManager's lock also no longer work from concurrent Kafka process startup Reviewers: TengYao Chi <frankvicky@apache.org> # Conflicts: # shell/src/test/java/org/apache/kafka/shell/MetadataShellIntegrationTest.java
trunk PR: apache/kafka#19939 Bump the commons-beanutils for GHSA-wxr5-93ph-8wr9. Since commons-validator hasn't had new release with newer commons-beanutils versions, we manually bump it in kafka. Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
Trivial PR to replace the `rmr` command with the `delete` on for ZooKeeper znode deletions during KRaft migration. Signed-off-by: Paolo Patierno <ppatierno@live.com> Reviewers: Mickael Maison <mickael.maison@gmail.com>
…records per user op (#19974) If there are more deletion filters after we initially hit the `MAX_RECORDS_PER_USER_OP` bound, we will add an additional deletion record ontop of that for each additional filter. The current error message returned to the client is not useful either, adding logic so client doesn't just get `UNKNOWN_SERVER_EXCEPTION` with no details returned. Conflicts: - In AclControlManagerTest.java, use !isPresent instead of isEmpty so that this will work with java 8.
Previously, we could wait for up to half of the broker session timeout for an RPC to complete, and then delay by up to half of the broker session timeout. When taken together, these two delays could lead to brokers erroneously missing heartbeats. This change removes exponential backoff for heartbeats sent from the broker to the controller. The load caused by heartbeats is not heavy, and controllers can easily time out heartbeats when the queue length is too long. Additionally, we now set the maximum RPC time to the length of the broker period. This minimizes the impact of heavy load. Reviewers: José Armando García Sancio <jsancio@apache.org>, David Arthur <mumrah@gmail.com> Conflicts: - BrokerLifecycleManager.scala: fix minor conflict when removing ExponentialBackoff object
see apache/kafka#19756 (comment) Reviewers: Luke Chen <showuon@gmail.com>, TengYao Chi <frankvicky@apache.org>
We added back support for JoinGroup v0 & v1 in Kafka 4.0.1 and 4.1.0 due to KAFKA-19444. Given that, we should undeprecate these protocol api versions in 3.x. Reviewers: David Arthur <mumrah@gmail.com>
…(#20179) Update catch to handle compression errors Before :  After ``` Sent message: KR Message 376 [kafka-producer-network-thread | kr-kafka-producer] INFO org.apache.kafka.common.telemetry.internals.ClientTelemetryReporter - KR: Failed to compress telemetry payload for compression: zstd, sending uncompressed data Sent message: KR Message 377 ``` Reviewers: Apoorv Mittal <apoorvmittal10@gmail.com>, Bill Bejeck <bbejeck@gmail.com>, Chia-Ping Tsai <chia7712@gmail.com>
…ume pending writes (#20258) https://issues.apache.org/jira/browse/KAFKA-19561 Addresses a race condition during SASL reauthentication where the server-side `KafkaChannel.send()` queues a response, but OP_WRITE is removed before the channel becomes writable — resulting in stuck responses and client timeouts. Reviewers: Rajini Sivaram <rajinisivaram@googlemail.com>
…0272) All state updater threads use the same metrics instance, but do not use unique names for their sensors. This can have the following symptoms: 1) Data inserted into one sensor by one thread can affect the metrics of all state updater threads. 2) If one state updater thread is shutdown, the metrics associated to all state updater threads are removed. 3) If one state updater thread is started, while another one is removed, it can happen that a metric is registered with the `Metrics` instance, but not associated to any `Sensor` (because it is concurrently removed), which means that the metric will not be removed upon shutdown. If a thread with the same name later tries to register the same metric, we may run into a `java.lang.IllegalArgumentException: A metric named ... already exists`, as described in the ticket. This change fixes the bug giving unique names to the sensors. A test is added that there is no interference of the removal of sensors and metrics during shutdown. Reviewers: Matthias J. Sax <matthias@confluent.io>
The previous URL http://lambda-architecture.net/ seems to now be controlled by spammers Co-authored-by: Shashank <hsshashank.grad@gmail.com> Reviewers: Mickael Maison <mickael.maison@gmail.com>
…620) RoundRobinPartitioner does not handle the fact that on new batch creation, the partition method is called twice. Reviewers: Viktor Somogyi-Vass <viktorsomogyi@gmail.com>, Mickael Maison <mickael.maison@gmail.com>
…at INFO instead of ERROR (#20404) Lower the log level to INFO instead of ERROR. Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
Bump Commons-Lang for CVE-2025-48924. Signed-off-by: Federico Valeri <fedevaleri@gmail.com> Reviewers: Mickael Maison <mickael.maison@gmail.com>
…sing (#20435) **Changes:** - Remove outdated "missing features" references from docs - Updated ZooKeeper deprecation note to clarify removal in Kafka 4.0. Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
Docs change for [KAFKA-19668](https://issues.apache.org/jira/browse/KAFKA-19668) bug fix. Reviewers: Bill Bejeck <bill@confluent.io>
… loading transaction metadata (#20529) When loading transaction metadata from a transaction log partition, if the partition contains a segment ending with an empty batch, "currOffset" update logic at will be skipped for the last batch. Since "currOffset" is not advanced to next offset of last batch properly, TransactionStateManager.loadTransactionMetadata method will be stuck in the "while" loop. This change fixes the issue by updating "currOffset" after processing each batch, whether the batch is empty or not. --------- Co-authored-by: Vincent Jiang <84371940+vincent81jiang@users.noreply.github.com> Reviewers: Justine Olshan <jolshan@confluent.io>, Jun Rao <junrao@gmail.com>
…t unexpected fatal error (#20577) Cherry-pick changes (#20534) to 3.9 Conflicts: -> storage/src/main/java/org/apache/kafka/storage/internals/log/UnifiedLog.java - it is UnifiedLog.scala in 3.9 -> core/src/test/scala/unit/kafka/log/UnifiedLogTest.scala - had more changes than the 3.9 version, just added the test and kept everything else the same Reviewers: Justine Olshan <jolshan@confluent.io>, Chia-Ping Tsai <chia7712@gmail.com>
Backport KAFKA-19390 to v3.9, which includes PRs #19961 and #20131 Reviewers: Jun Rao <junrao@gmail.com>, Chia-Ping Tsai <chia7712@gmail.com>
- Test has been disabled for years + it tests ZK setup, but with KRaft there is no more a "controller", so cleaning it up. - Test has also been disabled for years + we did not get any incident about KS breaking during broker upgrade/downgrade
When using the zookeeper-security-migration tool without the '–enable.path.check' option, the script not only updates the ACLs for the existing znodes, but also creates any non-existing ones (with the ACL options specified) using null values based on the list defined in `ZkData.SecureRootPaths`. This is especially problematic for the /migration znode as the current logic only checks for the existence of the znode and later the migration process will hang when it tries to parse the null value over and over again. In summary, the migration cannot be completed if the zookeeper-security-migration script was run previously, and the only workaround is to manually remove the /migration znode in such cases. I propose a simple fix to circumvent the manual step by recreating the /migration znode if it contains a null value. --------- Co-authored-by: Gergely Harmadas <harmadasg@gmail.com>
…1 (#20624) backport KAFKA-19719 to 4.0 Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
Clear pendingTasksToInit on tasks clear. It matters in situations when we shutting down a thread in PARTITIONS_ASSIGNED state. In this case we may have locked some unassigned task directories (see TaskManager#tryToLockAllNonEmptyTaskDirectories). Then we may have gotten assigned to one or multiple of those tasks. In this scenario, we will not release the locks for the unassigned task directories (see TaskManager#releaseLockedUnassignedTaskDirectories), because TaskManager#allTasks includes pendingTasksToInit, but it hasn't been cleared. Reviewers: Matthias J. Sax <matthias@confluent.io>, Lucas Brutschy <lbrutschy@confluent.io>
Fix Java 11 vs Java 8 compilation issue. Reviewers: Matthias J. Sax <matthias@confluent.io>
see apache/kafka#20627 (comment) This pull request corrects a typo in the documentation by updating the JIRA link from KAFKA-19026 to KAFKA-19480 in the ops.html file. Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
…ges when the producer fails with MESSAGE_TOO_LARGE, violating delivery guarantees (#20285) Bug Fix in Producer where flush() does not wait for a batch to complete after splitting. Cf - apache/kafka#20254 (comment) and [KAFKA-19479](https://issues.apache.org/jira/browse/KAFKA-19479) for more details Reviewers: Jun Rao <junrao@gmail.com>
…1155) When a TimeoutException occurs while trying to put multiple active tasks back into running, we will add the timed out task back to the state updater, so that we retry it. However, if we run into a Task timeout (failing to make progress for a long time), we will rethrow a StreamsException wrapping the TimeoutException we have drained multiple tasks from the state updater, they will be lost, and not added back to the state updater, and therefore not be closed correctly. The task directories remain locked, causing issues trying to replace the stream thread. Reviewers: Matthias J. Sax <matthias@confluent.io>
…erIdsResponse (#21138) The handler in `RPCProducerIdManager` doesn't handle authentication exception and version mismatch exceptions gracefully which can result in an NPE in 3.9 and the broker crashing with a `FatalExitError`. This change ensures we retry on such failures and adds unit tests for these scenarios. Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
Convert HTML docs to Markdown for 3.9 Reviewers: David Arthur <mumrah@gmail.com>
This patch fixes a typo in `broker-configs.md`. Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
…… (#19263) …aft mode when using AlterConfigOp.OpType.SUBTRACT Modified ZkAdminManager.scala so that on OpType.SUBTRACT the policy receives the modified configs, as happens in KRaft mode. This similarly fixes the OpType.APPEND differences. Note that the policy behavior on OpType.DELETE is different when altering Broker and Topic resources. For topics the policy does not see a map entry, for brokers the config value is null. This was the existing behavior for KRaft and this commit does not change that. ClusterTest added.
mention the compatibility flag in the migration docs Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
…tion (#21137)
When preparing for the migration, there is a small chance that the
retrieved cluster id from ZooKeeper contains a leading dash. Currently
the 3.9 documentation specifies the following during the step
"Provisioning the KRaft controller quorum":
```
# Save the previously retrieved cluster ID from ZooKeeper in a variable called zk-cluster-id
$ bin/kafka-storage.sh format --standalone -t <zk-cluster-id> -c config/kraft/controller.properties
```
However the above command will fail with a less fortunate cluster id
like `-Yh6XyBIQ9CeAlMwJ8f7Hg`, eg.:
```
bin/kafka-storage.sh format --standalone -t -Yh6XyBIQ9CeAlMwJ8f7Hg -c config/server.properties
25/12/12 09:24:54 INFO utils.Log4jControllerRegistration$: [main]: Registered `kafka:type=kafka.Log4jController` MBean
usage: kafka-storage format [-h] --config CONFIG --cluster-id CLUSTER_ID [--add-scram ADD_SCRAM] [--ignore-formatted] [--release-version RELEASE_VERSION] [--feature FEATURE] [--standalone |
--no-initial-controllers | --initial-controllers INITIAL_CONTROLLERS]
kafka-storage: error: argument --cluster-id/-t: expected one argument
```
A similar problem came up with KRaft which was addressed in
[KAFKA-13741](https://issues.apache.org/jira/browse/KAFKA-13741). In our
case the cluster id is already specified, there is no way to change it.
I was able to come up with a simple solution by using the full name of
the argument like `--cluster-id=<zk-cluster-id>` instead of the
shorthand `-t <zk-cluster-id>`. Full example:
```
bin/kafka-storage.sh format --cluster-id=-Yh6XyBIQ9CeAlMwJ8f7Hg -c config/kraft/controller.properties
```
I would propose to change the documentation based on my findings so we
will be able to cover also ZooKeeper based cluster ids with a leading
dash.
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
This PR fix the broken `quickstart` link on the connector page. <img width="1094" height="604" alt="image" src="https://github.com/user-attachments/assets/d9a160b4-de09-422b-bd5a-c3bca6e73cac" /> Reviewers: Ken Huang <s7133700@gmail.com>, Chia-Ping Tsai <chia7712@gmail.com>
…255) Upgraded Jose4J from 0.9.4 to 0.9.6 to remediate CVE-2024-29371 Reviewers: Shicheng Rao <shichengrao@users.noreply.github.com>, Chia-Ping Tsai <chia7712@gmail.com>
…… (#21288) … on the client in certain error cases (#21065) Client versions 2.8.0 and later are affected by a [change](apache/kafka@30bc21c) that exposes a latent bug in how BufferPool is used (BufferPool is a class used on the client side to allocate memory in ByteBuffers, for performance it will reuse them with the caller of the class doing manual memory management by calling free when they are done with the memory). The bug is that a pooled ByteBuffer can be freed while it is still in use by the network sending thread - this early freeing can happen when batches expire / brokers are disconnecting from clients. This bug has existed for more than a decade (since Kafka 0.x it seems), but never manifested because prior to 2.8.0 the pooled ByteBuffer (which contained record data aka your publishes) was copied into a freshly allocated ByteBuffer before any potential reuse and that fresh ByteBuffer was what got written over the network to the broker. With a change included in 2.8.0, the pooled ByteBuffer remains as-is inside of a MemoryRecords instance and this pooled ByteBuffer (which in some cases can be reused and overwritten with other data) is written over the network. Two contributing factors are that the checksum for Kafka records only includes the key/value/headers/etc and not the topic so there is no protection there, and also an implementation detail is that, also newly in the commit that exposed the bug, the produce request header (which includes the topic and partition of a group of message batches) is serialized in a buffer separately from the messages themselves (and the latter is what gets put in the pooled ByteBuffer) which allows you to get messages misrouted to a random recently used topic as opposed to simple duplicate messages on their intended topic. The key change is in Sender.sendProducerData, we cannot allow the pooled ByteBuffer to be reused for expired in-flight batches until the request completes. For these batches we avoid deallocating the buffer in the normal failBatch call, deferring it until we call completeBatch (or a different path of failBatch). There are some automated tests to cover this, and also manual testing done to reproduce the issue from KAFKA-19012 and verify that this is sufficient to stop it. Reviewers: Justine Olshan <jolshan@confluent.io>, Jun Rao <junrao@gmail.com>, Chia-Ping Tsai <chia7712@gmail.com> Co-authored-by: Donny Nadolny <donny.nadolny@hotmail.com>
The `templateData.js` is removed by apache/kafka#21195. We should also remove related paths. Reviewers: Chia-Ping Tsai <chia7712@gmail.com> Signed-off-by: PoAn Yang <payang@apache.org>
Bump year to 2026 in NOTICE file. Reviewers: Lianet Magrans <lmagrans@confluent.io> (cherry picked from commit acd6fac6b19d97acc5f3f65863ef74fd652b9cc2)
3.9.2-rc0
Bump DEV_VERSION to `3.9.2` to fix `StreamsUpgradeTest. test_app_upgrade` e2e. Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
This PR upgrades `jersey` libraries family from 2.39.1 to 2.46 to address [CVE-2025-12383](GHSA-7p63-w6x9-6gr7) Note: while 2.39.1 is not listed as vulnerable - security scanners still may alert it as vulnerable Reviewers: PoAn Yang <payang@apache.org>, Gaurav Narula <gaurav_narula2@apple.com>, Chia-Ping Tsai <chia7712@gmail.com>
Bump NOTICE-binary copyright to 2026. Reviewers: Maros Orsak <maros.orsak159@gmail.com>, TengYao Chi <frankvicky@apache.org> Signed-off-by: PoAn Yang <payang@apache.org> (cherry picked from commit 36eb8184b0312412068e05dfd81934527dd9304b)
Signed-off-by: Robin Han <hanxvdovehx@gmail.com>
Align the ElasticLogSegment cleaning path with KAFKA-17076 by retaining the final empty batch at the cleaning-round upper bound. Apache Kafka PR: apache/kafka#20535 Original PR: apache/kafka#17193
superhx
enabled auto-merge
September 7, 2026 08:09
|
|
1sonofqiu
approved these changes
Sep 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.