From 35d0b449680267ae7e7addd733ab1874e6c5270a Mon Sep 17 00:00:00 2001 From: Michael Ernest Date: Tue, 17 Mar 2026 14:00:24 -0700 Subject: [PATCH 1/5] feat(DOC-2012): document Kafka 4.x client compatibility and KIP-890 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Explicitly state Kafka 4.x clients are compatible with Redpanda - Add KIP-890 (Transactions Server-Side Defense) to unsupported features: Redpanda does not implement the server-side portion; Kafka 4.x clients fall back to the standard transaction protocol when connecting to Redpanda - Add Apache Kafka® trademark on first mention - Remove "currently" from unsupported features intro - Fix "reach out" phrasing and mid-sentence line break Co-Authored-By: Claude Sonnet 4.6 --- modules/develop/pages/kafka-clients.adoc | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/modules/develop/pages/kafka-clients.adoc b/modules/develop/pages/kafka-clients.adoc index ba55a37e30..299d5c81d1 100644 --- a/modules/develop/pages/kafka-clients.adoc +++ b/modules/develop/pages/kafka-clients.adoc @@ -5,11 +5,11 @@ :description: Kafka clients, version 0.11 or later, are compatible with Redpanda. Validations and exceptions are listed. // tag::single-source[] -Redpanda is compatible with Apache Kafka versions 0.11 and later, with specific exceptions noted on this page. +Redpanda is compatible with Apache Kafka® versions 0.11 and later, with specific exceptions noted on this page. == Kafka client compatibility -Clients developed for Kafka versions 0.11 or later are compatible with Redpanda. Modern clients auto-negotiate protocol versions or use an earlier protocol version accepted by Redpanda brokers. +Clients developed for Kafka versions 0.11 or later are compatible with Redpanda, including Kafka 4.x clients. Modern clients auto-negotiate protocol versions or use an earlier protocol version accepted by Redpanda brokers. TIP: Redpanda Data recommends always using the latest supported version of a client. @@ -42,12 +42,11 @@ a| Clients that have not been validated by Redpanda Data, but use the Kafka protocol, remain compatible with Redpanda subject to the limitations below (particularly those based on librdkafka, such as confluent-kafka-dotnet or confluent-python). -If you find a client that is not -supported, reach out to the Redpanda team in the community https://redpanda.com/slack[Slack^]. +If you find an unsupported client, contact the Redpanda team in the community https://redpanda.com/slack[Slack^]. == Unsupported Kafka features -Redpanda does not currently support the following Apache Kafka features: +Redpanda does not support the following Kafka features: * Multiple SCRAM mechanisms simultaneously for SASL users; for example, a user having both a `SCRAM-SHA-256` and a `SCRAM-SHA-512` credential. Redpanda supports only one SASL/SCRAM mechanism per user, either `SCRAM-SHA-256` or `SCRAM-SHA-512`. + @@ -66,6 +65,8 @@ ifndef::env-cloud[] + * Quotas per user for bandwidth and API request rates. However, xref:manage:cluster-maintenance/manage-throughput.adoc#client-throughput-limits[quotas per client and per client group] using AlterClientQuotas and DescribeClientQuotas APIs are supported. endif::[] ++ +* https://cwiki.apache.org/confluence/display/KAFKA/KIP-890[KIP-890^] (Transactions Server-Side Defense): Redpanda does not implement the server-side portion of KIP-890. Kafka 4.x clients use per-transaction epoch bumping to protect against erroneous transaction commits, but fall back to the standard transaction protocol when connecting to Redpanda. If you have any issues while working with a Kafka tool, you can https://github.com/redpanda-data/redpanda/issues/new[file an issue^]. From 57b3339f0221a38bd0381dfc6ae16aacd6bfa865 Mon Sep 17 00:00:00 2001 From: Michael Ernest Date: Mon, 23 Mar 2026 18:42:11 -0700 Subject: [PATCH 2/5] style(DOC-2012): apply style guide audit to kafka-clients.adoc MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Add page-topic-type, personas, and learning objectives metadata - "Redpanda Data recommends" → direct imperative "Always use..." - "limitations below" → "limitations in the next section" - Fix "See the ... guide for details" → "For details, see xref:..." - "you can file an issue" → imperative construction - Break SCRAM bullet semicolon chain into two sentences - (pandaproxy) → monospace code term - "contact the Redpanda team in the community Slack" → cleaner phrasing Co-Authored-By: Claude Sonnet 4.6 --- modules/develop/pages/kafka-clients.adoc | 29 ++++++++++++++++-------- 1 file changed, 19 insertions(+), 10 deletions(-) diff --git a/modules/develop/pages/kafka-clients.adoc b/modules/develop/pages/kafka-clients.adoc index 299d5c81d1..6c4d76b7d3 100644 --- a/modules/develop/pages/kafka-clients.adoc +++ b/modules/develop/pages/kafka-clients.adoc @@ -3,15 +3,24 @@ :page-categories: Clients, Development, Kafka Compatibility :pp: {plus}{plus} :description: Kafka clients, version 0.11 or later, are compatible with Redpanda. Validations and exceptions are listed. +:page-topic-type: reference +:personas: developer +:learning-objective-1: Identify which Kafka clients are validated with Redpanda +:learning-objective-2: Identify unsupported Kafka features when integrating with Redpanda // tag::single-source[] Redpanda is compatible with Apache Kafka® versions 0.11 and later, with specific exceptions noted on this page. +Use this reference to: + +* [ ] {learning-objective-1} +* [ ] {learning-objective-2} + == Kafka client compatibility Clients developed for Kafka versions 0.11 or later are compatible with Redpanda, including Kafka 4.x clients. Modern clients auto-negotiate protocol versions or use an earlier protocol version accepted by Redpanda brokers. -TIP: Redpanda Data recommends always using the latest supported version of a client. +TIP: Always use the latest supported version of a Kafka client. The following clients have been validated with Redpanda. @@ -34,29 +43,29 @@ The following clients have been validated with Redpanda. | https://github.com/kafka-rust/kafka-rust[kafka-rust^] | Node.js -a| -* https://kafka.js.org[KafkaJS^] +a| +* https://kafka.js.org[KafkaJS^] * https://github.com/confluentinc/confluent-kafka-javascript[confluent-kafka-javascript^] |=== -Clients that have not been validated by Redpanda Data, but use the Kafka protocol, remain compatible with Redpanda subject to the limitations below (particularly those based on librdkafka, such as confluent-kafka-dotnet or confluent-python). +Clients that have not been validated by Redpanda Data, but use the Kafka protocol, remain compatible with Redpanda subject to the limitations in the next section (particularly those based on librdkafka, such as confluent-kafka-dotnet or confluent-python). -If you find an unsupported client, contact the Redpanda team in the community https://redpanda.com/slack[Slack^]. +If you find an unsupported client, reach out in the https://redpanda.com/slack[Redpanda community Slack^]. == Unsupported Kafka features Redpanda does not support the following Kafka features: -* Multiple SCRAM mechanisms simultaneously for SASL users; for example, a user having both a `SCRAM-SHA-256` and a `SCRAM-SHA-512` credential. Redpanda supports only one SASL/SCRAM mechanism per user, either `SCRAM-SHA-256` or `SCRAM-SHA-512`. +* Multiple SCRAM mechanisms simultaneously for SASL users. For example, a user cannot have both a `SCRAM-SHA-256` and a `SCRAM-SHA-512` credential. Redpanda supports only one SASL/SCRAM mechanism per user: either `SCRAM-SHA-256` or `SCRAM-SHA-512`. + ifndef::env-cloud[] -See the xref:manage:security/authentication.adoc#sasl[Configure Authentication] guide for details. +For details, see xref:manage:security/authentication.adoc#sasl[Configure SASL authentication]. endif::[] ifdef::env-cloud[] -See the xref:security:cloud-authentication.adoc[Authentication] guide for details. +For details, see xref:security:cloud-authentication.adoc[Authentication]. endif::[] -* HTTP Proxy (pandaproxy): Unlike other REST proxy implementations in the Kafka ecosystem, Redpanda HTTP Proxy does not support topic and ACLs CRUD through the HTTP Proxy. HTTP Proxy is designed for clients producing and consuming data that do not perform administrative functions. +* HTTP Proxy (`pandaproxy`): Unlike other REST proxy implementations in the Kafka ecosystem, Redpanda HTTP Proxy does not support topic and ACLs CRUD through the HTTP Proxy. HTTP Proxy is designed for clients producing and consuming data that do not perform administrative functions. ifdef::env-cloud[] + * The `delete.retention.ms` topic configuration in Kafka is not supported for Tiered Storage topics. Cloud Topics and local storage topics support Tombstone marker deletion using `delete.retention.ms`, but in Tiered Storage topics, Tombstone markers are only removed in accordance with normal topic retention, and only if the cleanup policy is `delete` or `compact, delete`. @@ -68,6 +77,6 @@ endif::[] + * https://cwiki.apache.org/confluence/display/KAFKA/KIP-890[KIP-890^] (Transactions Server-Side Defense): Redpanda does not implement the server-side portion of KIP-890. Kafka 4.x clients use per-transaction epoch bumping to protect against erroneous transaction commits, but fall back to the standard transaction protocol when connecting to Redpanda. -If you have any issues while working with a Kafka tool, you can https://github.com/redpanda-data/redpanda/issues/new[file an issue^]. +If you find an unsupported feature or incompatibility, https://github.com/redpanda-data/redpanda/issues/new[file an issue^] with the Redpanda team. // end::single-source[] From a7223de70e074081304a8ab6f67b233f439116b6 Mon Sep 17 00:00:00 2001 From: Michael Ernest Date: Tue, 24 Mar 2026 17:43:39 -0700 Subject: [PATCH 3/5] docs(DOC-2012): explain why KIP-890 is not needed in Redpanda Add context that KIP-890 addresses transaction errors specific to Kafka replication, and that Redpanda Raft-based replication is not susceptible to this issue. Addresses bashtanov review comment. Co-Authored-By: Claude Sonnet 4.6 --- modules/develop/pages/kafka-clients.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/develop/pages/kafka-clients.adoc b/modules/develop/pages/kafka-clients.adoc index 6c4d76b7d3..081260db1f 100644 --- a/modules/develop/pages/kafka-clients.adoc +++ b/modules/develop/pages/kafka-clients.adoc @@ -75,7 +75,7 @@ ifndef::env-cloud[] * Quotas per user for bandwidth and API request rates. However, xref:manage:cluster-maintenance/manage-throughput.adoc#client-throughput-limits[quotas per client and per client group] using AlterClientQuotas and DescribeClientQuotas APIs are supported. endif::[] + -* https://cwiki.apache.org/confluence/display/KAFKA/KIP-890[KIP-890^] (Transactions Server-Side Defense): Redpanda does not implement the server-side portion of KIP-890. Kafka 4.x clients use per-transaction epoch bumping to protect against erroneous transaction commits, but fall back to the standard transaction protocol when connecting to Redpanda. +* https://cwiki.apache.org/confluence/display/KAFKA/KIP-890[KIP-890^] (Transactions Server-Side Defense): Redpanda does not implement the server-side portion of KIP-890. KIP-890 addresses a class of transaction errors specific to Kafka's replication model. Redpanda's Raft-based replication is not susceptible to this issue. Kafka 4.x clients use per-transaction epoch bumping and fall back to the standard transaction protocol when connecting to Redpanda. If you find an unsupported feature or incompatibility, https://github.com/redpanda-data/redpanda/issues/new[file an issue^] with the Redpanda team. From 82808d80c084db3e45afd295813e34596cbdd2bb Mon Sep 17 00:00:00 2001 From: Michael Ernest Date: Thu, 26 Mar 2026 15:02:46 -0700 Subject: [PATCH 4/5] fix(DOC-2012): clarify KIP-890 fallback behavior per bashtanov review MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Kafka 4.x clients only use per-transaction epoch bumping when the server supports Transactions V2. Since Redpanda doesn't implement V2, clients fall back to the original transaction protocol — epoch bumping doesn't apply in this case. Co-Authored-By: Claude Sonnet 4.6 --- modules/develop/pages/kafka-clients.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/develop/pages/kafka-clients.adoc b/modules/develop/pages/kafka-clients.adoc index 081260db1f..6d27d63764 100644 --- a/modules/develop/pages/kafka-clients.adoc +++ b/modules/develop/pages/kafka-clients.adoc @@ -75,7 +75,7 @@ ifndef::env-cloud[] * Quotas per user for bandwidth and API request rates. However, xref:manage:cluster-maintenance/manage-throughput.adoc#client-throughput-limits[quotas per client and per client group] using AlterClientQuotas and DescribeClientQuotas APIs are supported. endif::[] + -* https://cwiki.apache.org/confluence/display/KAFKA/KIP-890[KIP-890^] (Transactions Server-Side Defense): Redpanda does not implement the server-side portion of KIP-890. KIP-890 addresses a class of transaction errors specific to Kafka's replication model. Redpanda's Raft-based replication is not susceptible to this issue. Kafka 4.x clients use per-transaction epoch bumping and fall back to the standard transaction protocol when connecting to Redpanda. +* https://cwiki.apache.org/confluence/display/KAFKA/KIP-890[KIP-890^] (Transactions Server-Side Defense): Redpanda does not implement the server-side portion of KIP-890. KIP-890 addresses a class of transaction errors specific to Kafka's replication model. Redpanda's Raft-based replication is not susceptible to this issue. When connecting to Redpanda, Kafka 4.x clients detect that Transactions V2 is not supported and fall back to the original transaction protocol. Per-transaction epoch bumping is part of Transactions V2 and does not apply in this case. If you find an unsupported feature or incompatibility, https://github.com/redpanda-data/redpanda/issues/new[file an issue^] with the Redpanda team. From 69535fdfe6fd3869781d848dd32f987d87a42e50 Mon Sep 17 00:00:00 2001 From: Joyce Fee Date: Wed, 27 May 2026 17:23:22 -0400 Subject: [PATCH 5/5] docs(DOC-2012): narrow 4.x claim, fix unvalidated wording, tighten KIP-890 - Scope the 4.x compatibility claim to the Apache Kafka Java client per @mattschumpert's review note; other clients in the validated list have not been broadly re-validated at 4.x. - Replace "unsupported client" with neutral phrasing to resolve the CodeRabbit terminology inconsistency vs the earlier "not validated" paragraph. - Tighten the KIP-890 bullet from five sentences to three; move the epoch-bumping detail into a parenthetical so the bullet doesn't dominate the unsupported-features list visually. Co-Authored-By: Claude Opus 4.7 --- modules/develop/pages/kafka-clients.adoc | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/modules/develop/pages/kafka-clients.adoc b/modules/develop/pages/kafka-clients.adoc index 6d27d63764..7df9e0b69f 100644 --- a/modules/develop/pages/kafka-clients.adoc +++ b/modules/develop/pages/kafka-clients.adoc @@ -18,7 +18,7 @@ Use this reference to: == Kafka client compatibility -Clients developed for Kafka versions 0.11 or later are compatible with Redpanda, including Kafka 4.x clients. Modern clients auto-negotiate protocol versions or use an earlier protocol version accepted by Redpanda brokers. +Clients developed for Kafka versions 0.11 or later are compatible with Redpanda. The Apache Kafka Java client at version 4.x is validated against Redpanda using the ducktape and chaos test suites; other clients in the table below have not been broadly re-validated at 4.x but remain compatible with the Kafka protocol versions they have always supported. Modern clients auto-negotiate protocol versions or use an earlier protocol version accepted by Redpanda brokers. TIP: Always use the latest supported version of a Kafka client. @@ -51,7 +51,7 @@ a| Clients that have not been validated by Redpanda Data, but use the Kafka protocol, remain compatible with Redpanda subject to the limitations in the next section (particularly those based on librdkafka, such as confluent-kafka-dotnet or confluent-python). -If you find an unsupported client, reach out in the https://redpanda.com/slack[Redpanda community Slack^]. +If you find a client that does not work with Redpanda, reach out in the https://redpanda.com/slack[Redpanda community Slack^]. == Unsupported Kafka features @@ -75,7 +75,7 @@ ifndef::env-cloud[] * Quotas per user for bandwidth and API request rates. However, xref:manage:cluster-maintenance/manage-throughput.adoc#client-throughput-limits[quotas per client and per client group] using AlterClientQuotas and DescribeClientQuotas APIs are supported. endif::[] + -* https://cwiki.apache.org/confluence/display/KAFKA/KIP-890[KIP-890^] (Transactions Server-Side Defense): Redpanda does not implement the server-side portion of KIP-890. KIP-890 addresses a class of transaction errors specific to Kafka's replication model. Redpanda's Raft-based replication is not susceptible to this issue. When connecting to Redpanda, Kafka 4.x clients detect that Transactions V2 is not supported and fall back to the original transaction protocol. Per-transaction epoch bumping is part of Transactions V2 and does not apply in this case. +* https://cwiki.apache.org/confluence/display/KAFKA/KIP-890[KIP-890^] (Transactions Server-Side Defense): Redpanda does not implement the server-side portion of KIP-890, which addresses transaction errors specific to Kafka's replication model. Redpanda's Raft-based replication is not susceptible to this class of errors. When connecting to Redpanda, Kafka 4.x clients detect that Transactions V2 is unsupported and fall back to the original transaction protocol (per-transaction epoch bumping is part of V2 and does not apply). If you find an unsupported feature or incompatibility, https://github.com/redpanda-data/redpanda/issues/new[file an issue^] with the Redpanda team.