[ https://issues.apache.org/jira/browse/KAFKA-18659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ismael Juma updated KAFKA-18659: -------------------------------- Description: A large percentage of non Java clients are unable to produce compressed records (with lz4, gzip, snappy) to Apache Kafka 4.0 due to a bug in how librdkafka evaluates whether compression is supported (https://github.com/confluentinc/librdkafka/issues/4956) - if the api versions response does not include produce v0, compression with these formats is disallowed (zstd checks a newer version of produce). To fix this, there are two options: # Update KIP-896 not to remove produce v0-v2 and KIP-724 not to remove support for producing messages with record format v0-v1. # Update api versions to return produce v0-v2 as supported, but reject requests with such versions with `UnsupportedVersion` error. The second option seems better to me, even though it's not ideal. was: A large percentage of non Java clients are unable to produce compressed records (with lz4, gzip, snappy) to Apache Kafka 4.0 due to a bug in how librdkafka evaluates whether compression is supported - if the api versions response does not include produce v0, compression with these formats is disallowed (zstd checks a newer version of produce). To fix this, there are two options: # Update KIP-896 not to remove produce v0-v2 and KIP-724 not to remove support for producing messages with record format v0-v1. # Update api versions to return produce v0-v2 as supported, but reject requests with such versions with `UnsupportedVersion` error. The second option seems better to me, even though it's not ideal. > librdkafka compressed produce fails unless api versions returns produce v0 > -------------------------------------------------------------------------- > > Key: KAFKA-18659 > URL: https://issues.apache.org/jira/browse/KAFKA-18659 > Project: Kafka > Issue Type: Sub-task > Reporter: Ismael Juma > Assignee: Ismael Juma > Priority: Blocker > Fix For: 4.0.0 > > > A large percentage of non Java clients are unable to produce compressed > records (with lz4, gzip, snappy) to Apache Kafka 4.0 due to a bug in how > librdkafka evaluates whether compression is supported > (https://github.com/confluentinc/librdkafka/issues/4956) - if the api > versions response does not include produce v0, compression with these formats > is disallowed (zstd checks a newer version of produce). > To fix this, there are two options: > # Update KIP-896 not to remove produce v0-v2 and KIP-724 not to remove > support for producing messages with record format v0-v1. > # Update api versions to return produce v0-v2 as supported, but reject > requests with such versions with `UnsupportedVersion` error. > The second option seems better to me, even though it's not ideal. -- This message was sent by Atlassian Jira (v8.20.10#820010)