[
https://issues.apache.org/jira/browse/KAFKA-18659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ismael Juma updated KAFKA-18659:
--------------------------------
Description:
A large percentage of non Java clients are unable to produce compressed records
(with lz4, gzip, snappy) to Apache Kafka 4.0 due to a bug in how librdkafka
evaluates whether compression is supported
(https://github.com/confluentinc/librdkafka/issues/4956) - if the api versions
response does not include produce v0, compression with these formats is
disallowed (zstd checks a newer version of produce).
To fix this, there are two options:
# Update KIP-896 not to remove produce v0-v2 and KIP-724 not to remove support
for producing messages with record format v0-v1.
# Update api versions to return produce v0-v2 as supported, but reject
requests with such versions with `UnsupportedVersion` error.
The second option seems better to me, even though it's not ideal.
was:
A large percentage of non Java clients are unable to produce compressed records
(with lz4, gzip, snappy) to Apache Kafka 4.0 due to a bug in how librdkafka
evaluates whether compression is supported - if the api versions response does
not include produce v0, compression with these formats is disallowed (zstd
checks a newer version of produce).
To fix this, there are two options:
# Update KIP-896 not to remove produce v0-v2 and KIP-724 not to remove support
for producing messages with record format v0-v1.
# Update api versions to return produce v0-v2 as supported, but reject
requests with such versions with `UnsupportedVersion` error.
The second option seems better to me, even though it's not ideal.
> librdkafka compressed produce fails unless api versions returns produce v0
> --------------------------------------------------------------------------
>
> Key: KAFKA-18659
> URL: https://issues.apache.org/jira/browse/KAFKA-18659
> Project: Kafka
> Issue Type: Sub-task
> Reporter: Ismael Juma
> Assignee: Ismael Juma
> Priority: Blocker
> Fix For: 4.0.0
>
>
> A large percentage of non Java clients are unable to produce compressed
> records (with lz4, gzip, snappy) to Apache Kafka 4.0 due to a bug in how
> librdkafka evaluates whether compression is supported
> (https://github.com/confluentinc/librdkafka/issues/4956) - if the api
> versions response does not include produce v0, compression with these formats
> is disallowed (zstd checks a newer version of produce).
> To fix this, there are two options:
> # Update KIP-896 not to remove produce v0-v2 and KIP-724 not to remove
> support for producing messages with record format v0-v1.
> # Update api versions to return produce v0-v2 as supported, but reject
> requests with such versions with `UnsupportedVersion` error.
> The second option seems better to me, even though it's not ideal.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
