[ 
https://issues.apache.org/jira/browse/KAFKA-18659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ismael Juma updated KAFKA-18659:
--------------------------------
    Description: 
A large percentage of non Java clients are unable to produce compressed records 
(with lz4, gzip, snappy) to Apache Kafka 4.0 due to a bug in how librdkafka 
evaluates whether compression is supported 
(https://github.com/confluentinc/librdkafka/issues/4956) - if the api versions 
response does not include produce v0, compression with these formats is 
disallowed (zstd checks a newer version of produce).

To fix this, there are two options:
 # Update KIP-896 not to remove produce v0-v2 and KIP-724 not to remove support 
for producing messages with record format v0-v1.
 # Update api versions to return produce v0-v2 as supported, but reject 
requests with such versions with `UnsupportedVersion` error.

The second option seems better to me, even though it's not ideal.

  was:
A large percentage of non Java clients are unable to produce compressed records 
(with lz4, gzip, snappy) to Apache Kafka 4.0 due to a bug in how librdkafka 
evaluates whether compression is supported - if the api versions response does 
not include produce v0, compression with these formats is disallowed (zstd 
checks a newer version of produce).

To fix this, there are two options:
 # Update KIP-896 not to remove produce v0-v2 and KIP-724 not to remove support 
for producing messages with record format v0-v1.
 # Update api versions to return produce v0-v2 as supported, but reject 
requests with such versions with `UnsupportedVersion` error.

The second option seems better to me, even though it's not ideal.


> librdkafka compressed produce fails unless api versions returns produce v0
> --------------------------------------------------------------------------
>
>                 Key: KAFKA-18659
>                 URL: https://issues.apache.org/jira/browse/KAFKA-18659
>             Project: Kafka
>          Issue Type: Sub-task
>            Reporter: Ismael Juma
>            Assignee: Ismael Juma
>            Priority: Blocker
>             Fix For: 4.0.0
>
>
> A large percentage of non Java clients are unable to produce compressed 
> records (with lz4, gzip, snappy) to Apache Kafka 4.0 due to a bug in how 
> librdkafka evaluates whether compression is supported 
> (https://github.com/confluentinc/librdkafka/issues/4956) - if the api 
> versions response does not include produce v0, compression with these formats 
> is disallowed (zstd checks a newer version of produce).
> To fix this, there are two options:
>  # Update KIP-896 not to remove produce v0-v2 and KIP-724 not to remove 
> support for producing messages with record format v0-v1.
>  # Update api versions to return produce v0-v2 as supported, but reject 
> requests with such versions with `UnsupportedVersion` error.
> The second option seems better to me, even though it's not ideal.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Reply via email to