ramizmehran opened a new pull request, #19455: URL: https://github.com/apache/kafka/pull/19455
This pull request introduces a new configuration option to limit the maximum size of a record before compression in the Kafka producer. The changes ensure that the producer can handle highly compressible data more efficiently by setting a cap on the uncompressed record size. The most important changes include adding a new configuration parameter, updating the producer class to use this parameter, and modifying the error handling to reflect the new limit. ### New Configuration Parameter: * [`clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java`](diffhunk://#diff-1e3696628fc126b37a1c1e37a747e4073979946620354bc77264d83d6565d035R192-R199): Added `MAX_RECORD_SIZE_CONFIG` to define the maximum size of a record before compression, with a default value and documentation. [[1]](diffhunk://#diff-1e3696628fc126b37a1c1e37a747e4073979946620354bc77264d83d6565d035R192-R199) [[2]](diffhunk://#diff-1e3696628fc126b37a1c1e37a747e4073979946620354bc77264d83d6565d035R410-R415) ### Updates to Producer Class: * [`clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java`](diffhunk://#diff-42d8f5166459ee28f201ff9cec0080fc7845544a0089ac9e8f3e16864cc1193eR256): Introduced `maxRecordSize` field and updated constructors to initialize it using the new configuration parameter. [[1]](diffhunk://#diff-42d8f5166459ee28f201ff9cec0080fc7845544a0089ac9e8f3e16864cc1193eR256) [[2]](diffhunk://#diff-42d8f5166459ee28f201ff9cec0080fc7845544a0089ac9e8f3e16864cc1193eR411) [[3]](diffhunk://#diff-42d8f5166459ee28f201ff9cec0080fc7845544a0089ac9e8f3e16864cc1193eR499) ### Error Handling: * [`clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java`](diffhunk://#diff-42d8f5166459ee28f201ff9cec0080fc7845544a0089ac9e8f3e16864cc1193eL1141-R1147): Modified `ensureValidRecordSize` method to use `maxRecordSize` instead of `maxRequestSize` for validation and error messages. ### Configuration File: * [`config/producer.properties`](diffhunk://#diff-8d5d833805078295042353c0d8993f9498b77151ce72f1b288759669bec961b9R42-R44): Added commented-out entry for `max.record.size` to the producer properties file for user reference. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
