Guozhang Wang created KAFKA-3704:
------------------------------------
Summary: Use default block size in KafkaProducer
Key: KAFKA-3704
URL: https://issues.apache.org/jira/browse/KAFKA-3704
Project: Kafka
Issue Type: Bug
Reporter: Guozhang Wang
Assignee: Guozhang Wang
Fix For: 0.10.0.0
As discovered in https://issues.apache.org/jira/browse/KAFKA-3565, the current
default block size (1K) used in Snappy and GZIP may cause sub-optimal
compression ratio for Snappy, and hence reduce throughput.
A better solution would be using the default block size, which is 32K in Snappy
and 0.5K in GZIP. A notable side-effect is that with Snappy, this solution will
require more extra memory allocated out side of the bufferpoll, by {{(32 - 1)K
* num.total.partitions}} and hence users may need to bump up their JVM heap
size, especially for MirrorMakers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
