Hi Kafka dev, I would like to start the discussion of KIP-780: Support fine-grained compression options.
https://cwiki.apache.org/confluence/display/KAFKA/KIP-780%3A+Support+fine-grained+compression+options Here is some context or history on this feature; initially, this feature was intended to be a part of KIP-390: Support Compression Level, but when I was working on it, I could not find the evidence that these options can improve the performance, so it was excluded from the final proposal. Since this (tentative) conclusion was somewhat strange, KIP-390 was passed under the condition that a following work should be done for the buffer/block/window-related configuration options. And after some repetitive prototypes and benchmarks, it seems like I finally found the evidence. It is why I am submitting it as a separate proposal now. The document also includes what I found during the tests in the Benchmark section. All kinds of feedbacks are greatly appreciated! Best, Dongjin -- *Dongjin Lee* *A hitchhiker in the mathematical world.* *github: <http://goog_969573159/>github.com/dongjinleekr <https://github.com/dongjinleekr>keybase: https://keybase.io/dongjinleekr <https://keybase.io/dongjinleekr>linkedin: kr.linkedin.com/in/dongjinleekr <https://kr.linkedin.com/in/dongjinleekr>speakerdeck: speakerdeck.com/dongjin <https://speakerdeck.com/dongjin>*
