Hey Shaohan, Thanks for your reply, much appreciated. We are using Kafka 2.4.1. Although we use the Confluent platform under Confluent Licence, I don't think this matters.
We are using compression.type=GZIP for all our producers. We create all topics with compression.type=GZIP i.e. per topic level. When I do a "describe" of a kafka topic, I see metadata confirming that it's enabled for GZIP compressed data. That's all I could say. Regards, On Thu, 17 Sep 2020 at 15:01, Shaohan Yin <shaohan....@gmail.com> wrote: > Hi, > > Could you specify your version and the protocol of your broker? > > From the client of 2.5.1 I didn't see any changes that could be made to the > client compression. > Maybe you could check if the compression.type is set on the topic level or > the broker side? > > Cheers > > On Thu, 17 Sep 2020 at 20:19, M. Manna <manme...@gmail.com> wrote: > > > Hello, > > > > I am trying to understand the compression.type settings and the growth > > impact from this. With a compression type (valid) settings, this ensures > > that the message in the topic stays compressed and has to be decompressed > > by the consumer. > > > > However, it may be possible that the compression will not happen i.e. > NONE > > if the uncompressed size is the same as compressed. I am trying to use > this > > command to verify: > > > > ./kafka-run-class kafka.tools.DumpLogSegments --files > > <log-dir>/filename.log > > --print-data-log | grep -iE "compresscodec: NONE" > > > > I can see a lot of entries with NONE. But all my producers are writing > with > > compression.type=GZIP. Is this expected? > > > > Regards, > > >
