Hi, Are you using the official Java client or any third party clients?
On Thu, 17 Sep 2020 at 22:53, M. Manna <manme...@gmail.com> wrote: > Hey Shaohan, > > Thanks for your reply, much appreciated. We are using Kafka 2.4.1. Although > we use the Confluent platform under Confluent Licence, I don't think this > matters. > > We are using compression.type=GZIP for all our producers. > > We create all topics with compression.type=GZIP i.e. per topic level. When > I do a "describe" of a kafka topic, I see metadata confirming that it's > enabled for GZIP compressed data. > > That's all I could say. > > Regards, > > > > > On Thu, 17 Sep 2020 at 15:01, Shaohan Yin <shaohan....@gmail.com> wrote: > > > Hi, > > > > Could you specify your version and the protocol of your broker? > > > > From the client of 2.5.1 I didn't see any changes that could be made to > the > > client compression. > > Maybe you could check if the compression.type is set on the topic level > or > > the broker side? > > > > Cheers > > > > On Thu, 17 Sep 2020 at 20:19, M. Manna <manme...@gmail.com> wrote: > > > > > Hello, > > > > > > I am trying to understand the compression.type settings and the growth > > > impact from this. With a compression type (valid) settings, this > ensures > > > that the message in the topic stays compressed and has to be > decompressed > > > by the consumer. > > > > > > However, it may be possible that the compression will not happen i.e. > > NONE > > > if the uncompressed size is the same as compressed. I am trying to use > > this > > > command to verify: > > > > > > ./kafka-run-class kafka.tools.DumpLogSegments --files > > > <log-dir>/filename.log > > > --print-data-log | grep -iE "compresscodec: NONE" > > > > > > I can see a lot of entries with NONE. But all my producers are writing > > with > > > compression.type=GZIP. Is this expected? > > > > > > Regards, > > > > > >
