you can try DumpLogSegments tools to verify messages from log files. This will give compression type for each message. https://cwiki.apache.org/confluence/display/KAFKA/System+Tools#SystemTools- DumpLogSegment
On Thu, Sep 21, 2017 at 1:38 PM, Vincent Dautremont < vincent.dautrem...@olamobile.com.invalid> wrote: > Hi, > Snappy keeps a lot of parts in plain text : > look that example where only "pedia" is encoded/tokenized in the sentence. > https://en.wikipedia.org/wiki/Snappy_(compression) > > > Wikipedia is a free, web-based, collaborative, multilingual encyclopedia > > project. > > > your data is then probably compressed with snappy. > > Another try would be to change compression to other values (or remove > compression) and compare the tcp dump with the one you already have. > > > Vincent. > > On Wed, Sep 20, 2017 at 11:58 PM, Wesley Chow <w...@chartbeat.com> wrote: > > > I have a producer configured to snappy compress data sent to a cluster. > Is > > there some way to verify that the data indeed is being compressed? If I > > peek at the .log files on the broker, I can read some plain text amongst > > binary. Similarly, tcpdump shows plain text readable data. I do not know > if > > this is evidence that compression is not working, but is there a better > way > > to verify that compression is turned on? > > > > Wes > > > > -- > The information transmitted is intended only for the person or entity to > which it is addressed and may contain confidential and/or privileged > material. Any review, retransmission, dissemination or other use of, or > taking of any action in reliance upon, this information by persons or > entities other than the intended recipient is prohibited. If you received > this in error, please contact the sender and delete the material from any > computer. >
