We compress a batch of messages together, but we need to give each
message its own offset (and know its key if we want to use topic
compaction), so messages are un-compressed and re-compressed.
We are working on an improvement to add relative offsets which will
allow the broker to skip this re-com
Hi,
I was 100% sure that Kafka broker didn't compress data and I didn't
think that I had to upgrade my broker to 8.2.2.
I tried the upgrade and It works right now!
I still don't understand, why the broker need to compress data again
(if the data compression is already done in the producer). Have
You will need to upgrade the broker to 0.8.2.2. Broker currently
recompresses messages. In 0.8.2.1, the snappy jar has a bug that causes
data explosion. We fixed the snappy jar in 0.8.2.2. If you upgrade the
broker to 0.8.2.2, it will pick up the fixed snappy jar.
Thanks,
Jun
On Sat, Oct 17, 201
o dig into this problem
yet.
Lukas
-Original Message-
From: Jérôme BAROTIN
Sent: Saturday, October 17, 2015 1:21 AM
To: users@kafka.apache.org
Subject: Kafka 8.2.2 doesn't want to compress in snappy
Hello,
I want to check if the snappy compression works well with the java Kafka
cli
Hello,
I want to check if the snappy compression works well with the java Kafka client.
In order to handle this, I set up a small program. This program
generate 1024 messages of readable data. Their size are of 1024 bytes
each. I send these messages on tree new topics and after I check the
size o
