Magnus is right, in both old and new producer, the data is first partitioned, and within each partition batched and compressed for sending.
Guozhang On Tue, Aug 19, 2014 at 7:12 AM, Magnus Edenhill <mag...@edenhill.se> wrote: > Hi Lex, > > messages are batched (and compressed) after partitioning. > > > > 2014-08-19 15:46 GMT+02:00 Lian, Li <ll...@ebay.com>: > > > I am using Kafka 0.8 and message compression can only be enabled on > > producer side. > > > > However one batch of messages (decided by the producer configuration > > batch.num.messages) is compressed as a single message and stored at the > > broker side. I think one batch compressed message will be stored at one > > broker. > > > > I am wondering in this case how the producer side partitioner class will > > work? The batch will be partitioned to the partition of the first or last > > message in the batch? > > > > > > Lex > > > > > -- -- Guozhang
