Re: How kafka assign partition to stream

2014-03-10 Thread Guozhang Wang
Hi Yonghui, In 0.8 the load balance logic in consumers is based on range partitioning with consumer-id as {consumer-1-stream-1, consumer-1-stream-2, ... consumer-1-stream-10, consumer-2-stream-1, ...} and partitions are assigned to this list in round robin. So yes, this behavior is expected. If yo

How kafka assign partition to stream

2014-03-10 Thread Yonghui Zhao
In my environment, I have 2 brokers and only 1 topic, each brokers has 10 partitions, so there are 20 partitions in total. I have 4 consumers in one consumer group, each consumer use createMessageStreams to create 10 streams, 40 streams in total. Since partition can not be split, so there are