Hi Jun,

This is my original concern:
Assume the number of consumer threads is the same as the number of partitions 
of a topic.
And we want to leverage the fact that messages in a partition are ordered. But 
when starting 
multiple consumer threads or rebalancing, it is possible that one consumer 
thread will 
temporarily consume two or more partitions.

After thinking it over, I realize this can be solved in the consumer thread as 
long as we never
assume one consumer thread will only consume one partition.

Regards,

Libo


-----Original Message-----
From: Jun Rao [mailto:jun...@gmail.com] 
Sent: Thursday, August 22, 2013 12:01 AM
To: users@kafka.apache.org
Subject: Re: ordering

Actually, I am not sure if I understand the trouble that you mentioned.
Could you elaborate that a bit more?

Thanks,

Jun


On Wed, Aug 21, 2013 at 12:30 PM, Yu, Libo <libo...@citi.com> wrote:

> Hi,
>
> This is from kafka faq:
>
>
>   *   Each partition is not consumed by more than one consumer
> thread/process in each consumer group. This allows to have each 
> process consume in a single threaded fashion to guarantee ordering to 
> the consumer within the partition (if we split up a partition of 
> ordered messages and handed them out to multiple consumers even though 
> the messages were stored in order they would be processed out of order at 
> times).
>
> Is this doable?
>
> Say a topic has 3 partitions. And there are 3 consumer processes in a 
> consumer group each of which has a single thread.
> When we start the 3 processes, when the first process is up, it will 
> consume all 3 partitions. When the second process is up,
> 1 process consumes 2 partitions and 1 consumes 1 partition. Only when 
> the third process is up, will each process consume One partition. This 
> will cause trouble. There seems to be no way to bound a stream to one 
> partition.
>
>
> Regards,
>
> Libo
>
>

Reply via email to