A consumer can consume more than one partition. Messages from the same partition will still be delivered to the consumer in order. There is no ordering guarantee for messages across partitions.
Thanks, Jun On Thu, Aug 22, 2013 at 6:52 AM, Yu, Libo <libo...@citi.com> wrote: > Hi Jun, > > This is my original concern: > Assume the number of consumer threads is the same as the number of > partitions of a topic. > And we want to leverage the fact that messages in a partition are ordered. > But when starting > multiple consumer threads or rebalancing, it is possible that one consumer > thread will > temporarily consume two or more partitions. > > After thinking it over, I realize this can be solved in the consumer > thread as long as we never > assume one consumer thread will only consume one partition. > > Regards, > > Libo > > > -----Original Message----- > From: Jun Rao [mailto:jun...@gmail.com] > Sent: Thursday, August 22, 2013 12:01 AM > To: users@kafka.apache.org > Subject: Re: ordering > > Actually, I am not sure if I understand the trouble that you mentioned. > Could you elaborate that a bit more? > > Thanks, > > Jun > > > On Wed, Aug 21, 2013 at 12:30 PM, Yu, Libo <libo...@citi.com> wrote: > > > Hi, > > > > This is from kafka faq: > > > > > > * Each partition is not consumed by more than one consumer > > thread/process in each consumer group. This allows to have each > > process consume in a single threaded fashion to guarantee ordering to > > the consumer within the partition (if we split up a partition of > > ordered messages and handed them out to multiple consumers even though > > the messages were stored in order they would be processed out of order > at times). > > > > Is this doable? > > > > Say a topic has 3 partitions. And there are 3 consumer processes in a > > consumer group each of which has a single thread. > > When we start the 3 processes, when the first process is up, it will > > consume all 3 partitions. When the second process is up, > > 1 process consumes 2 partitions and 1 consumes 1 partition. Only when > > the third process is up, will each process consume One partition. This > > will cause trouble. There seems to be no way to bound a stream to one > > partition. > > > > > > Regards, > > > > Libo > > > > >
