No, there isn't, not at the very start when there is no state in Zookeeper. Once there is state the Kafka team have told me that rebalancing will not result in any dupes.
However, if there is no state in Zookeeper and your partitions are empty, simply wait until all consumers have balanced before sending any data. Philip On Wed, Aug 21, 2013 at 12:30 PM, Yu, Libo <libo...@citi.com> wrote: > Hi, > > This is from kafka faq: > > > * Each partition is not consumed by more than one consumer thread/process > in each consumer group. This allows to have each process consume in a single > threaded fashion to guarantee ordering to the consumer within the partition > (if we split up a partition of ordered messages and handed them out to > multiple consumers even though the messages were stored in order they would > be processed out of order at times). > > Is this doable? > > Say a topic has 3 partitions. And there are 3 consumer processes in a > consumer group each of which has a single thread. > When we start the 3 processes, when the first process is up, it will consume > all 3 partitions. When the second process is up, > 1 process consumes 2 partitions and 1 consumes 1 partition. Only when the > third process is up, will each process consume > One partition. This will cause trouble. There seems to be no way to bound a > stream to one partition. > > > Regards, > > Libo >
