The offsets are keyed on <group, topic, partition> so if you have more than one owner per partition, they will rewrite each other's offsets and lead to incorrect state.
On Tue, Dec 2, 2014 at 2:32 PM, hsy...@gmail.com <hsy...@gmail.com> wrote: > Thanks Neha, another question, so if offsets are stored under group.id, > dose it mean in one group, there should be at most one subscriber for each > topic partition? > > Best, > Siyuan > > On Tue, Dec 2, 2014 at 12:55 PM, Neha Narkhede <neha.narkh...@gmail.com> > wrote: > > > 1. In this doc it says kafka consumer will automatically do load balance. > > Is it based on throughtput or same as what we have now balance the > > cardinality among all consumers in same ConsumerGroup? In a real case > > different partitions could have different peak time. > > > > Load balancing is still based on # of partitions for the subscribed > topics > > and > > ensuring that each partition has exactly one consumer as the owner. > > > > 2. In the API, threre is subscribe(partition...) method saying not using > > group management, does it mean the group.id property will be discarded > and > > developer has full control of distributing partitions to consumers? > > > > group.id is also required for offset management, if the user chooses to > > use > > Kafka based offset management. The user will have full control over > > distribution > > of partitions to consumers. > > > > 3. Is new API compatible with old broker? > > > > Yes, it will. > > > > 4. Will simple consumer api and high-level consumer api still be > supported? > > > > Over time, we will phase out the current high-level and simple consumer > > since the > > 0.9 API supports both. > > > > Thanks, > > Neha > > > > On Tue, Dec 2, 2014 at 12:07 PM, hsy...@gmail.com <hsy...@gmail.com> > > wrote: > > > > > Hi guys, > > > > > > I'm interested in the new Consumer API. > > > http://people.apache.org/~nehanarkhede/kafka-0.9-consumer-javadoc/doc/ > > > > > > I have couple of question. > > > 1. In this doc it says kafka consumer will automatically do load > balance. > > > Is it based on throughtput or same as what we have now balance the > > > cardinality among all consumers in same ConsumerGroup? In a real case > > > different partitions could have different peak time. > > > 2. In the API, threre is subscribe(partition...) method saying not > using > > > group management, does it mean the group.id property will be discarded > > and > > > developer has full control of distributing partitions to consumers? > > > 3. Is new API compatible with old broker? > > > 4. Will simple consumer api and high-level consumer api still be > > supported? > > > > > > Thanks! > > > > > > Best, > > > Siyuan > > > > > >
