All the consumers in the same consumer group will share the load across given topic/partitions. So for any consumer failure, there will be a re-balance to assign the failed topic/partitions to live consumers.
pl check consumer documentation here https://kafka.apache.org/documentation.html#introduction On Mon, May 11, 2015 at 11:17 AM, dinesh kumar <dinesh...@gmail.com> wrote: > But why? What is reason for triggering a rebalance if none of the topics of > a consumers are affected? Is there some reason for triggering a rebalance > irrespective of the consumers topics getting affected ? > > On 11 May 2015 at 11:06, Manikumar Reddy <ku...@nmsworks.co.in> wrote: > > > If both C1,C2 belongs to same consumer group, then the re-balance will be > > triggered. > > A consumer subscribes to event changes of the consumer id registry within > > its group. > > > > On Mon, May 11, 2015 at 10:55 AM, dinesh kumar <dinesh...@gmail.com> > > wrote: > > > > > Hi, > > > I am looking at the code of > > kafka.consumer.ZookeeperConsumerConnector.scala > > > (link here > > > < > > > > > > https://github.com/apache/kafka/blob/0.8.2/core/src/main/scala/kafka/consumer/ZookeeperConsumerConnector.scala > > > >) > > > and I see that all ids registered to a particular group ids are > > registered > > > to the path /consumers/[group_id]/ids in zookeeper. the ids contain > the > > > consumer_id -> topics mapping. > > > > > > A watcher is registered in zookeeper that is triggered when there is a > > > change to /consumers/[group_id]/ids. This watcher event is handled by > the > > > class ZKRebalancerListener. This class calls a synced rebalance > whenever > > a > > > watcher event is received. > > > > > > So here is my question. > > > 1. Lets consider a scenario where there a two topics T1 and T2 and two > > > consumers C1 and C2. C1 consumes only from T1 and C2 only from T2. Say > if > > > C2 dies for some reason as explained before, C1 will get a watcher > event > > > from zookeeper and a synced rebalance will be triggered. Why does C2 > > dying > > > which has absolutely nothing with C1 (there is no intersection of > topics > > > between C1 and C2) should trigger a rebalance event in C1. Is there > some > > > condition where this is necessary that I am missing? > > > > > > Thanks, > > > Dinesh > > > > > >
