Thanks Manikumar for you super fast replies. Let me go through the docs and will raise my questions, if any.
Thanks, Dinesh On 11 May 2015 at 11:46, Manikumar Reddy <ku...@nmsworks.co.in> wrote: > All the consumers in the same consumer group will share the load across > given topic/partitions. > So for any consumer failure, there will be a re-balance to assign the > failed topic/partitions to live consumers. > > pl check consumer documentation here > https://kafka.apache.org/documentation.html#introduction > > On Mon, May 11, 2015 at 11:17 AM, dinesh kumar <dinesh...@gmail.com> > wrote: > > > But why? What is reason for triggering a rebalance if none of the topics > of > > a consumers are affected? Is there some reason for triggering a > rebalance > > irrespective of the consumers topics getting affected ? > > > > On 11 May 2015 at 11:06, Manikumar Reddy <ku...@nmsworks.co.in> wrote: > > > > > If both C1,C2 belongs to same consumer group, then the re-balance will > be > > > triggered. > > > A consumer subscribes to event changes of the consumer id registry > within > > > its group. > > > > > > On Mon, May 11, 2015 at 10:55 AM, dinesh kumar <dinesh...@gmail.com> > > > wrote: > > > > > > > Hi, > > > > I am looking at the code of > > > kafka.consumer.ZookeeperConsumerConnector.scala > > > > (link here > > > > < > > > > > > > > > > https://github.com/apache/kafka/blob/0.8.2/core/src/main/scala/kafka/consumer/ZookeeperConsumerConnector.scala > > > > >) > > > > and I see that all ids registered to a particular group ids are > > > registered > > > > to the path /consumers/[group_id]/ids in zookeeper. the ids contain > > the > > > > consumer_id -> topics mapping. > > > > > > > > A watcher is registered in zookeeper that is triggered when there is > a > > > > change to /consumers/[group_id]/ids. This watcher event is handled by > > the > > > > class ZKRebalancerListener. This class calls a synced rebalance > > whenever > > > a > > > > watcher event is received. > > > > > > > > So here is my question. > > > > 1. Lets consider a scenario where there a two topics T1 and T2 and > two > > > > consumers C1 and C2. C1 consumes only from T1 and C2 only from T2. > Say > > if > > > > C2 dies for some reason as explained before, C1 will get a watcher > > event > > > > from zookeeper and a synced rebalance will be triggered. Why does C2 > > > dying > > > > which has absolutely nothing with C1 (there is no intersection of > > topics > > > > between C1 and C2) should trigger a rebalance event in C1. Is there > > some > > > > condition where this is necessary that I am missing? > > > > > > > > Thanks, > > > > Dinesh > > > > > > > > > >
