I think you are hitting this - https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Whysomeoftheconsumersinaconsumergroupneverreceiveanymessage%3F
Let us know if we can improve the documentation to make it clearer. Thanks, Neha On Wed, Sep 11, 2013 at 5:28 PM, prashant amar <amasin...@gmail.com> wrote: > Also attempted another pattern where > > 1. Created a topic T with 'n' partitions. > 2. Created a consumer group process with 'n + 1' threads subscribing from > topic 'T' with a groupID 'y' > 3. Added another consumer group process with 'n + 1' threads subscribing > from same topic 'T' with same groupID 'z' > (Note that 2 and 3 subscribe from same topic but different groups) > > Can a single topic with multiple partitions abetted with multiple consumer > groups increase parallelism is consumption? > > > > > > > > > On Wed, Sep 11, 2013 at 4:48 PM, prashant amar <amasin...@gmail.com> > wrote: > > > A Design Question that needs verification: > > > > 1. Created a topic T with 'n' partitions. > > 2. Created a consumer group process with 'n + 1' threads subscribing from > > topic 'T' with a groupID 'y' > > 3. Added another consumer group process with 'n + 1' threads subscribing > > from same topic 'T' with same groupID 'y' > > > > On doing so, I noticed that the previous consumer group stops consuming > > and the new consumer beings to consume > > > > I was attempting to model on demand parallelization in an event where an > > consumer group cannot keep up with the events produced. Rather than > > increase the threadpool capacity in the same process, it would make sense > > to distribute the load across multiple processes. > > > > Advice please? > > > > Regards > > Amardeep > > >
