1 consumer group can have many partitions, if the consumer group has 1 consumer and there are N partitions, it will consume from N, if you have a spike you can add up to N more consumers to that consumer group.
2016-01-16 11:32 GMT+01:00 Jason Williams <jasonjwwilli...@gmail.com>: > Thanks Jens! > > So assuming you've already paired your partition count to the consumer > count...if you experience a spike in messages and want to spin up more > consumers to add temporary processing capacity, what's the suggested way to > handle this? (since it would seem you can't just add consumers as they > would remain idle...and adding partitions appears to require manual > rebalancing). > > -J > > Sent via iPhone > > > On Jan 16, 2016, at 01:54, Jens Rantil <jens.ran...@tink.se> wrote: > > > > Hi, > > > > > > You are correct. The others will remain idle. This is why you generally > want to have at least the same number of partitions as consumers. > > > > > > > > > > Cheers, > > > > Jens > > > > > > > > > > > > – > > Skickat från Mailbox > > > > On Sat, Jan 16, 2016 at 2:34 AM, Jason J. W. Williams > > <jasonjwwilli...@gmail.com> wrote: > > > >> Hi, > >> I'm trying to make sure I understand this statement in the docs: > >> "Each broker partition is consumed by a single consumer within a given > >> consumer group. The consumer must establish its ownership of a given > >> partition before any consumption can begin." > >> If I have: > >> * a topic with 1 partition > >> * subscribe a consumer group to the topic > >> * the consumer group has 10 consumers belonging to it > >> Will only 1 consumer of the 10 ever receive messages from the topic, and > >> the other 9 remain idle? Or does this mean only 1 consumer at a time > from > >> the group will be consuming...in a round-robin fashion? > >> -J >
