If the 100 partitions are all for the same topic, you can have up to 100 consumers working as part of a single consumer group for that topic. You cannot have more consumers than there are partitions within a given consumer group.
On 29 April 2015 at 08:41, Nimi Wariboko Jr <n...@channelmeter.com> wrote: > Hi, > > I was wondering what options there are for horizontally scaling kafka > consumers? Basically if I have 100 partitions and 10 consumers, and want to > temporarily scale up to 50 consumers, what options do I have? > > So far I've thought of just simply tracking consumer membership somehow > (either through Raft or zookeeper's znodes) on the consumers. >
