I'm assuming from your description here that all of these topics are being consumed by a single consumer (i.e. a single process that does something different with each topic it sees). In general, you're going to get more efficiency out of a single consumer instance that consumes multiple topics than you will out of multiple consumers that each consume a single topic. Which means that you should go with a single consumer group to describe the topics consumed by a single consumer.
If, on the other hand, you have separate processes/threads/components that consume each topic, you'll find that it doesn't matter much either way. In that case I would probably go with individual groups for isolation. -Todd On Mon, Feb 16, 2015 at 3:30 PM, Scott Chapman <sc...@woofplanet.com> wrote: > We have several dozen topics, each with only one topic (replication factor > or 2). > > We are wanting to launch console-consumer for these in a manner that will > support saving offsets (so they can resume where they left off if they need > to be restarted). And I know consumer groups is the mechanism for doing > that. > > My question is, should we use a single consumer-group for all the console > consumers (we are launching one for each topic) or should be be generating > topic-specific consumer groups? > > Thanks in advance! > > -Scott. >
