I think I found my answer reading the 0.8.2 release notes about the changes to consumer groups; basically they are triples (group, topic, partition) for the key. That's all I needed to know.
Thanks! (but got a different follow up question coming!) On Tue Feb 17 2015 at 9:34:29 AM Todd Palino <tpal...@gmail.com> wrote: > I'm assuming from your description here that all of these topics are being > consumed by a single consumer (i.e. a single process that does something > different with each topic it sees). In general, you're going to get more > efficiency out of a single consumer instance that consumes multiple topics > than you will out of multiple consumers that each consume a single topic. > Which means that you should go with a single consumer group to describe the > topics consumed by a single consumer. > > If, on the other hand, you have separate processes/threads/components that > consume each topic, you'll find that it doesn't matter much either way. In > that case I would probably go with individual groups for isolation. > > -Todd > > > On Mon, Feb 16, 2015 at 3:30 PM, Scott Chapman <sc...@woofplanet.com> > wrote: > > > We have several dozen topics, each with only one topic (replication > factor > > or 2). > > > > We are wanting to launch console-consumer for these in a manner that will > > support saving offsets (so they can resume where they left off if they > need > > to be restarted). And I know consumer groups is the mechanism for doing > > that. > > > > My question is, should we use a single consumer-group for all the console > > consumers (we are launching one for each topic) or should be be > generating > > topic-specific consumer groups? > > > > Thanks in advance! > > > > -Scott. > > >
