Re: Need to understand consumer groups.

2015-02-17 Thread Scott Chapman
I think I found my answer reading the 0.8.2 release notes about the changes to consumer groups; basically they are triples (group, topic, partition) for the key. That's all I needed to know. Thanks! (but got a different follow up question coming!) On Tue Feb 17 2015 at 9:34:29 AM Todd Palino wr

Re: Need to understand consumer groups.

2015-02-17 Thread Todd Palino
I'm assuming from your description here that all of these topics are being consumed by a single consumer (i.e. a single process that does something different with each topic it sees). In general, you're going to get more efficiency out of a single consumer instance that consumes multiple topics tha

Need to understand consumer groups.

2015-02-16 Thread Scott Chapman
We have several dozen topics, each with only one topic (replication factor or 2). We are wanting to launch console-consumer for these in a manner that will support saving offsets (so they can resume where they left off if they need to be restarted). And I know consumer groups is the mechanism for