Hey Todd, For my purposes, I wrote a Partitioner that reads all of the ConsumerOffsets for a particular group from Zookeeper, and then sends its message to the one with least lag (with some probabilistic weighting thrown in to prevent it from getting overwhelmed). I basically read and reimplemented the code from ConsumerOffsetChecker.scala to do this. Does that help at all? Would it help to see my code?
--Ian On Mar 19, 2014, at 12:47 PM, Todd Gatts <tga...@us.ibm.com> wrote: > > > I'd like to avoid creating messages for a topic is the topic currently has > no consumers. Is there a way a producer can subscribe to the number of > consumers of a topic? Or if that's not possible, can a producer request > the current number of consumers of a topic? > > The searchable hadoop kafka topic didn't return anything for "no consumer" > or "zero consumers", and a find on "zero" or "count" on the documentation > html page found lots of matches, but none about knowing the number of > consumers. > > I'm certain I can code a solution around this problem, but I'd hate to > reinvent the wheel for such a straightforward problem. > > Todd Gatts
