Yup, sounds like https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Whyisdatanotevenlydistributedamongpartitionswhenapartitioningkeyisnotspecified ?
This should go away with 0.8.2 with the default partions now being 1 =8^) with auto create topics. /******************************************* Joe Stein Founder, Principal Consultant Big Data Open Source Security LLC http://www.stealth.ly Twitter: @allthingshadoop ********************************************/ On Nov 13, 2014 8:34 PM, "Chia-Chun Shih" <chiachun.s...@gmail.com> wrote: > Hi Palur, > > When producing messages, did you specify a key in your KeyedMessage? If > not, producer will send all messages to ONE randomly selected partition and > stick to this partition for 10 minutes by default. > > regards, > Chia-Chun > > 2014-11-14 7:19 GMT+08:00 Jagat Singh <jagatsi...@gmail.com>: > > > It would be worth reading once the consumer section from the > documentation. > > > > https://kafka.apache.org/documentation.html > > > > > > > > On Fri, Nov 14, 2014 at 10:09 AM, Palur Sandeep <psand...@hawk.iit.edu> > > wrote: > > > > > Yes, they are on the same consumer group, but I have two partitions. > > > > > > On Thu, Nov 13, 2014 at 5:04 PM, Jagat Singh <jagatsi...@gmail.com> > > wrote: > > > > > > > Are both of them in same Consumer Group? > > > > > > > > On Fri, Nov 14, 2014 at 9:12 AM, Palur Sandeep < > psand...@hawk.iit.edu> > > > > wrote: > > > > > > > > > Dear Developers, > > > > > > > > > > I am 2nd year masters student at IIT. I am using Kafka for one of > my > > > > > research projects.My question is the following: > > > > > > > > > > 1. I have a producer, consumer and a broker(that contains 1st > > partition > > > > of > > > > > my topic) on node1 > > > > > 2. I have a producer, consumer, zookeeper and a broker(that > contains > > > 2nd > > > > > partition of my topic) on node2 > > > > > 3. Here comes my problem: though I have two partitions only one > > > consumer > > > > > pulls messages and the other one is always idle. > > > > > > > > > > What is that I can do to keep both of my consumer busy? > > > > > > > > > > Thank you > > > > > > > > > > > > > > > -- > > > > > Regards, > > > > > Sandeep Palur > > > > > Data-Intensive Distributed Systems Laboratory, CS/IIT > > > > > Department of Computer Science, Illinois Institute of Technology > > (IIT) > > > > > Phone : 312-647-9833 > > > > > Email : psand...@hawk.iit.edu <sraja...@hawk.iit.edu> > > > > > > > > > > > > > > > > > > > > > -- > > > Regards, > > > Sandeep Palur > > > Data-Intensive Distributed Systems Laboratory, CS/IIT > > > Department of Computer Science, Illinois Institute of Technology (IIT) > > > Phone : 312-647-9833 > > > Email : psand...@hawk.iit.edu <sraja...@hawk.iit.edu> > > > > > >
