Palur, If the 8 partitions are hosted on each one of the nodes, assuming replication factor 1 then each node will get roughly 100000 / 8 messages due to the random partitioner. If you want to know exactly how many messages is on each broker then you can use a simple consumer which allows you to specify the partition id you want to consume from.
In the new consumer (0.9), each of the consumed message will contain the partition id as part of its message metadata. Guozhang On Tue, Nov 25, 2014 at 7:47 AM, Palur Sandeep <psand...@hawk.iit.edu> wrote: > Dear Developers, > > I am using the default partitioning logic(Random Partitioning) to produce > messages into brokers. That is I don't use a partitioner.class. > > My requirement is If I produce 100000 messages using the below code for a > broker that has 8 partitions across 8 nodes. How many messages will each > partition have? Is there any API that can help me find the broker id of the > each message I consume from the consumer side? > > PS: I dont want to use partitioner.class. I want use the kafka's default > partitioning logic. > > KeyedMessage<String,String> data = new KeyedMessage<String, > String>(topic_name,new_mes); > > producer.send(data); > > -- > Regards, > Sandeep Palur > Data-Intensive Distributed Systems Laboratory, CS/IIT > Department of Computer Science, Illinois Institute of Technology (IIT) > Phone : 312-647-9833 > Email : psand...@hawk.iit.edu <sraja...@hawk.iit.edu> > -- -- Guozhang
