hi, I've had the same issue with the kafka producer.
you need to use a different partitioner than the default one provided for kafka. I've created a round robin partitioner that works well for equally distributing data across partitions. https://github.com/gerritjvv/pseidon/blob/master/pseidon-kafka/java/pseidon/kafka/util/RoundRobinPartitioner.java On Tue, Dec 17, 2013 at 5:32 PM, Hanish Bansal < hanish.bansal.agar...@gmail.com> wrote: > Hi All, > > We are having kafka cluster of 2 nodes. (using 0.8.0 final release) > Replication Factor: 2 > Number of partitions: 2 > > I have created a topic "test-topic1" in kafka. > > When i am listing status of that topic using bin/kafka-list-topic.sh, the > status is: > > topic: test-topic1 partition: 0 leader: 0 replicas: 0,1 isr: > 0,1 > topic: test-topic1 partition: 1 leader: 1 replicas: 1,0 isr: > 1,0 > > As both partition are on two separate nodes so when we produce the data it > should be go to both nodes. > > But when i insert the data, it is going to only one node. > > For example if i insert 1000 messages then all 1000 messages will go either > node1 or node2. Data is not evenly distributed on both nodes. > > Expected: 500 messages should go to node1 and 500 messages should go to > node2. > > Any suggestion why i am facing this behavior? > > -- > *Thanks & Regards* > *Hanish Bansal* >
