Thanks for response Gerrit and Guozhang !!
Hi Gerrit,
I am trying to use same round robin partitioner shared by you but hard
luck, still round robin partitioning not working.
I have successfully registered RoundRobinPartitioner in kafka producer.
Code of RoundRobinPartitioner class as:
public RoundRobinPartitioner(VerifiableProperties props){
log.info("Using Round Robin Partitioner class...");
}
@Override
public int partition(String key, int partitions) {
log.info("Inside partition method");
int i = counter.getAndIncrement();
if(i == Integer.MAX_VALUE){
counter.set(0);
return 0;
}else
return i % partitions;
}
When i produce the data, first log message "Using Round Robin Partitioner
class..." is printed and second message "Inside partition method" is not
printed.
>From that we can ensure that RoundRobinPartitioner has been successfully
registered but logic of round robin is not getting called.
Any help to resolve what i am missing ?
Thanks in advance !!
On Tue, Dec 17, 2013 at 5:59 PM, Guozhang Wang <wangg...@gmail.com> wrote:
> Hello,
>
> This issue is known as in this JIRA:
>
> https://issues.apache.org/jira/browse/KAFKA-1067
>
> Guozhang
>
>
> On Tue, Dec 17, 2013 at 8:48 AM, Gerrit Jansen van Vuuren <
> gerrit...@gmail.com> wrote:
>
> > hi,
> >
> > I've had the same issue with the kafka producer.
> >
> > you need to use a different partitioner than the default one provided for
> > kafka.
> > I've created a round robin partitioner that works well for equally
> > distributing data across partitions.
> >
> >
> >
> https://github.com/gerritjvv/pseidon/blob/master/pseidon-kafka/java/pseidon/kafka/util/RoundRobinPartitioner.java
> >
> >
> >
> >
> >
> > On Tue, Dec 17, 2013 at 5:32 PM, Hanish Bansal <
> > hanish.bansal.agar...@gmail.com> wrote:
> >
> > > Hi All,
> > >
> > > We are having kafka cluster of 2 nodes. (using 0.8.0 final release)
> > > Replication Factor: 2
> > > Number of partitions: 2
> > >
> > > I have created a topic "test-topic1" in kafka.
> > >
> > > When i am listing status of that topic using bin/kafka-list-topic.sh,
> the
> > > status is:
> > >
> > > topic: test-topic1 partition: 0 leader: 0 replicas: 0,1
> > isr:
> > > 0,1
> > > topic: test-topic1 partition: 1 leader: 1 replicas: 1,0
> > isr:
> > > 1,0
> > >
> > > As both partition are on two separate nodes so when we produce the data
> > it
> > > should be go to both nodes.
> > >
> > > But when i insert the data, it is going to only one node.
> > >
> > > For example if i insert 1000 messages then all 1000 messages will go
> > either
> > > node1 or node2. Data is not evenly distributed on both nodes.
> > >
> > > Expected: 500 messages should go to node1 and 500 messages should go to
> > > node2.
> > >
> > > Any suggestion why i am facing this behavior?
> > >
> > > --
> > > *Thanks & Regards*
> > > *Hanish Bansal*
> > >
> >
>
>
>
> --
> -- Guozhang
>
--
*Thanks & Regards*
*Hanish Bansal*
