Re: Random Partitioning Issue

2013-09-14 Thread Joel Koshy
> > > Thanks for bringing this up - it is definitely an important point to > discuss. The underlying issue of KAFKA-1017 was uncovered to some degree by > the fact that in our deployment we did not significantly increase the total > number of partitions over 0.7 - i.e., in 0.7 we had say four parti

Re: Random Partitioning Issue

2013-09-14 Thread Joe Stein
How about creating a new class called RandomRefreshPartioner and copy the DefaultPartitioner code to it and then revert the DefaultPartitioner code. I appreciate this is a one time burden for folks using the existing 0.8-beta1 bumping into KAFKA-1017 in production having to switch to the Rando

Re: Review Request 14091: Patch for KAFKA-1053

2013-09-14 Thread Swapnil Ghike
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/14091/#review26115 --- kafka-patch-review.py

Re: Random Partitioning Issue

2013-09-14 Thread Jun Rao
Joe, Thanks for bringing this up. I want to clarify this a bit. 1. Currently, the producer side logic is that if the partitioning key is not provided (i.e., it is null), the partitioner won't be called. We did that because we want to select a random and "available" partition to send messages so t