On Fri, Jan 11, 2013 at 12:37 AM, Jun Rao <jun...@gmail.com> wrote: > Our current partitioning strategy is to mod key by # of partitions, not # > brokers. For better balancing partitions over brokers, one simple strategy > is to over partition, i.e., you have a few times of more partitions than > brokers. That way, if one adds more brokers overtime, you can just move > some existing partitions to the new broker. > > Consistent hashing requires changing # partitions dynamically.However, for > some applications, they may prefer not to change partitions. >
> What's your use case for consistent hashing? > My use case is essentially the same as above, i.e., dynamic load balancing. I now understand why the current partitioning strategy is used as opposed to consistent hashing; partition "stickiness" is definitely to be desired for the sake of moving computation to data. However, the dynamic rebalancing as described in "Kafka Replication Design", sect. 1.2 looks very similar to what's typically achieved by using consistent hashing. Is this rebalancing implemented in 0.8 or am I reading the now obsolete documentation? :) (If yes, could you please refer me to the code.) Thanks, stan
