The consumer load balancing logic today is pretty simple. It just tries to divide the partitions evenly among the consumers. It doesn't try to balance by load.
Thanks, Jun On Thu, Nov 14, 2013 at 3:34 PM, hsy...@gmail.com <hsy...@gmail.com> wrote: > Hi, > > I have questions about the load balancing of kafka high-level consumer > Suppose I have 4 partition > And the producer throughput to these 4 partitions are like this > 0 1 2 3 > 10MB/s 10MB/s 1MB/s 1MB/s > 1kMsg/s, 10kMsg/s 1kMsg/s 10kMsg/s > > And I have only 2 consumers, what partitions will the 2 consumers pick up? > > If the throughput is changing dynamically, will the consumer re-balancing > accordingly? > > Thanks, > Siyuan >
