In your instance if you have four JVMs (i.e., consumer processes), six threads per consumer process and 12 partitions, then each thread would only get one partition but the first two processes will get all the partitions and the last two processes would be idle. We could tweak the assignment strategy to sort the consumer threads differently to get a better spread across _processes_. E.g., the current sort order is: c0-0, c0-1, c0-2,..., c1-0, c1-1, ...
If we change that to (say) c0-0, c1-0, c2-0, ..., c0-1, c1-1, c2-1, ... It would in fact spread the partitions across processes evenly. Alternatively you could increase the number of partitions or reduce the number of threads per process. I think in your earlier example you had 32 partitions. In this case you would get a better balance. On Thu, Oct 30, 2014 at 08:29:28PM -0700, Bhavesh Mistry wrote: > Hi Joel, > > Yes, I am on Kafka Trunk branch. In my scenario, if you have back-up > threads does that impact the allocation. If I have 24 threads (6 thread > for each JVM total of 4 JVMS) in above example , does partition allocation > gets evenly distributed (3 on each JVM) ? is this supported use case ? > > Thanks, > > Bhavesh
