Thanks Mayuresh, Can I at least control the rebalance of consumers? Currently consumes die after specific partition has no more messages, and there is rebalance of consumes triggered, which causes more consumers to die who get assigned to empty partition(because zookeeper treat empty partition no differently).
Is there way I can control so that there is no rebalancing of consumers when some consumer die? Thanks -Pranay On Tue, Jul 21, 2015 at 11:25 AM, Mayuresh Gharat < gharatmayures...@gmail.com> wrote: > Not sure if you can do that with High level consumer. > > Thanks, > > Mayuresh > > On Tue, Jul 21, 2015 at 10:53 AM, Pranay Agarwal <agarwalpran...@gmail.com > > > wrote: > > > Any ideas? > > > > On Mon, Jul 20, 2015 at 2:34 PM, Pranay Agarwal < > agarwalpran...@gmail.com> > > wrote: > > > > > Hi all, > > > > > > Is there any way I can force Zookeeper/Kafka to rebalance new consumers > > > only for subset of total number of partitions. I have a situation where > > out > > > of 120 partitions 60 have been already consumed, but the zookeeper also > > > assigns these empty/inactive partitions as well for the re-balancing, I > > > want my resources to be used only for the partitions which still have > > some > > > messages left to read. > > > > > > Thanks > > > -Pranay > > > > > > > > > -- > -Regards, > Mayuresh R. Gharat > (862) 250-7125 >
