Re: Reg. Partition Rebalancing

2016-03-31 Thread Srikanth Chandika
Thanks Todd for the valuable information. Partition re-balance: I am testing a scenario (1 topic with 8 partitions, 3 replicas with 3 brokers) where I brought down one broker2 acting as a leader to partitions then immediately leaders and isr's have changed to 2 live brokers but not evenly balance

Re: Reg. Partition Rebalancing

2016-03-29 Thread Todd Palino
There’s no data moved when a PLE occurs, so that is not the problem here. The issue is that the controller reelects all partitions at the same time. This forces all the replica fetchers on the broker to stop and restart (for the partitions that have changed leaders) at the same time. We have seen,

Re: Reg. Partition Rebalancing

2016-03-29 Thread James Cheng
> On Mar 29, 2016, at 10:33 AM, Todd Palino wrote: > > There’s two things that people usually mean when they talk about > rebalancing. > > One is leader reelection, or preferred replica election, which is sometimes > confusingly referred to as “leader rebalance”. This is when we ask the > control

Re: Reg. Partition Rebalancing

2016-03-29 Thread Rick Mangi
Of possible interest, we just released a script that we use to pick partition assignments. https://github.com/chartbeat-labs/trepl > On Mar 29, 2016, at 1:33 PM, Todd Palino wrote: > > There’s two things that people usually mean when they talk about > rebalancing. > > One is leader reelectio

Re: Reg. Partition Rebalancing

2016-03-29 Thread Todd Palino
There’s two things that people usually mean when they talk about rebalancing. One is leader reelection, or preferred replica election, which is sometimes confusingly referred to as “leader rebalance”. This is when we ask the controller in the cluster to pick the preferred replica for all partition

Reg. Partition Rebalancing

2016-03-29 Thread Srikanth Chandika
Hi, I am new to kafka I am testing all the options in kafka. I am confused about the re-balancing? How and where to configure the re-balancing option? Regards, Srikanth