Hi Tod, I am not able to access the link https://github.com/linkedin/kafka-toolskafka-assigner
Regards, Abhimanyu On Fri, Jun 16, 2017 at 12:26 AM, karan alang <karan.al...@gmail.com> wrote: > Thanks Todd.. for the detailed reply. > > regds, > Karan Alang > > On Tue, Jun 13, 2017 at 3:19 PM, Todd Palino <tpal...@gmail.com> wrote: > > > A few things here… > > > > 1) auto.leader.rebalance.enable can have serious performance impacts on > > larger clusters. It’s currently in need of some development work to > enable > > it to batch leader elections into smaller groups and back off between > them, > > as well as have a better backoff after broker startup. I don’t recommend > > using it. > > > > 2) auto.leader.rebalance.enable is not going to get you what you’re > looking > > for. It only changes the leader for a partition to the “optimal” leader > (I > > put that in quotes because it’s a pretty dumb algorithm. It’s whichever > > replica is listed first). It does not move partitions around to assure > you > > have a balance of traffic across the cluster. > > > > If you want to rebalance partitions, you have a couple options right now: > > 1) Run kafka-reassign-partitions.sh. It will move all of the partitions > > around and try and assure an even count on each broker. It does not > balance > > traffic, however, (if you have a really busy partition and a really slow > > partition, it considers them equal). > > 2) Use an external tool like https://github.com/linkedin/kafka-tools > > kafka-assigner. This is a script we developed at LinkedIn for doing > > operations that involve moving partitions around and provides a number of > > different ways to rebalance traffic. > > > > There are other tools available for doing this, but right now it requires > > something external to the Apache Kafka core. > > > > -Todd > > > > > > On Tue, Jun 13, 2017 at 5:30 PM, karan alang <karan.al...@gmail.com> > > wrote: > > > > > Hi All, > > > > > > Fpr Re-balancing Kafka partitions, we can set property -> > > > > > > > > > *auto.leader.rebalance.enable = true in server.properties file.* > > > > > > *Is that the recommended way or is it better to reBalance the kafka > > > partitions manually ?(using *scripts - *kafka-preferred-replica- > > > election.sh, > > > * > > > > > > *kafka-reassign-partition.sh)* > > > *One of the blogs mentioned that - it is preferable to Re-balance Kafka > > > topics manually, since setting * > > > > > > *auto.leader.rebalance.enable = true causes issues.* > > > > > > Pls let me know. > > > Any other best practices wrt. Re-balancing kafka topics ? > > > > > > thanks! > > > > > > > > > > > -- > > *Todd Palino* > > Senior Staff Engineer, Site Reliability > > Data Infrastructure Streaming > > > > > > > > linkedin.com/in/toddpalino > > >
