Hi Tod,
I am not able to access the link
https://github.com/linkedin/kafka-toolskafka-assigner



Regards,
Abhimanyu

On Fri, Jun 16, 2017 at 12:26 AM, karan alang <karan.al...@gmail.com> wrote:

> Thanks Todd.. for the detailed reply.
>
> regds,
> Karan Alang
>
> On Tue, Jun 13, 2017 at 3:19 PM, Todd Palino <tpal...@gmail.com> wrote:
>
> > A few things here…
> >
> > 1) auto.leader.rebalance.enable can have serious performance impacts on
> > larger clusters. It’s currently in need of some development work to
> enable
> > it to batch leader elections into smaller groups and back off between
> them,
> > as well as have a better backoff after broker startup. I don’t recommend
> > using it.
> >
> > 2) auto.leader.rebalance.enable is not going to get you what you’re
> looking
> > for. It only changes the leader for a partition to the “optimal” leader
> (I
> > put that in quotes because it’s a pretty dumb algorithm. It’s whichever
> > replica is listed first). It does not move partitions around to assure
> you
> > have a balance of traffic across the cluster.
> >
> > If you want to rebalance partitions, you have a couple options right now:
> > 1) Run kafka-reassign-partitions.sh. It will move all of the partitions
> > around and try and assure an even count on each broker. It does not
> balance
> > traffic, however, (if you have a really busy partition and a really slow
> > partition, it considers them equal).
> > 2) Use an external tool like https://github.com/linkedin/kafka-tools
> > kafka-assigner. This is a script we developed at LinkedIn for doing
> > operations that involve moving partitions around and provides a number of
> > different ways to rebalance traffic.
> >
> > There are other tools available for doing this, but right now it requires
> > something external to the Apache Kafka core.
> >
> > -Todd
> >
> >
> > On Tue, Jun 13, 2017 at 5:30 PM, karan alang <karan.al...@gmail.com>
> > wrote:
> >
> > > Hi All,
> > >
> > > Fpr Re-balancing Kafka partitions, we can set property ->
> > >
> > >
> > > *auto.leader.rebalance.enable = true in server.properties file.*
> > >
> > > *Is that the recommended way or is it better to reBalance the kafka
> > > partitions manually ?(using *scripts - *kafka-preferred-replica-
> > > election.sh,
> > > *
> > >
> > > *kafka-reassign-partition.sh)*
> > > *One of the blogs mentioned that - it is preferable to Re-balance Kafka
> > > topics manually, since setting   *
> > >
> > > *auto.leader.rebalance.enable = true causes issues.*
> > >
> > > Pls let me know.
> > > Any other best practices wrt. Re-balancing kafka topics ?
> > >
> > > thanks!
> > >
> >
> >
> >
> > --
> > *Todd Palino*
> > Senior Staff Engineer, Site Reliability
> > Data Infrastructure Streaming
> >
> >
> >
> > linkedin.com/in/toddpalino
> >
>

Reply via email to