Hi Colin,
Certainly there will be some interaction and good idea with that you said,
I've added it to my KIP.
Will start a new discussion thread and link this one.
Viktor
On Wed, Jun 26, 2019 at 11:39 PM Colin McCabe <cmcc...@apache.org> wrote:
> Hi Viktor,
>
> Good point. Sorry, I should have read the KIP more closely.
>
> It would be good to change the title of the mail thread to reflect the new
> title of the KIP, "Internal Partition Reassignment Batching."
>
> I do think there will be some interaction with KIP-455 here. One example
> is that we'll want a way of knowing what target replicas are currently
> being worked on. So maybe we'll have to add a field to the structures
> returned by listPartitionReassignments.
>
> best,
> Colin
>
>
> On Wed, Jun 26, 2019, at 06:20, Viktor Somogyi-Vass wrote:
> > Hey Colin,
> >
> > I think there's some confusion here so I might change the name of this.
> So
> > KIP-435 is about the internal batching of reassignments (so purely a
> > controller change) and not about client side APIs. As per this moment
> these
> > kind of improvements are listed on KIP-455's future work section so in my
> > understanding KIP-455 won't touch that :).
> > Let me know if I'm missing any points here.
> >
> > Viktor
> >
> > On Tue, Jun 25, 2019 at 9:02 PM Colin McCabe <cmcc...@apache.org> wrote:
> >
> > > Hi Viktor,
> > >
> > > Now that the 2.3 release is over, we're going to be turning our
> attention
> > > back to working on KIP-455, which provides an API for partition
> > > reassignment, and also solves the incremental reassignment problem.
> Sorry
> > > about the pause, but I had to focus on the stuff that was going into
> 2.3.
> > >
> > > I think last time we talked about this, the consensus was that KIP-455
> > > supersedes KIP-435, since KIP-455 supports incremental reassignment.
> We
> > > also don't want to add more technical debt in the form of a new
> > > ZooKeeper-based API that we'll have to support for a while. So let's
> focus
> > > on KIP-455 here. We have more resources now so I think we'll be able
> to
> > > get it done soonish.
> > >
> > > best,
> > > Colin
> > >
> > >
> > > On Tue, Jun 25, 2019, at 08:09, Viktor Somogyi-Vass wrote:
> > > > Hi All,
> > > >
> > > > I have added another improvement to this, which is to limit the
> parallel
> > > > leader movements. I think I'll soon (maybe late this week or early
> next)
> > > > start a vote on this too if there are no additional feedback.
> > > >
> > > > Thanks,
> > > > Viktor
> > > >
> > > > On Mon, Apr 29, 2019 at 1:26 PM Viktor Somogyi-Vass <
> > > viktorsomo...@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi Folks,
> > > > >
> > > > > I've updated the KIP with the batching which would work on both
> replica
> > > > > and partition level. To explain it briefly: for instance if the
> replica
> > > > > level is set to 2 and partition level is set to 3, then 2x3=6
> replica
> > > > > reassignment would be in progress at the same time. In case of
> > > reassignment
> > > > > for a single partition from (0, 1, 2, 3, 4) to (5, 6, 7, 8, 9) we
> would
> > > > > form the batches (0, 1) → (5, 6); (2, 3) → (7, 8) and 4 → 9 and
> would
> > > > > execute the reassignment in this order.
> > > > >
> > > > > Let me know what you think.
> > > > >
> > > > > Best,
> > > > > Viktor
> > > > >
> > > > > On Mon, Apr 15, 2019 at 7:01 PM Viktor Somogyi-Vass <
> > > > > viktorsomo...@gmail.com> wrote:
> > > > >
> > > > >> A follow up on the batching topic to clarify my points above.
> > > > >>
> > > > >> Generally I think that batching should be a core feature as Colin
> said
> > > > >> the controller should possess all information that are related.
> > > > >> Also Cruise Control (or really any 3rd party admin system) might
> build
> > > > >> upon this to give more holistic approach to balance brokers. We
> may
> > > cater
> > > > >> them with APIs that act like building blocks to make their life
> > > easier like
> > > > >> incrementalization, batching, cancellation and rollback but I
> think
> > > the
> > > > >> more advanced we go we'll need more advanced control surface and
> > > Kafka's
> > > > >> basic tooling might not be suitable for that.
> > > > >>
> > > > >> Best,
> > > > >> Viktor
> > > > >>
> > > > >>
> > > > >> On Mon, 15 Apr 2019, 18:22 Viktor Somogyi-Vass, <
> > > viktorsomo...@gmail.com>
> > > > >> wrote:
> > > > >>
> > > > >>> Hey Guys,
> > > > >>>
> > > > >>> I'll reply to you all in this email:
> > > > >>>
> > > > >>> @Jun:
> > > > >>> 1. yes, it'd be a good idea to add this feature, I'll write this
> into
> > > > >>> the KIP. I was actually thinking about introducing a dynamic
> config
> > > called
> > > > >>> reassignment.parallel.partition.count and
> > > > >>> reassignment.parallel.replica.count. The first property would
> > > control how
> > > > >>> many partition reassignment can we do concurrently. The second
> would
> > > go one
> > > > >>> level in granularity and would control how many replicas do we
> want
> > > to move
> > > > >>> for a given partition. Also one more thing that'd be useful to
> fix
> > > is that
> > > > >>> a given list of partition -> replica list would be executed in
> the
> > > same
> > > > >>> order (from first to last) so it's overall predictable and the
> user
> > > would
> > > > >>> have some control over the order of reassignments should be
> > > specified as
> > > > >>> the JSON is still assembled by the user.
> > > > >>> 2. the /kafka/brokers/topics/{topic} znode to be specific. I'll
> > > update
> > > > >>> the KIP to contain this.
> > > > >>>
> > > > >>> @Jason:
> > > > >>> I think building this functionality into Kafka would definitely
> > > benefit
> > > > >>> all the users and that CC as well as it'd simplify their
> software as
> > > you
> > > > >>> said. As I understand the main advantage of CC and other similar
> > > softwares
> > > > >>> are to give high level features for automatic load balancing.
> > > Reliability,
> > > > >>> stability and predictability of the reassignment should be a core
> > > feature
> > > > >>> of Kafka. I think the incrementalization feature would make it
> more
> > > stable.
> > > > >>> I would consider cancellation too as a core feature and we can
> leave
> > > the
> > > > >>> gate open for external tools to feed in their reassignment json
> as
> > > they
> > > > >>> want. I was also thinking about what are the set of features we
> can
> > > provide
> > > > >>> for Kafka but I think the more advanced we go the more need
> there is
> > > for an
> > > > >>> administrative UI component.
> > > > >>> Regarding KIP-352: Thanks for pointing this out, I didn't see
> this
> > > > >>> although lately I was also thinking about the throttling aspect
> of
> > > it.
> > > > >>> Would be a nice add-on to Kafka since though the above configs
> > > provide some
> > > > >>> level of control, it'd be nice to put an upper cap on the
> bandwidth
> > > and
> > > > >>> make it monitorable.
> > > > >>>
> > > > >>> Viktor
> > > > >>>
> > > > >>> On Wed, Apr 10, 2019 at 2:57 AM Jason Gustafson <
> ja...@confluent.io>
> > > > >>> wrote:
> > > > >>>
> > > > >>>> Hi Colin,
> > > > >>>>
> > > > >>>> On a related note, what do you think about the idea of storing
> the
> > > > >>>> > reassigning replicas in
> > > > >>>> > /brokers/topics/[topic]/partitions/[partitionId]/state, rather
> > > than
> > > > >>>> in the
> > > > >>>> > reassignment znode? I don't think this requires a major
> change
> > > to the
> > > > >>>> > proposal-- when the controller becomes aware that it should
> do a
> > > > >>>> > reassignment, the controller could make the changes. This
> also
> > > helps
> > > > >>>> keep
> > > > >>>> > the reassignment znode from getting larger, which has been a
> > > problem.
> > > > >>>>
> > > > >>>>
> > > > >>>> Yeah, I think it's a good idea to store the reassignment state
> at a
> > > > >>>> finer
> > > > >>>> level. I'm not sure the LeaderAndIsr znode is the right one
> though.
> > > > >>>> Another
> > > > >>>> option is /brokers/topics/{topic}. That is where we currently
> store
> > > the
> > > > >>>> replica assignment. I think we basically want to represent both
> the
> > > > >>>> current
> > > > >>>> state and the desired state. This would also open the door to a
> > > cleaner
> > > > >>>> way
> > > > >>>> to update a reassignment while it is still in progress.
> > > > >>>>
> > > > >>>> -Jason
> > > > >>>>
> > > > >>>>
> > > > >>>>
> > > > >>>>
> > > > >>>> On Mon, Apr 8, 2019 at 11:14 PM George Li <
> sql_consult...@yahoo.com
> > > > >>>> .invalid>
> > > > >>>> wrote:
> > > > >>>>
> > > > >>>> > Hi Colin / Jason,
> > > > >>>> >
> > > > >>>> > Reassignment should really be doing a batches. I am not too
> > > worried
> > > > >>>> about
> > > > >>>> > reassignment znode getting larger. In a real production
> > > > >>>> environment, too
> > > > >>>> > many concurrent reassignment and too frequent submission of
> > > > >>>> reassignments
> > > > >>>> > seemed to cause latency spikes of kafka cluster. So
> > > > >>>> > batching/staggering/throttling of submitting reassignments is
> > > > >>>> recommended.
> > > > >>>> >
> > > > >>>> > In KIP-236, The "originalReplicas" are only kept for the
> current
> > > > >>>> > reassigning partitions (small #), and kept in memory of the
> > > controller
> > > > >>>> > context partitionsBeingReassigned as well as in the znode
> > > > >>>> > /admin/reassign_partitions, I think below "setting in the RPC
> > > like
> > > > >>>> null =
> > > > >>>> > no replicas are reassigning" is a good idea.
> > > > >>>> >
> > > > >>>> > There seems to be some issues with the Mail archive server of
> this
> > > > >>>> mailing
> > > > >>>> > list? I didn't receive email after April 7th, and the
> archive for
> > > > >>>> April
> > > > >>>> > 2019 has only 50 messages (
> > > > >>>> >
> > > http://mail-archives.apache.org/mod_mbox/kafka-dev/201904.mbox/thread)
> > > > >>>> ?
> > > > >>>> >
> > > > >>>> > Thanks,
> > > > >>>> > George
> > > > >>>> >
> > > > >>>> > on, 08 Apr 2019 17:54:48 GMT Colin McCabe wrote:
> > > > >>>> >
> > > > >>>> > Yeah, I think adding this information to LeaderAndIsr makes
> > > sense.
> > > > >>>> It
> > > > >>>> > would be better to track
> > > > >>>> > "reassigningReplicas" than "originalReplicas", I think.
> Tracking
> > > > >>>> > "originalReplicas" is going
> > > > >>>> > to involve sending a lot more data, since most replicas in the
> > > system
> > > > >>>> are
> > > > >>>> > not reassigning
> > > > >>>> > at any given point. Or we would need a hack in the RPC like
> null
> > > = no
> > > > >>>> > replicas are reassigning.
> > > > >>>> >
> > > > >>>> > On a related note, what do you think about the idea of
> storing the
> > > > >>>> > reassigning replicas in
> > > > >>>> > /brokers/topics/[topic]/partitions/[partitionId]/state,
> rather
> > > than
> > > > >>>> in
> > > > >>>> > the reassignment znode?
> > > > >>>> > I don't think this requires a major change to the proposal--
> > > when the
> > > > >>>> > controller becomes
> > > > >>>> > aware that it should do a reassignment, the controller could
> make
> > > the
> > > > >>>> > changes. This also
> > > > >>>> > helps keep the reassignment znode from getting larger, which
> has
> > > been
> > > > >>>> a
> > > > >>>> > problem.
> > > > >>>> >
> > > > >>>> > best,
> > > > >>>> > Colin
> > > > >>>> >
> > > > >>>> >
> > > > >>>> > On Mon, Apr 8, 2019, at 09:29, Jason Gustafson wrote:
> > > > >>>> > > Hey George,
> > > > >>>> > >
> > > > >>>> > > For the URP during a reassignment, if the
> "original_replicas"
> > > is
> > > > >>>> kept
> > > > >>>> > for
> > > > >>>> > > > the current pending reassignment. I think it will be very
> > > easy to
> > > > >>>> > compare
> > > > >>>> > > > that with the topic/partition's ISR. If all
> > > "original_replicas"
> > > > >>>> are in
> > > > >>>> > > > ISR, then URP should be 0 for that topic/partition.
> > > > >>>> > >
> > > > >>>> > >
> > > > >>>> > > Yeah, that makes sense. But I guess we would need
> > > > >>>> "original_replicas" to
> > > > >>>> > be
> > > > >>>> > > propagated to partition leaders in the LeaderAndIsr request
> > > since
> > > > >>>> leaders
> > > > >>>> > > are the ones that are computing URPs. That is basically what
> > > > >>>> KIP-352 had
> > > > >>>> > > proposed, but we also need the changes to the reassignment
> path.
> > > > >>>> Perhaps
> > > > >>>> > it
> > > > >>>> > > makes more sense to address this problem in KIP-236 since
> that
> > > is
> > > > >>>> where
> > > > >>>> > you
> > > > >>>> > > have already introduced "original_replicas"? I'm also happy
> to
> > > do
> > > > >>>> KIP-352
> > > > >>>> > > as a follow-up to KIP-236.
> > > > >>>> > >
> > > > >>>> > > Best,
> > > > >>>> > > Jason
> > > > >>>> > >
> > > > >>>> > >
> > > > >>>> > > On Sun, Apr 7, 2019 at 5:09 PM Ismael Juma <
> isma...@gmail.com>
> > > > >>>> wrote:
> > > > >>>> > >
> > > > >>>> > > > Good discussion about where we should do batching. I
> think if
> > > > >>>> there is
> > > > >>>> > a
> > > > >>>> > > > clear great way to batch, then it makes a lot of sense to
> > > just do
> > > > >>>> it
> > > > >>>> > once.
> > > > >>>> > > > However, if we think there is scope for experimenting with
> > > > >>>> different
> > > > >>>> > > > approaches, then an API that tools can use makes a lot of
> > > sense.
> > > > >>>> They
> > > > >>>> > can
> > > > >>>> > > > experiment and innovate. Eventually, we can integrate
> > > something
> > > > >>>> into
> > > > >>>> > Kafka
> > > > >>>> > > > if it makes sense.
> > > > >>>> > > >
> > > > >>>> > > > Ismael
> > > > >>>> > > >
> > > > >>>> > > > On Sun, Apr 7, 2019, 11:03 PM Colin McCabe <
> > > cmcc...@apache.org>
> > > > >>>> wrote:
> > > > >>>> > > >
> > > > >>>> > > > > Hi George,
> > > > >>>> > > > >
> > > > >>>> > > > > As Jason was saying, it seems like there are two
> directions
> > > we
> > > > >>>> could
> > > > >>>> > go
> > > > >>>> > > > > here: an external system handling batching, and the
> > > controller
> > > > >>>> > handling
> > > > >>>> > > > > batching. I think the controller handling batching
> would be
> > > > >>>> better,
> > > > >>>> > > > since
> > > > >>>> > > > > the controller has more information about the state of
> the
> > > > >>>> system.
> > > > >>>> > If
> > > > >>>> > > > the
> > > > >>>> > > > > controller handles batching, then the controller could
> also
> > > > >>>> handle
> > > > >>>> > things
> > > > >>>> > > > > like setting up replication quotas for individual
> > > partitions.
> > > > >>>> The
> > > > >>>> > > > > controller could do things like throttle replication
> down
> > > if the
> > > > >>>> > cluster
> > > > >>>> > > > > was having problems.
> > > > >>>> > > > >
> > > > >>>> > > > > We kind of need to figure out which way we're going to
> go on
> > > > >>>> this one
> > > > >>>> > > > > before we set up big new APIs, I think. If we want an
> > > external
> > > > >>>> > system to
> > > > >>>> > > > > handle batching, then we can keep the idea that there is
> > > only
> > > > >>>> one
> > > > >>>> > > > > reassignment in progress at once. If we want the
> > > controller to
> > > > >>>> > handle
> > > > >>>> > > > > batching, we will need to get away from that idea.
> > > Instead, we
> > > > >>>> > should
> > > > >>>> > > > just
> > > > >>>> > > > > have a bunch of "ideal assignments" that we tell the
> > > controller
> > > > >>>> > about,
> > > > >>>> > > > and
> > > > >>>> > > > > let it decide how to do the batching. These ideal
> > > assignments
> > > > >>>> could
> > > > >>>> > > > change
> > > > >>>> > > > > continuously over time, so from the admin's point of
> view,
> > > there
> > > > >>>> > would be
> > > > >>>> > > > > no start/stop/cancel, but just individual partition
> > > > >>>> reassignments
> > > > >>>> > that we
> > > > >>>> > > > > submit, perhaps over a long period of time. And then
> > > > >>>> cancellation
> > > > >>>> > might
> > > > >>>> > > > > just mean cancelling just that individual partition
> > > > >>>> reassignment,
> > > > >>>> > not all
> > > > >>>> > > > > partition reassignments.
> > > > >>>> > > > >
> > > > >>>> > > > > best,
> > > > >>>> > > > > Colin
> > > > >>>> > > > >
> > > > >>>> > > > > On Fri, Apr 5, 2019, at 19:34, George Li wrote:
> > > > >>>> > > > > > Hi Jason / Viktor,
> > > > >>>> > > > > >
> > > > >>>> > > > > > For the URP during a reassignment, if the
> > > > >>>> "original_replicas" is
> > > > >>>> > kept
> > > > >>>> > > > > > for the current pending reassignment. I think it will
> be
> > > very
> > > > >>>> easy
> > > > >>>> > to
> > > > >>>> > > > > > compare that with the topic/partition's ISR. If all
> > > > >>>> > > > > > "original_replicas" are in ISR, then URP should be 0
> for
> > > that
> > > > >>>> > > > > > topic/partition.
> > > > >>>> > > > > >
> > > > >>>> > > > > > It would be also nice to separate the metrics
> > > MaxLag/TotalLag
> > > > >>>> for
> > > > >>>> > > > > > Reassignments. I think that will also require
> > > > >>>> "original_replicas"
> > > > >>>> > (the
> > > > >>>> > > > > > topic/partition's replicas just before reassignment
> when
> > > the
> > > > >>>> AR
> > > > >>>> > > > > > (Assigned Replicas) is set to Set(original_replicas) +
> > > > >>>> > > > > > Set(new_replicas_in_reassign_partitions) ).
> > > > >>>> > > > > >
> > > > >>>> > > > > > Thanks,
> > > > >>>> > > > > > George
> > > > >>>> > > > > >
> > > > >>>> > > > > > On Friday, April 5, 2019, 6:29:55 PM PDT, Jason
> > > Gustafson
> > > > >>>> > > > > > <ja...@confluent.io> wrote:
> > > > >>>> > > > > >
> > > > >>>> > > > > > Hi Viktor,
> > > > >>>> > > > > >
> > > > >>>> > > > > > Thanks for writing this up. As far as questions about
> > > overlap
> > > > >>>> with
> > > > >>>> > > > > KIP-236,
> > > > >>>> > > > > > I agree it seems mostly orthogonal. I think KIP-236
> may
> > > have
> > > > >>>> had a
> > > > >>>> > > > larger
> > > > >>>> > > > > > initial scope, but now it focuses on cancellation and
> > > > >>>> batching is
> > > > >>>> > left
> > > > >>>> > > > > for
> > > > >>>> > > > > > future work.
> > > > >>>> > > > > >
> > > > >>>> > > > > > With that said, I think we may not actually need a KIP
> > > for the
> > > > >>>> > current
> > > > >>>> > > > > > proposal since it doesn't change any APIs. To make it
> more
> > > > >>>> > generally
> > > > >>>> > > > > > useful, however, it would be nice to handle batching
> at
> > > the
> > > > >>>> > partition
> > > > >>>> > > > > level
> > > > >>>> > > > > > as well as Jun suggests. The basic question is at what
> > > level
> > > > >>>> > should the
> > > > >>>> > > > > > batching be determined. You could rely on external
> > > processes
> > > > >>>> (e.g.
> > > > >>>> > > > cruise
> > > > >>>> > > > > > control) or it could be built into the controller.
> There
> > > are
> > > > >>>> > tradeoffs
> > > > >>>> > > > > > either way, but I think it simplifies such tools if
> it is
> > > > >>>> handled
> > > > >>>> > > > > > internally. Then it would be much safer to submit a
> larger
> > > > >>>> > reassignment
> > > > >>>> > > > > > even just using the simple tools that come with Kafka.
> > > > >>>> > > > > >
> > > > >>>> > > > > > By the way, since you are looking into some of the
> > > > >>>> reassignment
> > > > >>>> > logic,
> > > > >>>> > > > > > another problem that we might want to address is the
> > > > >>>> misleading
> > > > >>>> > way we
> > > > >>>> > > > > > report URPs during a reassignment. I had a naive
> proposal
> > > for
> > > > >>>> this
> > > > >>>> > > > > > previously, but it didn't really work
> > > > >>>> > > > > >
> > > > >>>> > > > >
> > > > >>>> > > >
> > > > >>>> >
> > > > >>>>
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-352%3A+Distinguish+URPs+caused+by+reassignment
> > > > >>>> > > > > .
> > > > >>>> > > > > > Potentially fixing that could fall under this work as
> > > well if
> > > > >>>> you
> > > > >>>> > think
> > > > >>>> > > > > > it
> > > > >>>> > > > > > makes sense.
> > > > >>>> > > > > >
> > > > >>>> > > > > > Best,
> > > > >>>> > > > > > Jason
> > > > >>>> > > > > >
> > > > >>>> > > > > > On Thu, Apr 4, 2019 at 4:49 PM Jun Rao <
> j...@confluent.io>
> > > > >>>> wrote:
> > > > >>>> > > > > >
> > > > >>>> > > > > > > Hi, Viktor,
> > > > >>>> > > > > > >
> > > > >>>> > > > > > > Thanks for the KIP. A couple of comments below.
> > > > >>>> > > > > > >
> > > > >>>> > > > > > > 1. Another potential thing to do reassignment
> > > incrementally
> > > > >>>> is to
> > > > >>>> > > > move
> > > > >>>> > > > > a
> > > > >>>> > > > > > > batch of partitions at a time, instead of all
> > > partitions.
> > > > >>>> This
> > > > >>>> > may
> > > > >>>> > > > > lead to
> > > > >>>> > > > > > > less data replication since by the time the first
> batch
> > > of
> > > > >>>> > partitions
> > > > >>>> > > > > have
> > > > >>>> > > > > > > been completely moved, some data of the next batch
> may
> > > have
> > > > >>>> been
> > > > >>>> > > > > deleted
> > > > >>>> > > > > > > due to retention and doesn't need to be replicated.
> > > > >>>> > > > > > >
> > > > >>>> > > > > > > 2. "Update CR in Zookeeper with TR for the given
> > > partition".
> > > > >>>> > Which
> > > > >>>> > ZK
> > > > >>>> > > > > path
> > > > >>>> > > > > > > is this for?
> > > > >>>> > > > > > >
> > > > >>>> > > > > > > Jun
> > > > >>>> > > > > > >
> > > > >>>> > > > > > > On Sat, Feb 23, 2019 at 2:12 AM Viktor Somogyi-Vass
> <
> > > > >>>> > > > > > > viktorsomo...@gmail.com>
> > > > >>>> > > > > > > wrote:
> > > > >>>> > > > > > >
> > > > >>>> > > > > > > > Hi Harsha,
> > > > >>>> > > > > > > >
> > > > >>>> > > > > > > > As far as I understand KIP-236 it's about enabling
> > > > >>>> reassignment
> > > > >>>> > > > > > > > cancellation and as a future plan providing a
> queue of
> > > > >>>> replica
> > > > >>>> > > > > > > reassignment
> > > > >>>> > > > > > > > steps to allow manual reassignment chains. While I
> > > agree
> > > > >>>> that
> > > > >>>> > the
> > > > >>>> > > > > > > > reassignment chain has a specific use case that
> allows
> > > > >>>> fine
> > > > >>>> > grain
> > > > >>>> > > > > control
> > > > >>>> > > > > > > > over reassignment process, My proposal on the
> other
> > > hand
> > > > >>>> > doesn't
> > > > >>>> > > > talk
> > > > >>>> > > > > > > about
> > > > >>>> > > > > > > > cancellation but it only provides an automatic
> way to
> > > > >>>> > > > incrementalize
> > > > >>>> > > > > an
> > > > >>>> > > > > > > > arbitrary reassignment which I think fits the
> general
> > > use
> > > > >>>> case
> > > > >>>> > > > where
> > > > >>>> > > > > > > users
> > > > >>>> > > > > > > > don't want that level of control but still would
> like
> > > a
> > > > >>>> > balanced
> > > > >>>> > > > way
> > > > >>>> > > > > of
> > > > >>>> > > > > > > > reassignments. Therefore I think it's still
> relevant
> > > as an
> > > > >>>> > > > > improvement of
> > > > >>>> > > > > > > > the current algorithm.
> > > > >>>> > > > > > > > Nevertheless I'm happy to add my ideas to KIP-236
> as I
> > > > >>>> think
> > > > >>>> > it
> > > > >>>> > > > > would be
> > > > >>>> > > > > > > a
> > > > >>>> > > > > > > > great improvement to Kafka.
> > > > >>>> > > > > > > >
> > > > >>>> > > > > > > > Cheers,
> > > > >>>> > > > > > > > Viktor
> > > > >>>> > > > > > > >
> > > > >>>> > > > > > > > On Fri, Feb 22, 2019 at 5:05 PM Harsha <
> > > ka...@harsha.io>
> > > > >>>> > wrote:
> > > > >>>> > > > > > > >
> > > > >>>> > > > > > > > > Hi Viktor,
> > > > >>>> > > > > > > > > There is already KIP-236 for the same
> > > feature
> > > > >>>> > and
> > > > >>>> > > > George
> > > > >>>> > > > > > > made
> > > > >>>> > > > > > > > > a PR for this as well.
> > > > >>>> > > > > > > > > Lets consolidate these two discussions. If you
> have
> > > any
> > > > >>>> > cases
> > > > >>>> > > > that
> > > > >>>> > > > > are
> > > > >>>> > > > > > > > not
> > > > >>>> > > > > > > > > being solved by KIP-236 can you please mention
> them
> > > in
> > > > >>>> > that
> > > > >>>> > > > > thread. We
> > > > >>>> > > > > > > > can
> > > > >>>> > > > > > > > > address as part of KIP-236.
> > > > >>>> > > > > > > > >
> > > > >>>> > > > > > > > > Thanks,
> > > > >>>> > > > > > > > > Harsha
> > > > >>>> > > > > > > > >
> > > > >>>> > > > > > > > > On Fri, Feb 22, 2019, at 5:44 AM, Viktor
> > > Somogyi-Vass
> > > > >>>> wrote:
> > > > >>>> > > > > > > > > > Hi Folks,
> > > > >>>> > > > > > > > > >
> > > > >>>> > > > > > > > > > I've created a KIP about an improvement of the
> > > > >>>> reassignment
> > > > >>>> > > > > algorithm
> > > > >>>> > > > > > > > we
> > > > >>>> > > > > > > > > > have. It aims to enable partition-wise
> incremental
> > > > >>>> > > > reassignment.
> > > > >>>> > > > > The
> > > > >>>> > > > > > > > > > motivation for this is to avoid excess load
> that
> > > the
> > > > >>>> > current
> > > > >>>> > > > > > > > replication
> > > > >>>> > > > > > > > > > algorithm implicitly carries as in that case
> there
> > > > >>>> > are points
> > > > >>>> > > > in
> > > > >>>> > > > > the
> > > > >>>> > > > > > > > > > algorithm where both the new and old replica
> set
> > > could
> > > > >>>> > be
> > > > >>>> > > > online
> > > > >>>> > > > > and
> > > > >>>> > > > > > > > > > replicating which puts double (or almost
> double)
> > > > >>>> pressure
> > > > >>>> > on
> > > > >>>> > > > the
> > > > >>>> > > > > > > > brokers
> > > > >>>> > > > > > > > > > which could cause problems.
> > > > >>>> > > > > > > > > > Instead my proposal would slice this up into
> > > several
> > > > >>>> > steps
> > > > >>>> > > > where
> > > > >>>> > > > > each
> > > > >>>> > > > > > > > > step
> > > > >>>> > > > > > > > > > is calculated based on the final target
> replicas
> > > and
> > > > >>>> > the
> > > > >>>> > > > current
> > > > >>>> > > > > > > > replica
> > > > >>>> > > > > > > > > > assignment taking into account scenarios where
> > > brokers
> > > > >>>> > could be
> > > > >>>> > > > > > > offline
> > > > >>>> > > > > > > > > and
> > > > >>>> > > > > > > > > > when there are not enough replicas to fulfil
> the
> > > > >>>> > > > > min.insync.replica
> > > > >>>> > > > > > > > > > requirement.
> > > > >>>> > > > > > > > > >
> > > > >>>> > > > > > > > > > The link to the KIP:
> > > > >>>> > > > > > > > > >
> > > > >>>> > > > > > > > >
> > > > >>>> > > > > > > >
> > > > >>>> > > > > > >
> > > > >>>> > > > >
> > > > >>>> > > >
> > > > >>>> >
> > > > >>>>
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-435%3A+Incremental+Partition+Reassignment
> > > > >>>> > > > > > > > > >
> > > > >>>> > > > > > > > > > I'd be happy to receive any feedback.
> > > > >>>> > > > > > > > > >
> > > > >>>> > > > > > > > > > An important note is that this KIP and another
> > > one,
> > > > >>>> > KIP-236
> > > > >>>> > > > that
> > > > >>>> > > > > is
> > > > >>>> > > > > > > > > > about
> > > > >>>> > > > > > > > > > interruptible reassignment (
> > > > >>>> > > > > > > > > >
> > > > >>>> > > > > > > > >
> > > > >>>> > > > > > > >
> > > > >>>> > > > > > >
> > > > >>>> > > > >
> > > > >>>> > > >
> > > > >>>> >
> > > > >>>>
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-236%3A+Interruptible+Partition+Reassignment
> > > > >>>> > > > > > > > > )
> > > > >>>> > > > > > > > > > should be compatible.
> > > > >>>> > > > > > > > > >
> > > > >>>> > > > > > > > > > Thanks,
> > > > >>>> > > > > > > > > > Viktor
> > > > >>>> > > > > > > > > >
> > > > >>>> > > > > > > > >
> > > > >>>> > > > > > > >
> > > > >>>> > > > > > >
> > > > >>>> > > > > >
> > > > >>>> > > > >
> > > > >>>> > > >
> > > > >>>> > >
> > > > >>>> >
> > > > >>>>
> > > > >>>
> > > >
> > >
> >
>
