On Mon, May 6, 2019, at 07:39, Ismael Juma wrote: > Hi Colin, > > A quick comment. > > On Sat, May 4, 2019 at 11:18 PM Colin McCabe <cmcc...@apache.org> wrote: > > > The big advantage of doing batching on the controller is that the > > controller has more information about what is going on in the cluster. So > > it can schedule reassignments in a more optimal way. For instance, it can > > schedule reassignments so that the load is distributed evenly across > > nodes. This advantage is lost if we have to adhere to a rigid ordering > > that is set up in advance. We don't know exactly when anything will > > complete in any case. Just because one partition reassignment was started > > before another doesn't mean it will finish before another. > > > This is not quite true, right? The Controller doesn't know about partition > sizes, throughput per partition and other such information that external > tools like Cruise Control track.
Hi Ismael, That's a good point, and one I should have included. I guess when I think about "do batching in the controller" versus "do batching in an external system" I tend to think about the information the controller could theoretically collect, rather than what it actually does :) But certainly, adding this information to the controller would be a significant change, and maybe one we don't want to do if the external systems work well enough. Thinking about this a little bit more, I can see three advantages to controller-side batching. Firstly, doing batching in the controller saves memory because we don't use a separate JVM, and don't duplicate the in-memory map of all the partitions. Secondly, the information we're acting on would also be more up-to-date. (I'm not sure how important this would be.) Finally, it's one less thing to deploy. I don't know if those are really enough to motivate switching now, but in a greenfield system I would probably choose controller-side rebalancing. In any case, this KIP is orthogonal to controller-side rebalancing versus external rebalancing. That's why the KIP states that we will continue to perform all the given partition rebalances immediately. I was just responding to the idea that maybe we should have an "ordering" of rebalancing partitions. I don't think we want that, for controller-side rebalancing or externally batched rebalancing. best, Colin
