I will go back through the ticket and code and write more up. Should be able to-do that sometime next week. The intention was to not replace existing functionality by issue a WARN on use. The following version it is released we could then deprecate it... I will fix the KIP for that too.
On Fri, Jan 23, 2015 at 12:34 AM, Neha Narkhede <n...@confluent.io> wrote: > Hey Joe, > > 1. Could you add details to the Public Interface section of the KIP? This > should include the proposed changes to the partition reassignment tool. > Also, maybe the new option can be named --rebalance instead of > --re-balance? > 2. It makes sense to list --decommission-broker as part of this KIP. > Similarly, shouldn't we also have an --add-broker option? The way I see > this is that there are several events when a partition reassignment is > required. Before this functionality is automated on the broker, the tool > will generate an ideal replica placement for each such event. The users > should merely have to specify the nature of the event e.g. adding a broker > or decommissioning an existing broker or merely rebalancing. > 3. If I understand the KIP correctly, the upgrade plan for this feature > includes removing the existing --generate option on the partition > reassignment tool in 0.8.3 while adding all the new options in the same > release. Is that correct? > > Thanks, > Neha > > On Thu, Jan 22, 2015 at 9:23 PM, Jay Kreps <jay.kr...@gmail.com> wrote: > > > Ditto on this one. Can you give the algorithm we want to implement? > > > > Also I think in terms of scope this is just proposing to change the logic > > in ReassignPartitionsCommand? I think we've had the discussion various > > times on the mailing list that what people really want is just for Kafka > to > > do it's best to balance data in an online fashion (for some definition of > > balance). i.e. if you add a new node partitions would slowly migrate to > it, > > and if a node dies, partitions slowly migrate off it. This could > > potentially be more work, but I'm not sure how much more. Has anyone > > thought about how to do it? > > > > -Jay > > > > On Wed, Jan 21, 2015 at 10:11 PM, Joe Stein <joe.st...@stealth.ly> > wrote: > > > > > Posted a KIP for --re-balance for partition assignment in reassignment > > > tool. > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-6+-+New+reassignment+partition+logic+for+re-balancing > > > > > > JIRA https://issues.apache.org/jira/browse/KAFKA-1792 > > > > > > While going through the KIP I thought of one thing from the JIRA that > we > > > should change. We should preserve --generate to be existing > functionality > > > for the next release it is in. If folks want to use --re-balance then > > > great, it just won't break any upgrade paths, yet. > > > > > > /******************************************* > > > Joe Stein > > > Founder, Principal Consultant > > > Big Data Open Source Security LLC > > > http://www.stealth.ly > > > Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop> > > > ********************************************/ > > > > > > > > > -- > Thanks, > Neha >
