Hi Satish, Thanks for the update. What you suggest in the KIP is ideal, the only issue is the --execute option will not be backward compatible; so the same command that used to work before, may no longer work if this suggestion is implemented. One option for keeping backward compatibility would be using a new option, such as --enable-rack-aware that either works only with --execute, or works with other options too. This would not necessary be the best option, but at least it would not be a breaking change.
Let's see if others have better ideas. Thanks, --Vahid On Fri, Nov 22, 2019 at 5:59 PM Satish Bellapu <satishbabu...@gmail.com> wrote: > Hi Vahid, > After re-thinking on this, i have following updates on the KIP, with > aligning to the other options on ReassignPartitionsCommand, > > The --execute command by default take rack awareness into consideration, > and if the custom generated reassignment planner has conflicts along with > the racks then it will throw the error msg with appropriate reason and > conflict of partitions along with the racks info. The users need to > explicitly choose the option --disable-rack-aware if they want to ignore > the rack awareness. > > By this change the usage of options in --execute command will be aligned > with --generate option, the rack awareness will be consider by default both > for --generate as well as --execute unless explicitly set to > --disable-rack-aware. > > Let me know whats your thoughts on the same. > > --sbellapu > > On 2019/11/22 16:32:08, Vahid Hashemian <vahid.hashem...@gmail.com> > wrote: > > Thanks Satish for drafting the KIP. Looks good overall. I would suggest > > emphasizing on the default option for --disable-rack-aware option when > used > > with --execute option. > > Also, it would be great to also emphasize that the new format for > > --disable-rack-aware (which now takes a true/false value) would not > impact > > the existing usages (e.g. with --generate option) that did not require a > > value for the option. > > > > Victor, to answer your first question, in my experience the assignment > json > > file is not always created by the same command (through --generate > option): > > > > - Sometimes when a broker is not healthy we manually update the > existing > > assignment to change partition replicas to reduce load on the degraded > > broker. > > - In generating full partition assignment plan we also want use some > > custom assignment strategy to have more control over partition > placements > > and do not use the default strategy used by Kafka. > > > > In these scenarios, it would be very helpful to have the option of > > enforcing rack awareness with the command's --execute option. > > > > Regards, > > --Vahid > > > > On Fri, Nov 22, 2019 at 2:57 AM Viktor Somogyi-Vass < > viktorsomo...@gmail.com> > > wrote: > > > > > Hi Satish, > > > > > > Couple of questions/suggestions: > > > 1. You say that when you execute the planned reassignment then it would > > > throw an error if the generated reassignment doesn't comply with the > > > rack-aware requirement. Opposed to this: why don't you have the > --generate > > > option to generate a rack-aware reassignment plan? This way users won't > > > have to do the extra round. > > > 2. Please move your KIP under > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals > > > , > > > people will have a hard time finding it if it's under KIP-36. > > > (@Stan fyi: > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-548+Add+Option+to+enforce+rack-aware+custom+partition+reassignment+execution > > > ) > > > > > > Thanks, > > > Viktor > > > > > > On Fri, Nov 22, 2019 at 11:37 AM Stanislav Kozlovski < > > > stanis...@confluent.io> > > > wrote: > > > > > > > Hello Satish, > > > > > > > > Could you provide a link to the KIP? I am unable to find it in the > KIP > > > > parent page > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals > > > > > > > > Thanks, > > > > Stanislav > > > > > > > > On Fri, Nov 22, 2019 at 8:21 AM Satish Bellapu < > satishbabu...@gmail.com> > > > > wrote: > > > > > > > > > Hi All, > > > > > > > > > > This [KIP-548] is basically extending the capabilities of > > > > > "kafka-reassign-partitions" tool by adding rack-aware verification > > > option > > > > > when used along with custom or manually generated reassignment > planner > > > > with > > > > > --execute scenario. > > > > > > > > > > @sbellapu. > > > > > > > > > > > > > > > > > -- > > > > Best, > > > > Stanislav > > > > > > > > > > > > > -- > > > > Thanks! > > --Vahid > > > -- Thanks! --Vahid
