Hi Vahid, After re-thinking on this, i have following updates on the KIP, with aligning to the other options on ReassignPartitionsCommand,
The --execute command by default take rack awareness into consideration, and if the custom generated reassignment planner has conflicts along with the racks then it will throw the error msg with appropriate reason and conflict of partitions along with the racks info. The users need to explicitly choose the option --disable-rack-aware if they want to ignore the rack awareness. By this change the usage of options in --execute command will be aligned with --generate option, the rack awareness will be consider by default both for --generate as well as --execute unless explicitly set to --disable-rack-aware. Let me know whats your thoughts on the same. --sbellapu On 2019/11/22 16:32:08, Vahid Hashemian <vahid.hashem...@gmail.com> wrote: > Thanks Satish for drafting the KIP. Looks good overall. I would suggest > emphasizing on the default option for --disable-rack-aware option when used > with --execute option. > Also, it would be great to also emphasize that the new format for > --disable-rack-aware (which now takes a true/false value) would not impact > the existing usages (e.g. with --generate option) that did not require a > value for the option. > > Victor, to answer your first question, in my experience the assignment json > file is not always created by the same command (through --generate option): > > - Sometimes when a broker is not healthy we manually update the existing > assignment to change partition replicas to reduce load on the degraded > broker. > - In generating full partition assignment plan we also want use some > custom assignment strategy to have more control over partition placements > and do not use the default strategy used by Kafka. > > In these scenarios, it would be very helpful to have the option of > enforcing rack awareness with the command's --execute option. > > Regards, > --Vahid > > On Fri, Nov 22, 2019 at 2:57 AM Viktor Somogyi-Vass <viktorsomo...@gmail.com> > wrote: > > > Hi Satish, > > > > Couple of questions/suggestions: > > 1. You say that when you execute the planned reassignment then it would > > throw an error if the generated reassignment doesn't comply with the > > rack-aware requirement. Opposed to this: why don't you have the --generate > > option to generate a rack-aware reassignment plan? This way users won't > > have to do the extra round. > > 2. Please move your KIP under > > > > https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals > > , > > people will have a hard time finding it if it's under KIP-36. > > (@Stan fyi: > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-548+Add+Option+to+enforce+rack-aware+custom+partition+reassignment+execution > > ) > > > > Thanks, > > Viktor > > > > On Fri, Nov 22, 2019 at 11:37 AM Stanislav Kozlovski < > > stanis...@confluent.io> > > wrote: > > > > > Hello Satish, > > > > > > Could you provide a link to the KIP? I am unable to find it in the KIP > > > parent page > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals > > > > > > Thanks, > > > Stanislav > > > > > > On Fri, Nov 22, 2019 at 8:21 AM Satish Bellapu <satishbabu...@gmail.com> > > > wrote: > > > > > > > Hi All, > > > > > > > > This [KIP-548] is basically extending the capabilities of > > > > "kafka-reassign-partitions" tool by adding rack-aware verification > > option > > > > when used along with custom or manually generated reassignment planner > > > with > > > > --execute scenario. > > > > > > > > @sbellapu. > > > > > > > > > > > > > -- > > > Best, > > > Stanislav > > > > > > > > -- > > Thanks! > --Vahid >
