Hey there Mickael, thanks for the KIP. I think the proposal makes sense. My initial concern was that we may not want to allow this at all if the cluster is in an unhealthy state. After thinking about it, I don't see it hurting at all - we would already have topics that are under-replicated so creating new ones split across the same brokers (one of them presumably being offline) sounds reasonable. Maybe it would be good to add a bit more detail in the Motivation section, specifically cases where this feature would prove useful.
In terms of adding a new field to the CreateTopics/Partitions request, have we considered adding the `min_required_replication_factor` as a topic config? That way we could store it in the `config_entries` field and the CreatePartitions request could check the topic's config instead of explicitly setting the field. Could you help clarify the three `under.replicated.creation.policy` options for me please? disabled - how it currently works, no topic gets created if there arent enough brokers enabled - if automatic/manual assignment chooses an offline, observed broker - we allow it enabled_prefer_observed - automatic assignment will prefer offline, observed brokers? That is, we want to make sure that the replica assignment here <https://github.com/apache/kafka/blob/1fa02d5aef1febfc7258b88385545463386634a8/core/src/main/scala/kafka/admin/AdminUtils.scala#L129> knows how to deal with observed brokers? > enabled_prefer_observed - When enabled a sub-option is to always prefer using all known brokers and their racks and have some under replication I may be missing something, but it seems like we can achieve the desired behavior with two options only. If it is using the rack-aware assignment, it can choose offline brokers depending on whether the ` under.replicated.creation.policy` is enabled or disabled only it seems like. Is that correct? Thanks, Stanislav On Wed, Jan 30, 2019 at 2:24 AM Mickael Maison <mickael.mai...@gmail.com> wrote: > Another bump. > Feedback and suggestions are always appreciated! > > On Tue, Jan 8, 2019 at 5:14 PM Mickael Maison <mickael.mai...@gmail.com> > wrote: > > > > Hi, > > > > We've not received any feedback yet on this KIP. We still believe this > > would be a nice improvement. > > > > Thanks > > > > On Tue, Dec 18, 2018 at 4:27 PM Mickael Maison <mickael.mai...@gmail.com> > wrote: > > > > > > Hi, > > > > > > We have submitted a KIP to handle topics and partitions creation when > > > a cluster is not fully available: > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-409%3A+Allow+creating+under-replicated+topics+and+partitions > > > > > > As always, we welcome feedback and suggestions. > > > > > > Thanks > > > Mickael and Edoardo > -- Best, Stanislav
