for point 1 i agree, its not too strong. only addition i could come up with is that it allows any utility to have better forwards compatability. a cli written that can inspect how a topic will be created would be able to give insight/expectations about configs that didn't exist at compilation time.
for point 2, i am thinking about a situation where the user creating topics and the user managing brokers are different people with different skills/abilities. so in the given example lets assume a user creating the topic does not *really* understand what that config means, so they are likely to just leave it as default. and are happy for their admin to change these on the broker as needed. but say we have another user who is creating a topic who has much more experience and has done testing, they will be able to determine what the value is on the cluster and check to see if it matches expectations or needs to be set. possibly if this is set to something incorrect for their usecase they will have a reason to verify and speak with the admin about their usecase. moreover, i think without this added capability it makes it difficult/impossible to accurately use broker defaults for topics. right now a user is left to either decide configs a priori and lose this functionality, or guess/check what they need to set and end in a possibly bad situation until they can get their *live* topic configured. dan On Mon, Dec 4, 2017 at 2:50 PM, Dong Lin <lindon...@gmail.com> wrote: > Hey Dan, > > Thanks again for the update:) > > I am not sure I fully understand the points (1) and (2) in the "Always > Configure ALL Configs For a Topic". In my previous question, I don't mean > that users should specify full list of topics configs. I mean that user can > specify the full list of topic configs he/she wants to override. > > Your KIP proposes to allow user to query the default topic configs. In my > understanding, users want to know the default configs only if he/she > already has a specific list of (config, value) pairs for which he/she wants > to override. The workflow will be that user queries the default configs, > compares the default value with that specific list, and selectively > override the configs whose value is different from the default value. > Therefore, the alternative solution does not suffer the problem (1) because > user needs to know that specific list of (config, value) pair anyway. Does > this make sense? > > Regarding problem (2), I think you are saying that if the user wants to set > the min.cleanable.dirty.ratio to be 0.5 and the default value is currently > set to 0.5, then user would not want to explicitly override the config > during topic creation. The purpose is for the min.cleanable.dirty.ratio of > this topic to be changed to 0.6 if admin change the default > min.cleanable.dirty.ratio to 0.6 in the future. > > But I am not sure this is a reasonable example. If user wants to > compare default value of min.cleanable.dirty.ratio with its expected value > 0.5, then it seems reasonable for user to override it to be 0.5 during > topic creation if the default value is currently 0.6. The question is, why > would user not want to override the default value to be 0.5 if the default > value is changed from 0.5 to 0.6 later? > > Thanks, > Dong > > > On Mon, Dec 4, 2017 at 2:36 PM, dan <dan.norw...@gmail.com> wrote: > > > updated again :) > > > > by having users always set all configs you lose the ability to use the > > broker defaults as intended, since topic configs are overlaid. example in > > the kip doc. > > > > dan > > > > On Mon, Dec 4, 2017 at 11:47 AM, Dong Lin <lindon...@gmail.com> wrote: > > > > > Hey Dan, > > > > > > Thanks for the update. I just want to push the discussion a bit > further. > > > Another alternative, which currently is not described in the KIP, is > for > > > user to always create the topic with the full list of configs it may > want > > > to override. Can you help me understand what is the drawback of this > > > approach? > > > > > > Thanks, > > > Dong > > > > > > On Mon, Dec 4, 2017 at 11:30 AM, dan <dan.norw...@gmail.com> wrote: > > > > > > > Dong, > > > > > > > > i added a section on current state and workarounds along with my > > > arguments > > > > for why they are less than optimal to the wiki. but the jist of it is > > you > > > > can end up with messages in your topic in an incorrect/invalid state > if > > > you > > > > do this. > > > > > > > > thanks, > > > > dan > > > > > > > > On Mon, Dec 4, 2017 at 10:53 AM, Dong Lin <lindon...@gmail.com> > wrote: > > > > > > > > > Hey Dan, > > > > > > > > > > Thanks for the KIP. Can you help me understand the motivation by > > > > providing > > > > > a use-case that can not be easily completed without this KIP? > > > > > > > > > > It seems that most users will simply create the topic without > > worrying > > > > > about the default configs. If a user has specific requirement for > the > > > > > default configs, he/she can use AdminClient.describeConfigs() and > > > > > AdminClient.alterConfigs() after the topic has been created. This > > seems > > > > to > > > > > work well. Did I miss something? > > > > > > > > > > Thanks, > > > > > Dong > > > > > > > > > > > > > > > On Mon, Dec 4, 2017 at 9:25 AM, dan <dan.norw...@gmail.com> wrote: > > > > > > > > > > > I would like to start a discussion about KIP-234 > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > > > > > > 234%3A+add+support+for+getting+topic+defaults+from+AdminClient > > > > > > > > > > > > thanks > > > > > > dan > > > > > > > > > > > > > > > > > > > > >
