Re: [DISCUSS] KIP-574: CLI Dynamic Configuration with file input

2020-04-13 Thread Aneel Nazareth
Thanks to David Jacot for his suggestions. Would it be possible for a committer to have a look at this PR? Thanks! https://github.com/apache/kafka/pull/8184 On Wed, Apr 1, 2020 at 12:30 PM Aneel Nazareth wrote: > > I have a PR that I think is ready for review here: > https://github.com/apache/ka

Re: [DISCUSS] KIP-574: CLI Dynamic Configuration with file input

2020-04-01 Thread Aneel Nazareth
I have a PR that I think is ready for review here: https://github.com/apache/kafka/pull/8184 Feedback would be most welcome! On Mon, Mar 30, 2020 at 6:57 PM Colin McCabe wrote: > > Just as a note about whether we should support "-" as a synonym for STDIN... > I agree it's kind of inconsistent.

Re: [DISCUSS] KIP-574: CLI Dynamic Configuration with file input

2020-03-30 Thread Colin McCabe
Just as a note about whether we should support "-" as a synonym for STDIN... I agree it's kind of inconsistent. It may not be that big of a deal to drop support for STDIN. A lot of UNIX shells make it easy to create a temporary file in scripts-- for example, you could use a "here document" (

Re: [DISCUSS] KIP-574: CLI Dynamic Configuration with file input

2020-03-27 Thread Aneel Nazareth
Update: I have simplified the KIP down to just adding the single new --add-config-file option. Thanks for your input, everyone! On Thu, Mar 26, 2020 at 10:13 AM Aneel Nazareth wrote: > > Hi Kamal, > > Thanks for taking a look at this KIP. > > Unfortunately the user actually can't pass the argumen

Re: [DISCUSS] KIP-574: CLI Dynamic Configuration with file input

2020-03-26 Thread Aneel Nazareth
Hi Rajini, Thanks for taking a look at this. It seems like the consensus is to remove the --delete-config-file option, so I'll do so. On Wed, Mar 25, 2020 at 5:48 AM Rajini Sivaram wrote: > > Hi Aneel, > > Thanks for the KIP. As configurations get more complex, the ability to > provide compound

Re: [DISCUSS] KIP-574: CLI Dynamic Configuration with file input

2020-03-26 Thread Aneel Nazareth
Hi Kamal, Thanks for taking a look at this KIP. Unfortunately the user actually can't pass the arguments on the command line using the existing --add-config option if the values are complex structures that contain commas. --add-config assumes that commas separate distinct configuration properties

Re: [DISCUSS] KIP-574: CLI Dynamic Configuration with file input

2020-03-26 Thread Kamal Chandraprakash
Hi Colin, We should not support STDIN to maintain uniformity across scripts. If the user wants to pass the arguments in command line, they can always use the existing --add-config option. On Thu, Mar 26, 2020 at 7:20 PM David Jacot wrote: > Rajini has made a good point. I don't feel strong f

Re: [DISCUSS] KIP-574: CLI Dynamic Configuration with file input

2020-03-26 Thread David Jacot
Rajini has made a good point. I don't feel strong for either ways but if people are confused by this, it is probably better without it. Best, David On Thu, Mar 26, 2020 at 7:23 AM Colin McCabe wrote: > Hi Kamal, > > Are you suggesting that we not support STDIN here? I have mixed feelings. > >

Re: [DISCUSS] KIP-574: CLI Dynamic Configuration with file input

2020-03-25 Thread Colin McCabe
Hi Kamal, Are you suggesting that we not support STDIN here? I have mixed feelings. I think the ideal solution would be to support "-" in these tools whenever a file argument was expected. But that would be a bigger change than what we're talking about here. Maybe you are right and we should

Re: [DISCUSS] KIP-574: CLI Dynamic Configuration with file input

2020-03-25 Thread Colin McCabe
I agree with this. I think we'd be better off without the --delete-config-file option. It just seems confusing. best, Colin On Wed, Mar 25, 2020, at 03:48, Rajini Sivaram wrote: > Hi Aneel, > > Thanks for the KIP. As configurations get more complex, the ability to > provide compound configs i

Re: [DISCUSS] KIP-574: CLI Dynamic Configuration with file input

2020-03-25 Thread Rajini Sivaram
Hi Aneel, Thanks for the KIP. As configurations get more complex, the ability to provide compound configs in a file is really useful. I am not convinced about the `--delete-config-file` option though. I am not familiar with the Kubernetes case, but I guess if you create an entity with a file, it i

Re: [DISCUSS] KIP-574: CLI Dynamic Configuration with file input

2020-03-25 Thread Kamal Chandraprakash
STDIN wasn't standard practice in other scripts like kafka-console-consumer.sh, kafka-console-producer.sh and kafka-acls.sh in which the props file is accepted via consumer.config / producer.config / command-config parameter. Shouldn't we have to maintain the uniformity across scripts? On Mon, Ma

Re: [DISCUSS] KIP-574: CLI Dynamic Configuration with file input

2020-03-16 Thread David Jacot
Hi Aneel, Thanks for the updated KIP. I have made a second pass over it and the KIP looks good to me. Best, David On Tue, Mar 10, 2020 at 9:39 PM Aneel Nazareth wrote: > After reading a bit more about it in the Kubernetes case, I think it's > reasonable to do this and be explicit that we're ig

Re: [DISCUSS] KIP-574: CLI Dynamic Configuration with file input

2020-03-10 Thread Aneel Nazareth
After reading a bit more about it in the Kubernetes case, I think it's reasonable to do this and be explicit that we're ignoring the value, just deleting all keys that appear in the file. I've updated the KIP wiki page to reflect that: https://cwiki.apache.org/confluence/display/KAFKA/KIP-574%3A+C

Re: [DISCUSS] KIP-574: CLI Dynamic Configuration with file input

2020-03-09 Thread Aneel Nazareth
Hi Colin, Thanks for the suggestion. Using a dash does seem reasonable. I can make that change. On Mon, Mar 9, 2020 at 12:35 PM Colin McCabe wrote: > > Hi Aneel, > > Thanks for the KIP. I like the idea. > > You mention that "input from STDIN can be used instead of a file on disk." > The examp

Re: [DISCUSS] KIP-574: CLI Dynamic Configuration with file input

2020-03-09 Thread Aneel Nazareth
Hi David, Is the expected behavior that the keys are deleted without checking the values? Let's say I had this file new.properties: a=1 b=2 And ran: bin/kafka-configs --bootstrap-server localhost:9092 \ --entity-type brokers --entity-default \ --alter --add-config-file new.properties It se

Re: [DISCUSS] KIP-574: CLI Dynamic Configuration with file input

2020-03-09 Thread David Jacot
Hi Colin, Yes, you're right. This is weird but convenient because you don't have to duplicate the "keys". I was thinking about the kubernetes API which allows to create a Pod based on a file and allows to delete it as well with the same file. I have always found this convenient, especially when do

Re: [DISCUSS] KIP-574: CLI Dynamic Configuration with file input

2020-03-09 Thread Colin McCabe
Hi Aneel, Thanks for the KIP. I like the idea. You mention that "input from STDIN can be used instead of a file on disk." The example given in the KIP seems to suggest that the command defaults to reading from STDIN if no argument is given to --add-config-file. I would argue against this par

Re: [DISCUSS] KIP-574: CLI Dynamic Configuration with file input

2020-03-09 Thread David Jacot
Hi Aneel, Thank you for the KIP. I agree that managing complex configurations is not easy with the current tool. Having the possibility to use properties file sounds quite handy to me. It makes it easier to edit and to reuse base configurations. I wonder if we should also add a `--delete-config-f

Re: [DISCUSS] KIP-574: CLI Dynamic Configuration with file input

2020-02-27 Thread Aneel Nazareth
I've created a PR for a potential implementation of this: https://github.com/apache/kafka/pull/8184 if we decide to go ahead with this KIP. On Wed, Feb 26, 2020 at 12:36 PM Aneel Nazareth wrote: > Hi, > > I'd like to discuss adding a new argument to kafka-configs.sh > (ConfigCommand.scala). > >

[DISCUSS] KIP-574: CLI Dynamic Configuration with file input

2020-02-26 Thread Aneel Nazareth
Hi, I'd like to discuss adding a new argument to kafka-configs.sh (ConfigCommand.scala). Recently I've been working on some things that require complex configurations. I've chosen to represent them as JSON strings in my server.properties. This works well, and I'm able to update the configurations