Re: [VOTE] KIP-226 - Dynamic Broker Configuration

2018-01-10 Thread Rajini Sivaram
The vote has passed with 4 binding votes (Gwen, Jun, Jason, me) and 2 non-binding votes (Ted You, Tom Bentley). Many thanks for the reviews and votes. I will update the KIP page. Regards, Rajini On Tue, Jan 9, 2018 at 11:00 AM, Rajini Sivaram wrote: > > Thank you, Jun! I have updated the KIP.

Re: [VOTE] KIP-226 - Dynamic Broker Configuration

2018-01-09 Thread Rajini Sivaram
Thank you, Jun! I have updated the KIP. If there are no other comments or concerns, I will close the vote later today. Thanks, Rajini On Mon, Jan 8, 2018 at 10:57 PM, Jun Rao wrote: > Hi, Rajini, > > Thanks for the explanation. Then your suggestion sounds good to me. > > Jun > > On Mon, Jan 8

Re: [VOTE] KIP-226 - Dynamic Broker Configuration

2018-01-08 Thread Jun Rao
Hi, Rajini, Thanks for the explanation. Then your suggestion sounds good to me. Jun On Mon, Jan 8, 2018 at 1:32 PM, Rajini Sivaram wrote: > Hi Jun, > > No, password.encoder.secret cannot be updated dynamically at the moment. > Dynamic configs are stored in ZooKeeper and since ZK is not secure

Re: [VOTE] KIP-226 - Dynamic Broker Configuration

2018-01-08 Thread Rajini Sivaram
Hi Jun, No, password.encoder.secret cannot be updated dynamically at the moment. Dynamic configs are stored in ZooKeeper and since ZK is not secure, all password configs in ZK are encrypted using password.encoder.secret. We cannot make password.encoder.secret dynamic since it would need another s

Re: [VOTE] KIP-226 - Dynamic Broker Configuration

2018-01-08 Thread Jun Rao
Hi, Rajini, Could password.encoder.secret be updated dynamically? If so, each broker will still have access to the old secret when password.encoder.secret is updated. Perhaps that's a simpler way to handle changing secret than introducing an extra config. Thanks, Jun On Fri, Jan 5, 2018 at 3:09

Re: [VOTE] KIP-226 - Dynamic Broker Configuration

2018-01-05 Thread Rajini Sivaram
Hi Jun, We are using 2-way encryption. The password configs encoded are keystore/truststore passwords and JAAS configuration. We need to be able to extract the actual values for these, so we cannot use 1-way hash. So if we have the old secret, we can decrypt and get the original values. Thank you

Re: [VOTE] KIP-226 - Dynamic Broker Configuration

2018-01-04 Thread Jun Rao
Hi, Rajin, Does providing the old-secret help? My understanding is that the encoded passwd is the result of a 1-way hash with the secret. So, one can't decode the passwd with old-secret. If that's the case, one still needs to provide the unencrypted paaswd to re-encode with the new secret? Thanks

Re: [VOTE] KIP-226 - Dynamic Broker Configuration

2018-01-04 Thread Rajini Sivaram
Hi Jun/Jason, I was wondering whether it is worth adding a new property (static config in server.properties) to pass in the previous encoder password as well when changing encoder password. So you would set: - password.encoder.secret=new-password - password.encoder.old.secret=old-password

Re: [VOTE] KIP-226 - Dynamic Broker Configuration

2018-01-03 Thread Rajini Sivaram
Hi Jason, Thank you for reviewing and voting. Thanks, I had missed the rename. Have updated the KIP. The configs can be defined in the static server.properties or in ZooKeeper. If a ZK config cannot be decoded (or is not valid), we log an error and revert to the static config or default. When up

Re: [VOTE] KIP-226 - Dynamic Broker Configuration

2018-01-03 Thread Jason Gustafson
+1 Thanks for the KIP. One minor nit: I think we changed ConfigSource.TOPIC_CONFIG to ConfigSource.DYNAMIC_TOPIC_CONFIG in the PR. As far as updating secrets, I wasn't sure I understand how that will work. Do the password configs accept multiple values? On Wed, Jan 3, 2018 at 2:58 AM, Rajini Siva

Re: [VOTE] KIP-226 - Dynamic Broker Configuration

2018-01-03 Thread Rajini Sivaram
Hi Jun, Thank you for reviewing and voting. 50. I have updated the KIP to describe how the secret may be changed. All dynamically configurable passwords and per-broker configs. So the secret can be different across brokers and updated using rolling restart. In order to update the secret, each bro

Re: [VOTE] KIP-226 - Dynamic Broker Configuration

2018-01-02 Thread Jun Rao
Hi, Rajini, Thank for the KIP. +1. Just a couple of minor comments below. 50. config.secret.*: Could you document how the encryption/decryption of passwd work? In particular, how do we support changing config.secret? 51. At the topic level, we also have leader.replication.throttled.replicas and

Re: [VOTE] KIP-226 - Dynamic Broker Configuration

2017-12-12 Thread Gwen Shapira
+1 (binding). Thank you for leading this, Rajini. On Tue, Dec 12, 2017 at 8:35 AM Tom Bentley wrote: > +1 (nonbinding) > > On 12 December 2017 at 15:34, Ted Yu wrote: > > > +1 > > > > On Tue, Dec 12, 2017 at 5:44 AM, Rajini Sivaram > > > wrote: > > > > > Since there are no more outstanding com

Re: [VOTE] KIP-226 - Dynamic Broker Configuration

2017-12-12 Thread Tom Bentley
+1 (nonbinding) On 12 December 2017 at 15:34, Ted Yu wrote: > +1 > > On Tue, Dec 12, 2017 at 5:44 AM, Rajini Sivaram > wrote: > > > Since there are no more outstanding comments, I would like to start vote > > for KIP-226: > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > > 226+-+

Re: [VOTE] KIP-226 - Dynamic Broker Configuration

2017-12-12 Thread Ted Yu
+1 On Tue, Dec 12, 2017 at 5:44 AM, Rajini Sivaram wrote: > Since there are no more outstanding comments, I would like to start vote > for KIP-226: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 226+-+Dynamic+Broker+Configuration > > > The KIP enables dynamic update of commonly upd