On Tue, Jan 14, 2020, at 11:49, Rajini Sivaram wrote:
> Hi Ron,
>
> Thanks for the detailed explanation, sounds good to me.
>
> A few more questions:
>
> 1) At the moment, all sensitive broker configs including
> keystore/truststore passwords can be stored encrypted in ZooKeeper prior to
> starting up brokers. We are now adding SSL keystore/truststore passwords
> for ZooKeeper client that cannot be stored in ZooKeeper since you need
> these to connect to ZK. We should perhaps document that these configs can
> be encrypted using KIP-421.
That's a good point. Previously, we didn't have ZK on-the-wire security
support. However, now that we do, sending sensitive keystore and truststore
passwords to ZK in cleartext should be deprecated in favor of using KIP-421.
>
> 2) We can dynamically update keystores and trust stores used by brokers
> without restarting the broker. Can we support this easily for ZK clients
> created by the broker, for example by adding our own
> `zookeeper.ssl.context.supplier.class`?
>
Hmm. That might be better to consider in a follow-up, since the deadline for
2.5 KIPs is coming up?
best,
Colin
> 3) Looks like we are using config names that directly map to ZK configs.
> Have we considered using equivalent Kafka config names with prefixes,
> perhaps with inheritance from the non-prefixed names? Not sure if this is a
> good idea, but perhaps worth documenting in Rejected Alternatives at least.
>
>
> On Tue, Jan 14, 2020 at 5:14 PM Colin McCabe <cmcc...@apache.org> wrote:
>
> > Hi Ron,
> >
> > Thanks for the explanation. I guess thinking about it a little bit more,
> > we should just add --zk-tls-config-file to all of these commands.
> >
> > We will be removing this option (plus ZK in general) from these commands
> > in the next major release, but ZK is still supported in 2.5, so we should
> > just do the logical thing. (The exception is ZkSecurityMigrator, which
> > will stay).
> >
> > best,
> > Colin
> >
> >
> > On Tue, Jan 14, 2020, at 07:38, Ron Dagostino wrote:
> > > Hi Colin.
> > >
> > > <<< It seems like this [--zk-tls-config-file information] could just
> > appear
> > > in a configuration file, which all of these tools already accept (I
> > think)
> > >
> > > ZkSecurityMigrator has no such property file facility; adding a
> > > "--zk-tls-config-file" parameter is exactly for this purpose. If we add
> > > that to ZkSecurityMigrator then it is trivial to add it to other commands
> > > (the same code is simply reused; it ends up being just a few extra
> > lines).
> > > I do not see any parameter in the other two commands to adjust the ZK
> > > connection; ConfigCommand accepts a "--command-config" flag, but
> > according
> > > to the code "This is used only with --bootstrap-server option for
> > > describing and altering broker configs."
> > >
> > > I do agree there would be no need to add "--zk-tls-config-file" to
> > > ReassignPartitionsCommand if its direct ZK connectivity is replaced in
> > time
> > > for the next release.
> > >
> > > ConfigCommand supports the "--bootstrap-server" option and will have its
> > > direct ZooKeeper access formally deprecated as per KIP-555, but the
> > special
> > > use case of bootstrapping a ZooKeeper ensemble with encrypted credentials
> > > prior to starting Kafka will still exist, so it feels like while
> > > "--zk-tls-config-file" would never be used except for this use case it
> > > could probably still be added for this particular situation.
> > >
> > > Ron
> > >
> > > P.S. I responded on 1/6 but I just discovered that e, ail (and 3 more I
> > > sent) did not go through. I am trying to get emails through now to move
> > > this discussion forward.
> > >
> > > On Mon, Jan 6, 2020 at 5:07 PM Colin McCabe <cmcc...@apache.org> wrote:
> > >
> > > > On Fri, Dec 27, 2019, at 10:48, Ron Dagostino wrote:
> > > > > Hi everyone. I would like to make the following changes to the KIP.
> > > > >
> > > > > MOTIVATION:
> > > > > Include a statement that it will be difficult in the short term to
> > > > > deprecate direct Zookeeper communication in kafka-configs.{sh, bat}
> > > > (which
> > > > > invoke kafka.admin.ConfigCommand) because bootstrapping a Kafka
> > cluster
> > > > > with encrypted passwords in Zookeeper is an explicitly-supported use
> > > > case;
> > > > > therefore it is in scope to be able to securely configure the CLI
> > tools
> > > > > that still leverage non-deprecated direct Zookeeper communication
> > for TLS
> > > > > (the other 2 tools are kafka-reassign-partitions.{sh, bat} and
> > > > > zookeeper-security-migration.sh).
> > > >
> > > > Hi Ron,
> > > >
> > > > Thanks for the KIP.
> > > >
> > > > About deprecations:
> > > >
> > > > * zookeeper-security-migration: clearly, deprecating ZK access in this
> > one
> > > > would not make sense, since it would defeat the whole point of the
> > tool :)
> > > >
> > > > * kafka-reassign-partitions: ZK access should be deprecated here.
> > KIP-455
> > > > implementation has dragged a bit, but this should get done soon.
> > Certainly
> > > > before the next release.
> > > >
> > > > * kafka-configs: I think ZK access should be deprecated here as well.
> > I
> > > > agree there is a super-special bootstrapping case here, but that should
> > > > have its own tool, not use kafka-configs.
> > > >
> > > > I will post a separate KIP for this, though.
> > > >
> > > > >
> > > > > GOALS:
> > > > > Support the secure configuration of TLS-encrypted communication
> > between
> > > > > Zookeeper and:
> > > > > a) Kafka brokers
> > > > > b) The three CLI tools mentioned above that still support direct,
> > > > > non-deprecated communication to Zookeeper
> > > > > It is explicitly out-of-scope to deprecate any direct Zookeeper
> > > > > communication in CLI tools as part of this KIP; such work will occur
> > in
> > > > > future KIPs instead.
> > > > >
> > > > > PUBLIC INTERFACES:
> > > > > 1) The following new broker configurations will be recognized.
> > > > > zookeeper.client.secure (default value = false, for backwards
> > > > > compatibility)
> > > > > zookeeper.clientCnxnSocket
> > > > > zookeeper.ssl.keyStore.location
> > > > > zookeeper.ssl.keyStore.password
> > > > > zookeeper.ssl.trustStore.location
> > > > > zookeeper.ssl.trustStore.password
> > > > > It will be an error for any of the last 5 values to be left
> > unspecified
> > > > if
> > > > > zookeeper.client.secure is explicitly set to true.
> > > > >
> > > > > 2) In addition, the kafka.security.authorizer.AclAuthorizer class
> > > > supports
> > > > > the ability to connect to a different Zookeeper instance than the
> > one the
> > > > > brokers use. We therefore also add the following optional configs,
> > which
> > > > > override the corresponding ones from above when present:
> > > > > authorizer.zookeeper.client.secure
> > > > > authorizer.zookeeper.clientCnxnSocket
> > > > > authorizer.zookeeper.ssl.keyStore.location
> > > > > authorizer.zookeeper.ssl.keyStore.password
> > > > > authorizer.zookeeper.ssl.trustStore.location
> > > > > authorizer.zookeeper.ssl.trustStore.password
> > > > >
> > > > > 3) The three CLI tools mentioned above will support a new
> > > > --zk-tls-config-file
> > > > > <String: Zookeeper TLS configuration file path>" option. The
> > following
> > > > > properties will be recognized in that file, and unrecognized
> > properties
> > > > > will be ignored to allow the possibility of pointing
> > zk-tls-config-file
> > > > at
> > > > > the broker's config file.
> > > > > zookeeper.client.secure (default value = false)
> > > > > zookeeper.clientCnxnSocket
> > > > > zookeeper.ssl.keyStore.location
> > > > > zookeeper.ssl.keyStore.password
> > > > > zookeeper.ssl.trustStore.location
> > > > > zookeeper.ssl.trustStore.password
> > > > > It will be an error for any of the last 5 values to be left
> > unspecified
> > > > if
> > > > > zookeeper.client.secure is explicitly set to true.
> > > >
> > > > Do we really need a --zk-tls-config-file flag? It seems like this
> > could
> > > > just appear in a configuration file, which all of these tools already
> > > > accept (I think).
> > > >
> > > > best,
> > > > Colin
> > > >
> > > >
> > > > >
> > > > > Ron
> > > > >
> > > > > On Mon, Dec 23, 2019 at 3:03 PM Ron Dagostino <rndg...@gmail.com>
> > wrote:
> > > > >
> > > > > > Hi everyone. Let's get this discussion going again now that Kafka
> > 2.4
> > > > > > with Zookeeper 3.5.6 is out.
> > > > > >
> > > > > > First, regarding the KIP number, the other KIP that was using this
> > > > number
> > > > > > moved to KIP 534, so KIP 515 remains the correct number for this
> > > > > > discussion. I've updated the Kafka Improvement Proposal page to
> > list
> > > > this
> > > > > > KIP in the 515 slot, so we're all set there.
> > > > > >
> > > > > > Regarding the actual issues under discussion, I think there are
> > some
> > > > > > things we should clarify.
> > > > > >
> > > > > > 1) It is possible to use TLS connectivity to Zookeeper from Apache
> > > > Kafka
> > > > > > 2.4 -- the problem is that configuration information has to be
> > passed
> > > > via
> > > > > > system properties as "-D" command line options on the java
> > invocation
> > > > of
> > > > > > the broker, and those are not secure (anyone with access to the box
> > > > can see
> > > > > > the command line used to invoke the broker); the configuration
> > includes
> > > > > > sensitive information (e.g. a keystore password), so we need a
> > secure
> > > > > > mechanism for passing the configuration values. I believe the real
> > > > > > motivation for this KIP is to harden the configuration mechanism
> > for
> > > > > > Zookeeper TLS connectivity.
> > > > > >
> > > > > > 2) I believe the list of CLI tools that continue to use direct
> > > > Zookeeper
> > > > > > connectivity in a non-deprecated fashion is:
> > > > > > a)
> > zookeeper-security-migration.sh/kafka.admin.ZkSecurityMigrator
> > > > > > b)
> > > > > >
> > > >
> > kafka-reassign-partitions.{sh,bat}/kafka.admin.ReassignPartitionsCommand
> > > > > > c) kafka-configs.{sh, bat}/kafka.admin.ConfigCommand
> > > > > >
> > > > > > 3) I believe Kafka.admin.ConfigCommand presents a conundrum
> > because it
> > > > > > explicitly states in a comment that a supported use case is
> > > > bootstrapping a
> > > > > > Kafka cluster with encrypted passwords in Zookeeper (see
> > > > > >
> > > >
> > https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/admin/ConfigCommand.scala#L65
> > > > ).
> > > > > > This means it will be especially difficult to deprecate this
> > particular
> > > > > > direct Zookeeper connectivity without a different storage
> > mechanism for
> > > > > > dynamic configuration values being available (i.e. the self-managed
> > > > quorum
> > > > > > referred to in KIP-500).
> > > > > >
> > > > > > I think it would be easier and simpler to harden the Zookeeper TLS
> > > > > > configuration in both Kafka and in CLI tools compared to trying to
> > > > > > deprecate the direct Zookeeper connectivity in the above 3 CLI
> > tools --
> > > > > > especially when ConfigCommand has no obvious short-term path to
> > > > deprecation.
> > > > > >
> > > > > > Regarding how to harden the TLS configuration, adding the
> > appropriate
> > > > > > configs to Kafka is an obvious choice for the brokers. Not that
> > these
> > > > > > values cannot be dynamically reconfigurable because the values
> > would be
> > > > > > required to connect to Zookeeper via TLS in the first place.
> > > > > >
> > > > > > Regarding how to harden the TLS configuration for the 3 remaining
> > CLI
> > > > > > tools, it would be relatively straightforward to add support for a
> > > > > > "--zk-tls-config-file <String: Zookeeper TLS configuration system
> > > > > > properties file path>" option to each one.
> > > > > >
> > > > > > Thoughts?
> > > > > >
> > > > > > Ron
> > > > > >
> > > > > > On Thu, Sep 12, 2019 at 8:13 AM Pere Urbón Bayes <
> > pere.ur...@gmail.com
> > > > >
> > > > > > wrote:
> > > > > >
> > > > > >> True, the number is duplicated.
> > > > > >>
> > > > > >> do you know how can we get the problem fix? I was not aware, I
> > > > missed,
> > > > > >> sorry the need to add the KIP to the table.
> > > > > >>
> > > > > >> -- Pere
> > > > > >>
> > > > > >> Missatge de Jorge Esteban Quilcate Otoya <
> > quilcate.jo...@gmail.com>
> > > > del
> > > > > >> dia
> > > > > >> dt., 3 de set. 2019 a les 13:43:
> > > > > >>
> > > > > >> > Hi Pere,
> > > > > >> >
> > > > > >> > Have you add your KIP to the list here
> > > > > >> >
> > > > > >>
> > > >
> > https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
> > > > > >> ?
> > > > > >> >
> > > > > >> > I found the KIP number assigned to another.
> > > > > >> >
> > > > > >> >
> > > > > >> >
> > > > > >> > On Mon, Sep 2, 2019 at 2:23 PM Pere Urbón Bayes <
> > > > pere.ur...@gmail.com>
> > > > > >> > wrote:
> > > > > >> >
> > > > > >> >> Thanks for your time Harsha,
> > > > > >> >> anyone else with comments? looking forward to hearing from
> > you.
> > > > > >> >>
> > > > > >> >> Stupid question: when do you move from discussion to vote?
> > > > > >> >>
> > > > > >> >> Missatge de Harsha Chintalapani <ka...@harsha.io> del dia
> > dv., 30
> > > > > >> d’ag.
> > > > > >> >> 2019 a les 21:59:
> > > > > >> >>
> > > > > >> >> > Thanks Pere. KIP looks good to me.
> > > > > >> >> > -Harsha
> > > > > >> >> >
> > > > > >> >> >
> > > > > >> >> > On Fri, Aug 30, 2019 at 10:05 AM, Pere Urbón Bayes <
> > > > > >> >> pere.ur...@gmail.com>
> > > > > >> >> > wrote:
> > > > > >> >> >
> > > > > >> >> >> Not really,
> > > > > >> >> >> my idea is to keep the JAAS parameter, so people don't see
> > > > major
> > > > > >> >> >> changes. But if you pass a properties file, then this takes
> > > > > >> precedence
> > > > > >> >> over
> > > > > >> >> >> the other, with the idea that you can do sasl as well with
> > the
> > > > > >> >> properties
> > > > > >> >> >> files.
> > > > > >> >> >>
> > > > > >> >> >> Makes sense?
> > > > > >> >> >>
> > > > > >> >> >> -- Pere
> > > > > >> >> >>
> > > > > >> >> >> Missatge de Harsha Chintalapani <ka...@harsha.io> del dia
> > dv.,
> > > > 30
> > > > > >> >> d’ag.
> > > > > >> >> >> 2019 a les 19:00:
> > > > > >> >> >>
> > > > > >> >> >>> Hi Pere,
> > > > > >> >> >>> Thanks for the KIP. Enabling SSL for zookeeper
> > for
> > > > Kafka
> > > > > >> >> makes
> > > > > >> >> >>> sense.
> > > > > >> >> >>> "The changes are planned to be introduced in a compatible
> > way,
> > > > by
> > > > > >> >> >>> keeping the current JAAS variable precedence."
> > > > > >> >> >>> Can you elaborate a bit here. If the user configures a JAAS
> > > > file
> > > > > >> with
> > > > > >> >> >>> Client section it will take precedence over zookeeper SSL
> > > > configs?
> > > > > >> >> >>>
> > > > > >> >> >>> Thanks,
> > > > > >> >> >>> Harsha
> > > > > >> >> >>>
> > > > > >> >> >>>
> > > > > >> >> >>>
> > > > > >> >> >>> On Fri, Aug 30, 2019 at 7:50 AM, Pere Urbón Bayes <
> > > > > >> >> pere.ur...@gmail.com>
> > > > > >> >> >>> wrote:
> > > > > >> >> >>>
> > > > > >> >> >>>> Hi,
> > > > > >> >> >>>> quick question, I saw in another mail that 2.4 release is
> > > > planned
> > > > > >> for
> > > > > >> >> >>>> September. I think it would be really awesome to have
> > this for
> > > > > >> this
> > > > > >> >> >>>> release, do you think we can make it?
> > > > > >> >> >>>>
> > > > > >> >> >>>> -- Pere
> > > > > >> >> >>>>
> > > > > >> >> >>>> Missatge de Pere Urbón Bayes <pere.ur...@gmail.com> del
> > dia
> > > > dj.,
> > > > > >> 29
> > > > > >> >> >>>> d’ag. 2019 a les 20:10:
> > > > > >> >> >>>>
> > > > > >> >> >>>> Hi,
> > > > > >> >> >>>> this is my first KIP for a change in Apache Kafka, so I'm
> > > > really
> > > > > >> need
> > > > > >> >> >>>> to the process. Looking forward to hearing from you and
> > learn
> > > > the
> > > > > >> >> best
> > > > > >> >> >>>> ropes here.
> > > > > >> >> >>>>
> > > > > >> >> >>>> I would like to propose this KIP-515 to enable the
> > > > > >> ZookeeperClients
> > > > > >> >> to
> > > > > >> >> >>>> take full advantage of the TLS communication in the new
> > > > Zookeeper
> > > > > >> >> 3.5.5.
> > > > > >> >> >>>> Specially interesting it the Zookeeper Security Migration,
> > > > that
> > > > > >> >> without
> > > > > >> >> >>>> this change will not work with TLS, disabling users to use
> > > > ACLs
> > > > > >> when
> > > > > >> >> the
> > > > > >> >> >>>> Zookeeper cluster use TLS.
> > > > > >> >> >>>>
> > > > > >> >> >>>> link:
> > > > > >> >> >>>>
> > > > > >> >> >>>>
> > > > > >> >>
> > > > > >>
> > > >
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-515%3A+Enable+ZK+client+to+use+the+new+TLS+supported+authentication
> > > > > >> >> >>>>
> > > > > >> >> >>>> Looking forward to hearing from you on this,
> > > > > >> >> >>>>
> > > > > >> >> >>>> /cheers
> > > > > >> >> >>>>
> > > > > >> >> >>>> --
> > > > > >> >> >>>> Pere Urbon-Bayes
> > > > > >> >> >>>> Software Architect
> > > > > >> >> >>>> http://www.purbon.com
> > > > > >> >> >>>> https://twitter.com/purbon
> > > > > >> >> >>>> https://www.linkedin.com/in/purbon/
> > > > > >> >> >>>>
> > > > > >> >> >>>> --
> > > > > >> >> >>>> Pere Urbon-Bayes
> > > > > >> >> >>>> Software Architect
> > > > > >> >> >>>> http://www.purbon.com
> > > > > >> >> >>>> https://twitter.com/purbon
> > > > > >> >> >>>> https://www.linkedin.com/in/purbon/
> > > > > >> >> >>>>
> > > > > >> >> >>>
> > > > > >> >> >>>
> > > > > >> >> >>
> > > > > >> >> >> --
> > > > > >> >> >> Pere Urbon-Bayes
> > > > > >> >> >> Software Architect
> > > > > >> >> >> http://www.purbon.com
> > > > > >> >> >> https://twitter.com/purbon
> > > > > >> >> >> https://www.linkedin.com/in/purbon/
> > > > > >> >> >>
> > > > > >> >> >
> > > > > >> >> >
> > > > > >> >>
> > > > > >> >> --
> > > > > >> >> Pere Urbon-Bayes
> > > > > >> >> Software Architect
> > > > > >> >> http://www.purbon.com
> > > > > >> >> https://twitter.com/purbon
> > > > > >> >> https://www.linkedin.com/in/purbon/
> > > > > >> >>
> > > > > >> >
> > > > > >>
> > > > > >> --
> > > > > >> Pere Urbon-Bayes
> > > > > >> Software Architect
> > > > > >> http://www.purbon.com
> > > > > >> https://twitter.com/purbon
> > > > > >> https://www.linkedin.com/in/purbon/
> > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
>
