Not sure if this is entirely relevant, but figure folks might be interested: it's early days for us, but we're currently running Kafka+ZK in AWS on Kubernetes StatefulSets with the magic annotation to publish "not ready" addresses (can't recall what it is off the top of my head). It seems stable so far and we can deploy & redeploy without issue. The jury is still out on whether running Kafka+ZK on Kubernetes is *advisable*, but I can talk about what we had to do to get to where we are.
The "worst" thing we had to do was patch the ZK client libs on the Kafka side to replace the default HostProvider implementation with a special "kubernetes-friendly" HostProvider to force DNS lookups: default behavior of StaticHostProvider is to resolve once & cache IPs at the time the client is created. We also had to configure our JVM DNS TTLs to essentially nothing. We were bitten by KAFKA-2729 in one of our testing environments for reasons I still don't fully understand, so we're chomping at the bit for 1.1.0. :) Otherwise ... so far so good -- but again, very early days. Also maybe interesting is that the ZK 3.5 client libs allow pluggable HostProviders, which will give you folks (Kafka devs) more control over the ZK DNS lookups without needing to do crazy patching stuff like we did. All the necessary mechanics to do this are in ZK 3.4.x, but not exposed at the API level. I've contemplated sending a patch upstream to see if they'd consider allowing a system property to control the HostProvider implementation in 3.4 as sort of a poor-man's backport -- then we wouldn't need to patch the client libs to make things play nice with Kubernetes. No idea what the temperature would be on that, though. Anyway: don't consider this an endorsement of the idea and I'm not familiar enough with etcd to comment on whether ZK is remarkably more difficult to manage as the KIP seems to imply, just wanted to comment on our experience so far. Cheers, Tom On Wed, Mar 28, 2018 at 2:00 PM, Ismael Juma <ism...@juma.me.uk> wrote: > Here's a link to the previous discussion: > > https://lists.apache.org/thread.html/2bc187040051008452b40b313db06b > 476c248ef7a5ed7529afe7b118@1448997154@%3Cdev.kafka.apache.org%3E > > Ismael > > On Wed, Mar 28, 2018 at 10:40 AM, Ismael Juma <ism...@juma.me.uk> wrote: > > > Hi Gwen, > > > > I don't think the reasons why a pluggable metastore is not desirable have > > changed. My suggestion is that the KIP should try to address the concerns > > raised previously as part of the proposal. > > > > Ismael > > > > On Wed, Mar 28, 2018 at 10:24 AM, Gwen Shapira <g...@confluent.io> > wrote: > > > >> While I'm not in favor of the proposal, I want to point out that the > >> ecosystem changed quite a bit since KIP-30 was first proposed. > Kubernetes > >> deployments are far more common now and are growing in popularity, and > the > >> problem in deployment, discovery and management that ZK poses is > therefore > >> more relevant now than it was at the time. There are reasons for the > >> community to change its collective mind even if the objections are still > >> valid. > >> > >> Since the KIP doesn't include the etcd implementation, the proposal > looks > >> like very simple refactoring. Of course, the big change is a new public > >> API. But it's difficult to judge from the KIP if the API is a good one > >> because it is built to 100% match the one implementation we have. I'm > >> curious if the plan includes contributing the Etcd module to Apache > Kafka? > >> > >> > >> On Wed, Mar 28, 2018 at 9:54 AM, Ismael Juma <ism...@juma.me.uk> wrote: > >> > >> > Thanks for the KIP. This was proposed previously via "KIP-30 Allow for > >> > brokers to have plug-able consensus and meta data storage sub systems" > >> and > >> > the community was not in favour. Have you considered the points > >> discussed > >> > then? > >> > > >> > Ismael > >> > > >> > On Wed, Mar 28, 2018 at 9:18 AM, Molnár Bálint < > molnarcsi...@gmail.com> > >> > wrote: > >> > > >> > > Hi all, > >> > > > >> > > I have created KIP-273: Kafka to support using ETCD beside Zookeeper > >> > > > >> > > Here is the link to the KIP: > >> > > > >> > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > >> > > 273+-+Kafka+to+support+using+ETCD+beside+Zookeeper > >> > > > >> > > Looking forward to the discussion. > >> > > > >> > > Thanks, > >> > > Balint > >> > > > >> > > >> > >> > >> > >> -- > >> *Gwen Shapira* > >> Product Manager | Confluent > >> 650.450.2760 | @gwenshap > >> Follow us: Twitter <https://twitter.com/ConfluentInc> | blog > >> <http://www.confluent.io/blog> > >> > > > > > -- *Tom Lee */ http://tomlee.co / @tglee <http://twitter.com/tglee>
