On Fri, Sep 14, 2018, at 00:46, lbari...@gmail.com wrote: > Hello, Colin! > > I have read the discussion on KIP-273 and want to raise this question. > Deploying Kafka to Kubernetes is a crucial problem in my current work. > And usage of ZooKeeper makes troubles, because it works unstable in > container.
Hi Ibarikyn, I haven't heard any reports of ZooKeeper being unstable in containers. There are a lot of companies already deploying Kafka and ZooKeeper this way, so I am surprised to hear that you had trouble. Can you be a little more clear about what issues you are seeing with ZooKeeper in containers? > In this discussion you have mentioned that the goal is to get rid of it > in future releases. But since then there were no posts about it. Also, i > can not see such plan in a roadmap. Cuold you please tell us more about > work made in this stream. ZooKeeper (and etcd, and consul, and the others) were originally designed as configuration management systems. They're really not set up to do what Kafka does with them, which is use them as one half of a data management system. We need things like uniform access control, the ability to cache metadata locally, the ability to propagate only what has changed to brokers, etc. That requires us to manage our own metadata log. Kafka is a system for managing logs, so this is a natural design. There have been a lot of discussions about pluggable consensus before on the mailing list. Even if we wanted to keep using a configuration management system for metadata, pluggable consensus is a really bad idea. It means we would have to support multiple systems, getting correspondingly less testing in each. We also would have to use least common denominator APIs, since each system is somewhat different. And configuration would get harder for new users. best, Colin
