Hello, Hope you are doing good.
There are some issues we are experiencing while using Kafka in our environments. I will explain the scenario of the issues below for better understanding. We are setting up Kafka using a Docker compose file (running this docker compose on a server with RHEL operating system), along with that Kafka we are setting up Zookeeper and Kafdrop as well. So, once we do the setup and check the End-to-End connectivity, it works fine. By fine, I mean that the Kafka Consumer, Monitor and Producer container logs have live data, when we do the testing, which I suppose indicates that the Kafka broker node and all other components are working fine. This current setup works fine for some random time and suddenly, the containers get deleted. By deleted, I mean that the containers are built with a restart-always condition, so they are not even in exited state. They just vanish out and we had to do the setup all again. The reason we are raising this issue is that we are not able to pinpoint what exactly is causing the Kafka containers to get deleted when the Kafdrop and Zookeeper are still there and just the Kafka broker container gets deleted. For solutions, We tried reinstalling the Operating System. We tried using Kafka images from Bitnami and Wurstmeister. We tried changing the Internal and External port configuration in Docker compose file used to stand-up the containers. Docker compose used for Kafka setup attached below as well. [test-kafka: image: wurstmeister/kafka ports: - "9094:9094" expose: - "9095" environment: KAFKA_ADVERTISED_LISTENERS: INSIDE://test-kafka:9095,OUTSIDE://192.168.X.X:9094 KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INSIDE:PLAINTEXT,OUTSIDE:PLAINTEXT KAFKA_LISTENERS: INSIDE://0.0.0.0:9095,OUTSIDE://0.0.0.0:9094 KAFKA_INTER_BROKER_LISTENER_NAME: INSIDE KAFKA_ZOOKEEPER_CONNECT: test-zookeeper:2181 restart: unless-stopped] Please feel free to ping if any questions. Thanks, Smit Shanischara
