Hello, The question might trigger people to reply with "Confluent" - but it's not related to confluent as the kubernetes offering is not for publi/community edition. So, discussing Helm charts and intro to Confluent isn't our objective here.
What I am trying to understand is how does the log files (kafka message logs, consumer offsets) are managed in Kubernetes (e.g. persistent volume, statefulsets, etc). I have a 3 node cluster running over 3 physical Linux VMs, and would like to move this setup to Kubernetes. The only part where we are strugging is with the following: 1) how does logs.dir configuration work per Pod? 2) Assuming I have 3 PODs (3 brokers), and one of the Pod Goes down - how do I manage the message log and offset files ? If a POD goes does, that means it will delete everything in the logs.dir location, won't it? 3) I am assuming broker.id will be supplied using some form of configMap, but if there is anything worth knowing here, please do share. We have a reliable service on bare metal cloud, so we don't want to disrupt it unless we are sure about the changes. Does anyone have any experience with this? If so, it would be great if you can share any gists or configs. Much appreciated. Regards,
