Re: Kafka Connect on Kubernetes: Statefulset vs Deployment

2025-06-14 Thread Federico Valeri
Hi Prateek. In a Kafka Connect cluster, the advertised address represents the identity of the worker node. Connectors and tasks are scheduled to the individual worker nodes based on their identity. If you use a Kubernetes Deployment, when you roll the cluster, new Pods with new IPs will be be cre

Re: Kafka Connect on Kubernetes: Statefulset vs Deployment

2025-06-14 Thread Prateek Kohli
Thanks a lot for the detailed reply. But when I tried restarting one of the pods in my connect cluster(which was a statefulset set with worker_id set as ...svc.cluster.local), it still took 5 minutes for the tasks to get reassigned. The behaviour was same with Deployment as well. Did I get som

Kafka Connect on Kubernetes: Statefulset vs Deployment

2025-06-14 Thread Prateek Kohli
Hi All, I'm building a custom Docker image for kafka Connect and planning to run it on Kubernetes. I'm a bit stuck on whether I should use a Deployment or a StatefulSet. >From what I understand, the main difference that could affect Kafka Connect is the hostname/IP behaviour. With a Deployment, p

Kafka Connect on Kubernetes: Statefulset vs Deployment

2025-06-14 Thread Prateek Kohli
Hi All, I'm building a custom Docker image for kafka Connect and planning to run it on Kubernetes. I'm a bit stuck on whether I should use a Deployment or a StatefulSet. >From what I understand, the main difference that could affect Kafka Connect is >the hostname/IP behaviour. With a Deploymen