Multi Cluster Failover Kafka Client

2022-03-14 Thread Chirag Dewan
Hi, I am designing a Geo Red Kafka cluster with MirrorMaker. My use case is an active/passive cluster where the follower is kept up-to-date by MirrorMaker. Now, I want that my consumers and producers should switch to the follower cluster when none of the brokers in the primary cluster are reachab

Command to Dynamically Remove Listeners

2023-05-01 Thread Chirag Dewan
Hi, We are running a Kafka cluster in Kubernetes. We have a use case where we are trying to dynamically remove advertised listeners from a side car container if our Kafka cluster loses heartbeat with our other application service. And ofcourse add if the heartbeat is ever a success. I tried usin

Broker Failure Recovery - Disk or Replication?

2023-06-06 Thread Chirag Dewan
Hi, I wanted to understand more on how a broker recovers from a failure. As far as I understand, when a broker recovers from a failure, it 1) loads data of all partitions it hosted from the disk (the metadata about the partitions would exist in zookeeper, right?)2) then syncs with the leader of