Hi,

Today, the configuration of rack awareness for brokers (broker.rack=my-rack-id) 
is either manual or via a script during deployment of a broker.

Other systems, like Hadoop or Kubernetes, have support for auto-detection of 
topologies, often called well-known labels like racks, zones or regions.

Many Kafka deployments I support on the VMware vSphere hypervisor have rack 
awareness as a requirement to increase availability for the brokers in case of 
host (hypervisor) failure. For this to work, several actions (and 
communication) across different teams has to happen, which is error-prone.

Would there be interest in having a functionality to auto-detect the underlying 
topology, e.g. via a pluggable mechanism (interface) or a dedicated 
implementation for vSphere? A Kafka broker could make a call to "GetZones()" 
(or "GetRacks") and the specific implementation of the cloud provider would 
respond with the current rack/zone where this broker is running.

Thx for any feedback,
Michael

Reply via email to