I've recently started to enable rack awareness in my sources following this:
https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/datastream/kafka/#rackid E.g., ```java .setRackIdSupplier(() -> System.getenv("TM_NODE_AZ")) ``` This approach allows to decouple the AZ between jobmanager & taskamangers. There are indeed some Kafka-compatible solutions like WarpStream which support rack awareness for producers, too. E.g., https://docs.warpstream.com/warpstream/byoc/configure-kafka-client/configuring-kafka-client-id-features#warpstream_az The problem is that they pass the rack id using a normal producer property. Because the Kafka sink builder gets the properties and serializes them in the jobmanager, all the taskmanagers will use the same AZ as that of the jobmanager. Is there a way to easily pass "dynamic" properties so to say? Would it make sense to consider a supplier-based override for the property setters of the Kafka builders? Regards, Salva
