Hi Levani, Thanks for this KIP! I think this is really high value; it was something I was disappointed I didn’t get to do as part of KIP-441.
Rack awareness is a feature provided by other distributed systems as well. I wonder if your KIP could devote a section to summarizing what rack awareness looks like in other distributed systems, to help us put this design in context. Thanks! John On Tue, Jan 26, 2021, at 16:46, Levani Kokhreidze wrote: > Hello all, > > I’d like to start discussion on KIP-708 [1] that aims to introduce rack > aware standby task distribution in Kafka Streams. > In addition to changes mentioned in the KIP, I’d like to get some ideas > on additional change I have in mind. > Assuming KIP moves forward, I was wondering if it makes sense to > configure Kafka Streams consumer instances with the rack ID passed with > the new StreamsConfig#RACK_ID_CONFIG property. > In practice, that would mean that when “rack.id <http://rack.id/>” is > configured in Kafka Streams, it will automatically translate into > ConsumerConfig#CLIENT_RACK_ID config for all the KafkaConsumer clients > that is used by Kafka Streams internally. > > [1] > https://cwiki.apache.org/confluence/display/KAFKA/KIP-708%3A+Rack+aware+Kafka+Streams+with+pluggable+StandbyTask+assignor > > <https://cwiki.apache.org/confluence/display/KAFKA/KIP-708:+Rack+aware+Kafka+Streams+with+pluggable+StandbyTask+assignor> > > P.S > I have draft PR ready, if it helps the discussion moving forward, I can > provide the draft PR link in this thread. > > Regards, > Levani
