Hi, I'd like some help with understanding how exactly max.task.idle.ms works. I have a topology that consumes from an input topic A, and a join operator that enriches the topic A messages with a KTable from a compacted topic B. The enriched messages are output to topic C.
If I set max.task.idle.ms to 5000, what should be the expected behaviour in the following scenario (assuming all messages have the same key and topics have same partition count): 1. Message published to A with timestamp t_0 2. Three seconds of wall clock time elapses 3. Message published to B with timestamp t_0 My understanding is that a message should be output to topic C containing the enriched result after max.task.idle.ms elapses since it should account for the late producer of topic B. The join operator should only be invoked after max.task.idle.ms has elapsed. However, what actually happens is that nothing is published in topic C. I suspect my understanding of max.task.idle.ms is not entirely correct so would appreciate any insight here! Here is a small project demonstrating the above https://github.com/ThousandEyes-Derek/max-task-idle-ms MaxTaskIdleStreamsApp contains the streams application that describes the topology. MessageProducer produces messages into the input and compacted topics based on the scenario I described above. Thanks! Derek
