unsubscribe On Mon, Aug 29, 2022 at 8:56 AM Derek Mok <derek.mok9...@gmail.com> wrote:
> Hi, I'd like some help with understanding how exactly max.task.idle.ms > works. I have a topology that consumes from an input topic A, and a join > operator that enriches the topic A messages with a KTable from a compacted > topic B. The enriched messages are output to topic C. > > If I set max.task.idle.ms to 5000, what should be the expected behaviour > in > the following scenario (assuming all messages have the same key and topics > have same partition count): > > 1. Message published to A with timestamp t_0 > 2. Three seconds of wall clock time elapses > 3. Message published to B with timestamp t_0 > > My understanding is that a message should be output to topic C containing > the enriched result after max.task.idle.ms elapses since it should account > for the late producer of topic B. The join operator should only be invoked > after max.task.idle.ms has elapsed. However, what actually happens is that > nothing is published in topic C. I suspect my understanding of > max.task.idle.ms is not entirely correct so would appreciate any insight > here! > > Here is a small project demonstrating the above > https://github.com/ThousandEyes-Derek/max-task-idle-ms > > MaxTaskIdleStreamsApp contains the streams application that describes the > topology. > MessageProducer produces messages into the input and compacted topics based > on the scenario I described above. > > Thanks! > Derek >
