unsubscribe

On Mon, Aug 29, 2022 at 8:56 AM Derek Mok <derek.mok9...@gmail.com> wrote:

> Hi, I'd like some help with understanding how exactly max.task.idle.ms
> works. I have a topology that consumes from an input topic A, and a join
> operator that enriches the topic A messages with a KTable from a compacted
> topic B. The enriched messages are output to topic C.
>
> If I set max.task.idle.ms to 5000, what should be the expected behaviour
> in
> the following scenario (assuming all messages have the same key and topics
> have same partition count):
>
>    1. Message published to A with timestamp t_0
>    2. Three seconds of wall clock time elapses
>    3. Message published to B with timestamp t_0
>
> My understanding is that a message should be output to topic C containing
> the enriched result after max.task.idle.ms elapses since it should account
> for the late producer of topic B. The join operator should only be invoked
> after max.task.idle.ms has elapsed. However, what actually happens is that
> nothing is published in topic C. I suspect my understanding of
> max.task.idle.ms is not entirely correct so would appreciate any insight
> here!
>
> Here is a small project demonstrating the above
> https://github.com/ThousandEyes-Derek/max-task-idle-ms
>
> MaxTaskIdleStreamsApp contains the streams application that describes the
> topology.
> MessageProducer produces messages into the input and compacted topics based
> on the scenario I described above.
>
> Thanks!
> Derek
>

Reply via email to