Ah and I forgot to mention, this is with Flink 1.18.1 Am Mo., 5. Feb. 2024 um 18:00 Uhr schrieb Alexis Sarda-Espinosa < sarda.espin...@gmail.com>:
> Hello, > > I have 2 Kafka sources that are configured with a watermark strategy > instantiated like this: > > WatermarkStrategy.<T>forBoundedOutOfOrderness(maxAllowedWatermarkDrift) > .withIdleness(idleTimeout) // 5 seconds currently > .withWatermarkAlignment(alignmentGroup, > maxAllowedWatermarkDrift, Duration.ofSeconds(1L)) > > The alignment group is the same for both, but each one consumes from a > different topic. During a test, I ensured that one of the topics didn't > receive any messages, but when I check the logs I see multiple entries like > this: > > Distributing maxAllowedWatermark=1707149933770 of group=dispatcher to > subTaskIds=[0] for source Source: GenericChangeMessageDeserializer. > > where maxAllowedWatermark grows all the time. > > Maybe my understanding is wrong, but I think this means the source is > never marked as idle even though it didn't receive any new messages in the > Kafka topic? > > Regards, > Alexis. > >
