Hi All, Gentle reminder about bow query.
Thanks Banu On Tue, 9 Jul, 2024, 1:42 pm banu priya, <banuke...@gmail.com> wrote: > Hi All, > > I have a Flink job with a RMQ source, tumbling windows (fires for each > 2s), an aggregator, then a RMQ sink. Incremental RocksDB checkpointing is > enabled with an interval of 5 minutes. > > I was trying to understand Flink failure recovery. My checkpoint X is > started, I have sent one event to my source. As windows are triggered every > 2s, my sink is updated with the aggregated result. But when I checked the > RabbitMQ console, my source queue still had unacked messages. (It is > expected and it is as per design > https://nightlies.apache.org/flink/flink-docs-release-1.19/docs/connectors/datastream/rabbitmq/#rabbitmq-source > ). > > Now I restarted my task manager, as restart happens within the same > checkpoint interval and checkpoint X has not yet completed. The message is > not acknowledged and is sent again. Duplicate processing of events happens. > > How to avoid these duplicates? > > > Thanks > > Banu >
