Hi, I am having few issues with event time windowing. Here is my scenario, data is ingested from a kafka consumer and then keyed by user followed by a Tumbling event window for 10 seconds. The max lag tolerance limit is 1 second.
I have the BoundedOutOfOrdernessGenerator that extends AssignerWithPeriodicWatermarks to assign watermarks. When the data is ingested even after receiving multiple messages per user the window never gets evicted. What am I missing here? https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/event_timestamps_watermarks.html .window(TumblingEventTimeWindows.of(Time.seconds(10))) The other issue I am having is there will be scenarios where there is just one message per user for more than a minute. In that case I want the window content to be evicted after the defined window interval of 10 seconds. Is there a way to evict the window data even when there is no more incoming data for that key? I have tried setAutoWatermarkInterval(10000) but still no luck. How do I get the current watermark to be displayed in the flink dashboard UI under watermarks sections? Currently it shows no watermarks. Also is there a way to count the number of messages that missed the time window due to late arrival? Thanks and appreciate all the help.
