Hello, I've been given some flink application code and asked to implement and ensure that our query is updated for late arriving entries. We're currently creating a table using a Tumbling SQL query similar to the first example in
https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/table/sql.html#group-windows We then turn the result table back into a datastream using toAppendStream, and eventually add a derivative stream to a sink. We've configured TimeCharacteristic to event-time processing. >From reading the documentation I was trying to configure using withIdleStateRetentionTime, with the expectation that this setting would allow me to deal with late arrivals past a given watermark time, but within the retention time. Then to test this I created a simple source which triggers the watermark, so that I'd have next a late arrival. However so far the watermark seems to cause something to discriminate the late arrival. Then in my test sink where I'm trying to capture all emitted outputs, and hopefully the updated value I don't find one. So it seems that my understanding of how to deal with late events, or my test platform is wrong. Can anyone recognize what I'm doing wrong? Best, Colin Williams
