Hi,
I have a setup where I'm feeding a rolling window with event time:
https://gist.github.com/lofifnc/dd946fef6f4b3eb25ef1 (Obviously i'm using
Flinkspector)
The first case behaves as expected I'm emitting three records which are all
in the time frame of the first window triggering:
("hans", "elephant", 15) at second 0 with watermark 0
("susi", "arctic", 20) at second 30 with watermark 30
("pete", "elephant", 40) at second 50 with watermark 50
(You can see this as well in the console output).
The result is I'm getting:
(susi,arctic,20)
(hans,elephant,55)
Each 5 times as expected.
In the second case it gets interesting as i'm emitting a 4th record which
does not get evaluated within the first triggering of the window:
("hans", "elephant", 15) at second 0 with watermark 0
("susi", "arctic", 20) at second 30 with watermark 30
("pete", "elephant", 40) at second 50 with watermark 50
("grace", "arctic", 25) at second 90 with watermark 90
Now the second time the of the window is evaluated looks like this:
(grace,arctic,25)
(susi,arctic,45)
(hans,elephant,55)
The 4th record is emitted in a non aggregated form, but the grouping for the
"arctic" key has been updated.
In the third case it gets really interesting... I'm emitting a 5th record
which will fall into the third evaluation of the window.
This time the record is shown twice in the output:
(sven,elephant,5)
(grace,arctic,25)
(sven,elephant,5)
(hans,elephant,60)
I played around with this a little bit and if you insert a record into the
4th evaluation it is shown 3 times, in the 5th it will be in the output 4
times and so on. The test itself is deterministic between test runs.
Lastly i've been able to reproduce the behavior using only vanilla flink so
I'm fairly certain this is not a side effect of flinkspector.
I'm slowly getting a headache trying to wrap my head around why I'm seeing
this behavior, but I can't find a satisfying explanation.
Best Alex!
--
View this message in context:
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Trying-to-comprehend-rolling-windows-event-time-tp5034.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at
Nabble.com.
