Re: Tumbling time window cannot group events properly

2016-07-06 Thread Yukun Guo
You're right, I forgot to check that the "events in this window" line actually showed the number of events inside each window was what I expected, despite being printed a bit out of order. Thank you for the help! On 5 July 2016 at 17:37, Aljoscha Krettek wrote: > The order in which elements are

Re: Tumbling time window cannot group events properly

2016-07-05 Thread Aljoscha Krettek
The order in which elements are added to internal buffers and the point in time when FoldFunction.fold() is called don't indicate to which window elements are added. Flink will internally keep a buffer for each window and emit the window once the watermark passes the end of the window. In your case

Re: Tumbling time window cannot group events properly

2016-07-04 Thread Yukun Guo
The output is the timestamps of events in string. (For convenience, the payload of each event is exactly the timestamp of it.) As soon as the folding of a time window is finished, the code will print "# events in this window" indicating the end of the window. The 10s windows should be [19:10:40, 1

Re: Tumbling time window cannot group events properly

2016-07-04 Thread Aljoscha Krettek
Could you please elaborate a bit on what exactly the output means and how you derive that events are leaking into the previous window? On Mon, 4 Jul 2016 at 13:20 Yukun Guo wrote: > Thanks for the information. Strange enough, after I set the time > characteristic to EventTime, the events are lea

Re: Tumbling time window cannot group events properly

2016-07-04 Thread Yukun Guo
Thanks for the information. Strange enough, after I set the time characteristic to EventTime, the events are leaking into the previous window: ... Mon, 04 Jul 2016 19:10:49 CST Mon, 04 Jul 2016 19:10:50 CST # ? Mon, 04 Jul 2016 19:10:50 CST Mon, 04 Jul 2016 19:10:50 CST Mon, 04 Jul 2016 19:10:50 C

Re: Tumbling time window cannot group events properly

2016-07-04 Thread Aljoscha Krettek
Hi, I think it should be as simple as setting event time as the stream time characteristic: env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime) The problem is that .timeWindow(Time.seconds(10)) will use processing time if you don't specify a time characteristic. You can enforce using an

Tumbling time window cannot group events properly

2016-07-03 Thread Yukun Guo
Hi, I wrote a program which constructs a WindowedStream to compute periodic data statistics every 10 seconds. However, I found that events have not been strictly grouped into windows of 10s duration, i.e., some events are leaking into the adjacent window. The output is like this: Mon, 04 Jul 201