Re: Windows and Watermarks Clarification

2016-09-01 Thread Aljoscha Krettek
s allowedLateness(Time.seconds(x)) then >> evaluated relative to maxEventTime - lastWaterMarkTime. So if (maxEventTime >> - lastWaterMarkTime) > x * 1000 then the window is evaluated? >> >> >> Paul >> -- >> *From:* Fabian Hueske

Re: Windows and Watermarks Clarification

2016-09-01 Thread Fabian Hueske
entTime - lastWaterMarkTime. So if (maxEventTime > - lastWaterMarkTime) > x * 1000 then the window is evaluated? > > > Paul > -- > *From:* Fabian Hueske > *Sent:* Thursday, September 1, 2016 1:25:55 PM > *To:* user@flink.apache.org > *Subje

Re: Windows and Watermarks Clarification

2016-09-01 Thread Paul Joireman
ent: Thursday, September 1, 2016 1:25:55 PM To: user@flink.apache.org Subject: Re: Windows and Watermarks Clarification Hi Paul, BoundedOutOfOrdernessTimestampExtractor implements the AssignerWithPeriodicWatermarks interface. This means, Flink will ask the assigner in regular intervals (configurable

Re: Windows and Watermarks Clarification

2016-09-01 Thread Fabian Hueske
Hi Paul, BoundedOutOfOrdernessTimestampExtractor implements the AssignerWithPeriodicWatermarks interface. This means, Flink will ask the assigner in regular intervals (configurable via StreamExecutionEnvironment.getConfig().setAutoWatermarkInterval()) for the current watermark. The watermark will

Windows and Watermarks Clarification

2016-09-01 Thread Paul Joireman
Hi all, Just a point of clarification on how watermarks are generated. I'd like to use a SlidingEventTime window of say 5 minutes with a 30 second slide. The incoming data stream has elements from which I can extract the timestamp but they may come out of order so I chose to implement the f