Re: Playing with EventTime in DataStreams

2016-02-26 Thread Stephan Ewen
Nice catch, actually. I think we should let the timestamp extracting operator emit the current watermark prior to shutting down. On Fri, Feb 26, 2016 at 11:49 AM, Aljoscha Krettek wrote: > Hi, > I think the problem is that the source finished before the extractor has > the chance to emit even a

Re: Playing with EventTime in DataStreams

2016-02-26 Thread Aljoscha Krettek
Hi, I think the problem is that the source finished before the extractor has the chance to emit even a single watermark. This means that the topology will shut down and the window operator does not emit in-flight windows upon shutdown. Cheers, Aljoscha > On 26 Feb 2016, at 11:40, Nam-Luc Tran w

Re: Playing with EventTime in DataStreams

2016-02-26 Thread Nam-Luc Tran
Great, that did it, thanks Robert ;) While I'm at it: Sometimes results are correctly returned, sometimes, the output of the job (print or writeAsText) is plain empty, like the job finished too quickly before the results are written. One way of "forcing" results to happen is to insert a "delay" i

Re: Playing with EventTime in DataStreams

2016-02-25 Thread Robert Metzger
Hi, I had a similar issue recently. Instead of input.assignTimestampsAndWatermarks you have to do: input = input.assignTimestampsAndWatermarks On Thu, Feb 25, 2016 at 6:14 PM, Nam-Luc Tran wrote: > Hello everyone, > > I am currently playing with streams which timestamp is defined by > Event

Playing with EventTime in DataStreams

2016-02-25 Thread Nam-Luc Tran
Hello everyone, I am currently playing with streams which timestamp is defined by EventTime. I currently have the following code: final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment(); env.getConfig().enableTimestamps();//.setAutoWatermarkInterval