Re: Eventime window

2017-08-02 Thread Eron Wright
Note that the built-in `BoundedOutOfOrdernessTimestampExtractor` generates watermarks based only on the timestamp of incoming events. Without new events, `BoundedOutOfOrdernessTimestampExtractor` will not advance the event-time clock. That explains why the window doesn't trigger immediately after

Re: Eventime window

2017-08-02 Thread Govindarajan Srinivasaraghavan
Thanks Timo. Basically my requirement is based on event time the window has to be created but the trigger has to happen either when it has received the next element >10s or 10s has passed. Exactly the same way as you described. Let me try the AssignerWithPeriodicWatermarks approach. Thanks, Go

Re: Eventime window

2017-08-02 Thread Timo Walther
I forgot about the AssignerWithPeriodicWatermarks [1]. I think it could solve your problem easily. Timo [1] https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/event_timestamps_watermarks.html#with-periodic-watermarks Am 02.08.17 um 16:30 schrieb Timo Walther: The question is wha

Re: Eventime window

2017-08-02 Thread Timo Walther
The question is what defines your `10 seconds`. In event-time the incoming events determine when 10 seconds have passed. Your description sounds like you want to have results after 10 seconds wall-clock/processing-time. So either you use a processing-time window or you implement a custom trigge

Re: Eventime window

2017-08-02 Thread Govindarajan Srinivasaraghavan
Thanks Timo. The next message will arrive only after a minute or so. Is there a way to evict whatever is there in window buffer just after 10 seconds irrespective of whether a new message arrives or not. Thanks, Govind > On Aug 2, 2017, at 6:56 AM, Timo Walther wrote: > > Hi Govind, > > if

Re: Eventime window

2017-08-02 Thread Timo Walther
Hi Govind, if the window is not triggered, this usually indicates that your timestamp and watermark assignment is not correct. According to your description, I don't think that you need a custom trigger/evictor. How often do events arrive from one device? There must be another event from the

Eventime window

2017-08-01 Thread Govindarajan Srinivasaraghavan
Hi, I have few questions regarding event time windowing. My scenario is devices from various timezones will send messages with timestamp and I need to create a window per device for 10 seconds. The messages will mostly arrive in order. Here is my sample code to perform windowing and aggregating t