Re: Window Functions and Empty Panes

2017-04-18 Thread Kostas Kloudas
I forgot to say that timers are fault-tolerant. You set them, and Flink takes care of checkpointing and restoring them after failure. The flag will also be fault-tolerant as, i suppose, you will use Flink’s keyed state. For more info, you can check the ProcessFunction documentation that Konstant

Re: Window Functions and Empty Panes

2017-04-18 Thread Kostas Kloudas
No problem! Glad I could help! Kostas > On Apr 18, 2017, at 11:01 PM, Ryan Conway > wrote: > > Hi Kostas, > > Re restarting: I missed that ProcessFunction.OnTimerContext extends > ProcessFunction.Context! Until now my thought was that OnTimerContext did not > provide a means of restarting a

Re: Window Functions and Empty Panes

2017-04-18 Thread Ryan Conway
Hi Kostas, Re restarting: I missed that ProcessFunction.OnTimerContext extends ProcessFunction.Context! Until now my thought was that OnTimerContext did not provide a means of restarting a timer. Re initial timer, you're right, I'll just need to track a boolean in a state variable that notes whet

Re: Window Functions and Empty Panes

2017-04-18 Thread Kostas Kloudas
Hi Ryan, “A periodic window like this requires the ability to start a timer without an element and to restart a timer when fired.” For the second part, i.e. “to restart a timer when fired”, you can re-register the timer in the onTimer() method (set a new timer for “now + T"), so that the next

Re: Window Functions and Empty Panes

2017-04-18 Thread Ryan Conway
Hi Konstantin, Thank you for your response. That is unfortunate as this is a mission-critical component of our application. I tried process functions first, but ran into problems because their timers can only be controlled when processing an element. A periodic window like this requires the abili

Re: Window Functions and Empty Panes

2017-04-17 Thread Konstantin Knauf
Hi Ryan, afaik there is no way to override this behaviour. Did you have a look at the ProcessFunction [1]? Depending on your specific requirements you might be able to re-implement the desired functionality on top of it, but you will have to do all the time-handling yourself and you do not have w

Window Functions and Empty Panes

2017-04-17 Thread Ryan Conway
Hello Flinkers, Is there a means of configuring a windowed stream such that its window function is invoked for empty panes? Looking at the source of WindowOperator in Flink 1.2.0 (line 473), I see that the user's window function is invoked only when the pane isn't empty. I am hoping for a means o