Re: Behaviour of CountWindowAll

2016-01-05 Thread Aljoscha Krettek
Hi, I’m afraid this is not possible right now. I’m also not sure about the Evictors as a whole. Using them makes window operations very slow because all elements in a window have to be kept, i.e. window results cannot be pre-aggregated. Cheers, Aljoscha > On 15 Dec 2015, at 12:23, Radu Tudoran

RE: Behaviour of CountWindowAll

2015-12-15 Thread Radu Tudoran
Hi, I believe this question might have been asked before - so sorry for repeating it (I just did not find the discussion on the mailing list). Is it possible somehow to create a new DataStream from the elements that are evicted from a window? A simple use case for this is: We have data

Re: Behaviour of CountWindowAll

2015-12-14 Thread Nirmalya Sengupta
Hello Aljoscha , Thanks again for taking time to explain the behaviour of CountWindowAll(m,n). To be honest, the behaviour seems a bit sketchy to me - and probably it needs a revisit - but if that's the way it is, then that's the way it is! :-) -- Nirmalya -- Software Technologist http://www.l

Re: Behaviour of CountWindowAll

2015-12-14 Thread Aljoscha Krettek
Hi, the current behavior is in fact that the window will be triggered every “slide-size” elements and the computation will take into account the last “window-size” elements. So for a window with window-size 10 and slide-size 5 the window will be triggered every 5 elements. This means that your o

Re: Behaviour of CountWindowAll

2015-12-14 Thread Nirmalya Sengupta
Hello Aljoscha , Thanks for the explanation about the semantics of CountWindowAll's parameters. However, I am thinking about it and what strikes me is this: If I call CountWindowAll(10,5) then what I am instructing Flink to do is to 1) Collect first 10 2) Call max() function *and, *then* begi

Re: Behaviour of CountWindowAll

2015-12-14 Thread Aljoscha Krettek
Hi Nirmalya, when using count windows the window will trigger after “slide-size” elements have been received. So, since in your example, slide-size is set to 1 it will emit a new max for every element received and once it accumulated 4 elements it will start removing one element for every new el