Re: Behavior of SlidingProessingTimeWindow with CountTrigger

2016-03-14 Thread Vishnu Viswanath
Hi Aljoscha, Thank you for the explanation and the link on IBM infosphere. That explains whey I am seeing (a,3) and (b,3) in my example. Yes, the name Evictor is confusing. Thanks and Regards, Vishnu Viswanath, www.vishnuviswanath.com On Mon, Mar 14, 2016 at 11:24 AM, Aljoscha Krettek wrote:

Re: Behavior of SlidingProessingTimeWindow with CountTrigger

2016-03-14 Thread Aljoscha Krettek
Hi, sure, the evictors are a bit confusing (especially the fact that they are called evictors). They should more correctly called “Keepers”. The process is the following: 1. Trigger Fires 2. Evictor decides what elements to keep, so a CountEvictor.of(3) says, keep only three elements, all other

Re: Behavior of SlidingProessingTimeWindow with CountTrigger

2016-03-14 Thread Vishnu Viswanath
Hi Aijoscha, Wow, great illustration. That was very clear explanation. Yes, I did enter the elements fast for case b and I was seeing more of case As. Also, sometimes I have seen a window getting triggered when I enter 1 or 2 elements, I believe that is expansion of case A, w.r.t to window 2. Al

Re: Behavior of SlidingProessingTimeWindow with CountTrigger

2016-03-14 Thread Aljoscha Krettek
Hi, I created a visualization to help explain the situation: http://s21.postimg.org/dofhcw52f/window_example.png The SlidingProcessingTimeWindows assigner assigns elements to windows based on the current processing time. The CountTrigger only fires if a window contains 5 elements (or more). In

Behavior of SlidingProessingTimeWindow with CountTrigger

2016-03-12 Thread Vishnu Viswanath
Hi All, I have the below code val sev = StreamExecutionEnvironment.getExecutionEnvironment val socTextStream = sev.socketTextStream("localhost",) val counts = socTextStream.flatMap{_.split("\\s")} .map { (_, 1) } .keyBy(0) .window(SlidingProcessingTimeWindows.of(Time.seconds(20),Time