Re: about window length and slide interval

2015-01-14 Thread Hoai-Thu Vuong
Oh I see, thank you all guys. On Wed Jan 14 2015 at 4:35:16 PM Sean Owen wrote: > You're just describing the normal operation of Spark Streaming them. > Windowing is for when you want overlapping intervals. Here you simply > do not window, and ask for intervals of whatever time you want. You > g

Re: about window length and slide interval

2015-01-14 Thread Sean Owen
You're just describing the normal operation of Spark Streaming them. Windowing is for when you want overlapping intervals. Here you simply do not window, and ask for intervals of whatever time you want. You get non-overlapping RDDs. On Wed, Jan 14, 2015 at 9:26 AM, Hoai-Thu Vuong wrote: > I would

Re: about window length and slide interval

2015-01-14 Thread Hoai-Thu Vuong
I would like to counting value in non overlap window, so that I think, I can do it with same value of window length and slide interval, (note that, this value is multiple of batch interval). However, nothing is calculated. Thanks! On Wed Jan 14 2015 at 4:16:27 PM wrote: > Why would you want to

Re: about window length and slide interval

2015-01-14 Thread francois . garillot
Why would you want to ? That would be equivalent to setting your batch interval to the same duration you’re suggesting to specify for both window length and slide interval, IIUC. Here’s a nice explanation of windowing by TD : https://groups.google.com/forum/#!topic/spark-users/GQoxJHAAtX4