Re: Difference among batchDuration, windowDuration, slideDuration

-spark-user-list.1001560.n3.nabble.com/Difference-among-batchDuration-windowDuration-slideDuration-tp9966p22119.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-mai

Re: Difference among batchDuration, windowDuration, slideDuration

ind is that window duration and slide >>> duration have to be multiples of batch duration, IDK if you made that >>> fully >>> clear >>> >>> >>> >>> -- >>> View this message in context: >>> http://apache-spark-user-list.1001560.n3.nabble.com/Difference-among-batchDuration-windowDuration-slideDuration-tp9966p9973.html >>> Sent from the Apache Spark User List mailing list archive at Nabble.com. >>> >> >> >

Re: Difference among batchDuration, windowDuration, slideDuration

n and slide >> duration have to be multiples of batch duration, IDK if you made that >> fully >> clear >> >> >> >> -- >> View this message in context: >> http://apache-spark-user-list.1001560.n3.nabble.com/Difference-among-batchDuration-windowDu

Re: Difference among batchDuration, windowDuration, slideDuration

t; http://apache-spark-user-list.1001560.n3.nabble.com/Difference-among-batchDuration-windowDuration-slideDuration-tp9966p9973.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. >

Re: Difference among batchDuration, windowDuration, slideDuration

The only other thing to keep in mind is that window duration and slide duration have to be multiples of batch duration, IDK if you made that fully clear -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Difference-among-batchDuration-windowDuration

Re: Difference among batchDuration, windowDuration, slideDuration

Here's what I understand: batchDuration: How often should the streaming context update? how many seconds of data should each dstream contain? windowDuration: What size windows are you looking for from this dstream? slideDuration: Once I've given you that slice, how many units forward do you

Difference among batchDuration, windowDuration, slideDuration

When I'm reading the API of spark streaming, I'm confused by the 3 different durations StreamingContext(conf: SparkConf , batchDuration: Duration