Are you not just looking for the window() function that creates the sliding-window RDDs in the first place? That DStreams' RDDs give you all elements in the sliding window, and you can compute a mean or variance as you like.
You should be able to do this quite efficiently without recomputing each time by using reduceByWindow and a running mean / stdev formula. On Wed, May 21, 2014 at 1:42 PM, Laeeq Ahmed <laeeqsp...@yahoo.com> wrote: > Hi, > > I want to do union of all RDDs in each window of DStream. I found > Dstream.union and haven't seen anything like DStream.windowRDDUnion. > > Is there any way around it? > > I want to find mean and SD of all values which comes under each sliding > window for which I need to union all the RDDs in each window. This is not a > running mean and SD. > > Regards, > Laeeq >
