Re: Finding moving average using Spark and Scala

2015-07-17 Thread Anupam Bagchi
ong)bollingerBytesTransferrred.getLastUpperThreshold(), (long)bollingerBytesIn.getLastLowerThreshold(), (long)bollingerBytesOut.getLastLowerThreshold(), (long)bollingerBytesTransferrred.getLastLowerThreshold()); } }); deviceCharacteristics.values().saveAsTextFile

Re: Finding moving average using Spark and Scala

2015-07-13 Thread Anupam Bagchi
? Anupam Bagchi > On Jul 13, 2015, at 7:06 PM, Feynman Liang wrote: > > Dimensions mismatch when adding new sample. Expecting 8 but got 14. > > Make sure all the vectors you are summarizing over have the same dimension. > > Why would you want to write a MultivariateOnlineSumm

Re: Finding moving average using Spark and Scala

2015-07-13 Thread Anupam Bagchi
tell where exactly I went wrong. Also, how do I take the MultivariateOnlineSummary object and write it to HDFS? I have the MultivariateOnlineSummary object with me, but I really need an RDD to call saveAsTextFile() on it. Anupam Bagchi (c) 408.431.0780 (h) 408-873-7909 > On Jul 13, 2015, at 4:

Re: Finding moving average using Spark and Scala

2015-07-13 Thread Anupam Bagchi
"," + stddev) }) //val vector: Vector = Vectors.dense(byteValues) //println(vector) //val summary: MultivariateStatisticalSummary = Statistics.colStats(vector) sc.stop() } } Can you show me how to write the ‘foreach’ loop in a Spark-friendly way? Thanks a lot

Finding moving average using Spark and Scala

2015-07-13 Thread Anupam Bagchi
diate values as RDD so that I can directly use the RDD methods to do the job. - At the end I also need to write the results to HDFS for which there is a method provided on the RDD class to do so, which is another reason I would like to retain everything as RDD. Thanks in advance for your help. Anupam Bagchi 

Moving average using Spark and Scala

2015-07-11 Thread Anupam Bagchi
he RDD methods to do the job. At the end I also need to write the results to HDFS for which there is a method provided on the RDD class to do so, which is another reason I would like to retain everything as RDD. Thanks in advance for your help. Anupam Bagchi

Calculating moving average of dataset in Apache Spark and Scala

2015-07-11 Thread Anupam Bagchi
irectly use the RDD methods to do the job. At the end I also need to write the results to HDFS for which there is a method provided on the RDD class to do so, which is another reason I would like to retain everything as RDD. Anupam Bagchi