Re: save a histogram to a file

2015-01-23 Thread Sean Owen
As you can see, the result of histogram() is a pair of arrays, since of course it's small. It's not necessary and in fact is huge overkill to make it back into an RDD so you can save it across a bunch of partitions. This isn't a job for Spark, but simple Scala code. Off the top of my head (maybe n

Re: save a histogram to a file

2015-01-23 Thread madhu phatak
Hi, histogram method return normal scala types not a RDD. So you will not have saveAsTextFile. You can use makeRDD method make a rdd out of the data and saveAsObject file val hist = a.histogram(10) val histRDD = sc.makeRDD(hist) histRDD.saveAsObjectFile(path) On Fri, Jan 23, 2015 at 5:37 AM, SK