Re: processing large number of files

2014-09-30 Thread Liquan Pei
rray() > > // aggregate the count for the year > ucount = ucount + ar(0)._2 > } > > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/processing-large-number-of-files-tp15429.html > Sent from the Apache

processing large number of files

2014-09-30 Thread SK
extract the count. Need to find if there is an easier way to do this. var ar = ut_rdd.toArray() // aggregate the count for the year ucount = ucount + ar(0)._2 } -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/processing-large-numb