Hi all,
So I'm currently figuring out how to accumulate three separate accumulators:
val a:Accumulator
val b:Accumulator
val c:Accumulator
I have an r:RDD[thing] and the code currently reads
r.foreach{
thing =>
a += thing
b += thing
c += thing
}
Ideally, I would much prefer to split this up so that I can separate
concerns. I'm considering creating something along the lines of:
def handleA(a:Accumulator, r:RDD[Thing]){
//a's logic
r.foreach{ thing => a += thing}
}
def handleB(b:Accumulator, r:RDD[Thing]){
//a's logic
r.foreach{ thing => b += thing}
}
and so on. However Im worried that this would cause a performance hit. Does
anyone have any thoughts as to whether this would be a bad idea?
thank you!
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Running-multiple-foreach-loops-tp26256.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org
