Hello,
Is there a way to instruct treeReduce() to reduce RDD partitions on the
same node locally?
In my case, I'm using treeReduce() to reduce map results in parallel. My
reduce function is just arithmetically adding map results (i.e. no notion
of aggregation by key). As far as I understand, a sh
nt to the driver.
I have few nodes and lots of partitions so I think this will give better
performance.
Thank you,
Ayman
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Locality-aware-tree-reduction-tp26885.html
Sent from the Apache Spark User List mailing