Re: GraphX optimizations

2016-03-06 Thread Takeshi Yamamuro
Hi, mapReduceTriplets you said has been removed in master and you need to use a newer api, aggregateMessages, instead of it (See SPARK-3936 and SPARK-12995 for details). The memory-based shuffling opt. is a topic of not only graphx but also spark itself. You can see SPARK-3376 for related discussi

GraphX optimizations

2016-03-04 Thread Khaled Ammar
Hi all, I wonder if the optimizations mentioned in the GraphX paper ( https://amplab.cs.berkeley.edu/wp-content/uploads/2014/09/graphx.pdf ) are currently implemented. In particular, I am looking for mrTriplets optimizations and memory-based shuffle. -- Thanks, -Khaled