I apologize in advance if the answer to this question is obvious, but here goes ...
The GraphX merge and reduction functions take two objects of type A and return one object of type A. Is there a nice, efficient way to implement a reduction function like median or mode? These reduction functions must identify each unique input and use the number of times each one appears to set the output. I guess A could include a collection, but doing a bunch of collection allocation, merge and discards for each vertex sounds horrible. As long as I'm at it, is there a nice way to pass a vertex's state from the last several iterations to Pregel's vprog so they can be among the inputs used to set the current vertex state? Putting an array into A seems sort of right, but can A be updated by vprog? Thanks, Dan -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Messy-GraphX-merge-reduce-functions-tp2109.html Sent from the Apache Spark User List mailing list archive at Nabble.com.
