Re: My spark job runs faster in spark 1.6 and much slower in spark 2.0

ully long in spark 2.0. >> >> I am using spark 1.6 & spark 2.0 on HDP 2.5.3 >> >> >> >> >> -- >> View this message in context: http://apache-spark-user-list. >> 1001560.n3.nabble.com/My-spark-job-runs-faster-in-spark-1-6- >> and-much-slower-in-spark-2-0-tp28390.html >> Sent from the Apache Spark User List mailing list archive at Nabble.com. >> >> - >> To unsubscribe e-mail: user-unsubscr...@spark.apache.org >> >> > -- Regards, Arun Kumar Natva

Spark GraphX remembering old message

Hi I am trying to find belief for a graph using GraphX Pragel implementation .My use case is like if vertex 2,3,4 are sending message m2,m3,m4 to vertex 6 .In vertex 6 I will multiple all the messages (m2*m3*m4) =m6 and then from vertex6 the message (m6/m2) will be send to vertex 2,m6/m3 to vert

Spark 0.9.2 sbt build issue

Hi While trying to build spark0.9.2 using sbt the build is failing due to the non resolving of most of the libraries .sbt cannot fetch the libraries in the specified location. Please tel me what changes are required to build spark using sbt Regards Arun

Re: GraphX Pragel implementation

this. Regards Arun On Fri, Jul 25, 2014 at 3:04 PM, Arun Kumar wrote: > Hi > > Thanks for the quick response.I am new to scala and some help will be > required > > Regards > -Arun > > > On Fri, Jul 25, 2014 at 10:37 AM, Ankur Dave wrote: > >> On Thu, Jul

Re: GraphX Pragel implementation

Hi Thanks for the quick response.I am new to scala and some help will be required Regards -Arun On Fri, Jul 25, 2014 at 10:37 AM, Ankur Dave wrote: > On Thu, Jul 24, 2014 at 9:52 AM, Arun Kumar wrote: > >> While using pregel API for Iterations how to figure out which super

Re: GraphX Pragel implementation

Hi While using pregel API for Iterations how to figure out which super step the iteration currently in. Regards Arun On Thu, Jul 17, 2014 at 4:24 PM, Arun Kumar wrote: > Hi > > > > I am trying to implement belief propagation algorithm in GraphX using the > pragel API.

Re: GraphX Pragel implementation

Thanks On Fri, Jul 18, 2014 at 12:22 AM, Ankur Dave wrote: > If your sendMsg function needs to know the incoming messages as well as > the vertex value, you could define VD to be a tuple of the vertex value and > the last received message. The vprog function would then store the incoming > mess

GraphX Pragel implementation

Hi I am trying to implement belief propagation algorithm in GraphX using the pragel API. *def* pregel[A] (initialMsg*:* A, maxIter*:* Int = *Int*.*MaxValue*, activeDir*:* EdgeDirection = *EdgeDirection*.*Out*) (vprog*:* (VertexId, VD, A) *=>* *VD*, sendMsg*