Re: Why a program would receive null from send message of mapReduceTriplets

2015-02-13 Thread James
I have a question: *How could the attributes of triplets of a graph get update after mapVertices() func? * My code ``` // Initial the graph, assign a counter to each vertex that contains the vertex id only var anfGraph = graph.mapVertices { case (vid, _) => val counter = new HyperLogLog(5) c

Re: Why a program would receive null from send message of mapReduceTriplets

2015-02-12 Thread James
I am trying to run the data on spark-shell mode to find whether there is something wrong in the code or data. As I could only reproduce the error on a 50B edge graph. 2015-02-13 14:50 GMT+08:00 Reynold Xin : > Then maybe you actually had a null in your vertex attribute? > > > On Thu, Feb 12, 2015

Re: Why a program would receive null from send message of mapReduceTriplets

2015-02-12 Thread Reynold Xin
Then maybe you actually had a null in your vertex attribute? On Thu, Feb 12, 2015 at 10:47 PM, James wrote: > I changed the mapReduceTriplets() func to aggregateMessages(), but it > still failed. > > > 2015-02-13 6:52 GMT+08:00 Reynold Xin : > >> Can you use the new aggregateNeighbors method? I

Re: Why a program would receive null from send message of mapReduceTriplets

2015-02-12 Thread James
I changed the mapReduceTriplets() func to aggregateMessages(), but it still failed. 2015-02-13 6:52 GMT+08:00 Reynold Xin : > Can you use the new aggregateNeighbors method? I suspect the null is > coming from "automatic join elimination", which detects bytecode to see if > you need the src or ds

Re: Why a program would receive null from send message of mapReduceTriplets

2015-02-12 Thread Reynold Xin
Can you use the new aggregateNeighbors method? I suspect the null is coming from "automatic join elimination", which detects bytecode to see if you need the src or dst vertex data. Occasionally it can fail to detect. In the new aggregateNeighbors API, the caller needs to explicitly specifying that,

Why a program would receive null from send message of mapReduceTriplets

2015-02-12 Thread James
Hello, When I am running the code on a much bigger size graph, I met NullPointerException. I found that is because the sendMessage() function receive a triplet that edge.srcAttr or edge.dstAttr is null. Thus I wonder why it will happen as I am sure every vertices have a attr. Any returns is appr