Actually your Pregel code works for me:
import org.apache.spark._
import org.apache.spark.graphx._
import org.apache.spark.rdd.RDD
val vertexlist = Array((1L,"One"), (2L,"Two"), (3L,"Three"),
(4L,"Four"),(5L,"Five"),(6L,"Six"))
val edgelist = Array(Edge(6,5,"6 to 5"),Edge(5,4,"5 to 4"),Edge(4,3,"4 to 3"),
Edge(3,2,"3 to 2"), Edge(2,1,"2 to 1"))
val vertices: RDD[(VertexId, String)] = sc.parallelize(vertexlist)
val edges = sc.parallelize(edgelist)
val graph = Graph(vertices, edges)
val parentGraph = Pregel(
graph.mapVertices((id, attr) => Set[VertexId]()),
Set[VertexId](),
Int.MaxValue,
EdgeDirection.Out)(
(id, attr, msg) => (msg ++ attr),
edge => { if (edge.srcId != edge.dstId)
{ Iterator((edge.dstId, (edge.srcAttr + edge.srcId)))
}
else Iterator.empty
},
(a, b) => (a ++ b))
parentGraph.vertices.collect.foreach(println(_))
Output:
(4,Set(6, 5))
(1,Set(5, 6, 2, 3, 4))
(5,Set(6))
(6,Set())
(2,Set(6, 5, 4, 3))
(3,Set(6, 5, 4))
Maybe your data.csv has edges the wrong way round
Robin
> On 3 Mar 2015, at 16:32, Madabhattula Rajesh Kumar <[email protected]>
> wrote:
>
> Hi,
>
> I have tried below program using pergel API but I'm not able to get my
> required output. I'm getting exactly reverse output which I'm expecting.
>
> // Creating graph using above mail mentioned edgefile
> val graph: Graph[Int, Int] = GraphLoader.edgeListFile(sc,
> "/home/rajesh/Downloads/graphdata/data.csv").cache()
>
> val parentGraph = Pregel(
> graph.mapVertices((id, attr) => Set[VertexId]()),
> Set[VertexId](),
> Int.MaxValue,
> EdgeDirection.Out)(
> (id, attr, msg) => (msg ++ attr),
> edge => { if (edge.srcId != edge.dstId)
> { Iterator((edge.dstId, (edge.srcAttr + edge.srcId)))
> }
> else Iterator.empty
> },
> (a, b) => (a ++ b))
> parentGraph.vertices.collect.foreach(println(_))
>
> Output :
>
> (4,Set(1, 2, 3))
> (1,Set())
> (6,Set(5, 1, 2, 3, 4))
> (3,Set(1, 2))
> (5,Set(1, 2, 3, 4))
> (2,Set(1))
>
> But I'm looking below output.
>
> (4,Set(5, 6))
> (1,Set(2, 3, 4, 5, 6))
> (6,Set())
> (3,Set(4, 5, 6))
> (5,Set(6))
> (2,Set(3, 4, 5, 6))
>
> Could you please correct me where I'm doing wrong.
>
> Regards,
> Rajesh
>
>
> On Tue, Mar 3, 2015 at 8:42 PM, Madabhattula Rajesh Kumar
> <[email protected] <mailto:[email protected]>> wrote:
> Hi Robin,
>
> Thank you for your response. Please find below my question. I have a below
> edge file
>
> Source Vertex Destination Vertex
> 1 2
> 2 3
> 3 4
> 4 5
> 5 6
> 6 6
>
> In this graph 1st vertex is connected to 2nd vertex, 2nd Vertex is connected
> to 3rd vertex,..... 6th vertex is connected to 6th vertex. So 6th vertex is a
> root node. Please find below graph
>
> <image.png>
> In this graph, How can I compute the 1st vertex parents like 2,3,4,5,6.
> Similarly 2nd vertex parents like 3,4,5,6 .... 6th vertex parent like 6
> because this is the root node.
>
> I'm planning to use pergel API but I'm not able to define messages and vertex
> program in that API. Could you please help me on this.
>
> Please let me know if you need more information.
>
> Regards,
> Rajesh
>
>
> On Tue, Mar 3, 2015 at 8:15 PM, Robin East <[email protected]
> <mailto:[email protected]>> wrote:
> Rajesh
>
> I'm not sure if I can help you, however I don't even understand the question.
> Could you restate what you are trying to do.
>
> Sent from my iPhone
>
> On 2 Mar 2015, at 11:17, Madabhattula Rajesh Kumar <[email protected]
> <mailto:[email protected]>> wrote:
>
>> Hi,
>>
>> I have a below edge list. How to find the parents path for every vertex?
>>
>> Example :
>>
>> Vertex 1 path : 2, 3, 4, 5, 6
>> Vertex 2 path : 3, 4, 5, 6
>> Vertex 3 path : 4,5,6
>> vertex 4 path : 5,6
>> vertex 5 path : 6
>>
>> Could you please let me know how to do this? (or) Any suggestion
>>
>> Source Vertex Destination Vertex
>> 1 2
>> 2 3
>> 3 4
>> 4 5
>> 5 6
>>
>> Regards,
>> Rajesh
>
>
