Hi again,
Thank you for your suggestion :)
I've tried to implement this method but I'm stuck trying to union the
payload before creating the graph.
Below is a really simplified snippet of what have worked so far.
//Reading the articles given in json format
val articles = sqlContext.jsonFile(path)
articles.registerTempTable("TblArticles")
//Creating two abstract vertex types
class Vertices()
case class Paper(pid: Long, ptitle:String, aid:String) extends Vertices
case class Author(aid:String, aname:String) extends Vertices
//Using the subclasses as payload with only the fields I need
val filteredPapers=sqlContext.sql("SELECT id,title,authorid FROM
TblArticles")
val vP=filteredPapers.map(line => new Paper(line.getLong(0),
line.getString(1), line.getString(2)).cache
val filteredAuthors=sqlContext.sql("SELECT authorid, name FROM TblArticles")
val vA=filteredAuthors.map(line => new Author(line.getString(0),
line.getString(1)).cache
//Let's assume for now there's only one edge type - "AUTHORS"
val Authedges: RDD[Edge[String]] = vP.map { t => Edge(t.authorid.toLong,
t.id, "AUTHORS")}
1. vP and vA are RDDs, how do I convert them to vertexRDDs and perform the
union?
2. Should the graph be then created as
val graph=Graph[Vertices,Authedges,String]
Thanks!
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/GraphX-Types-for-the-Nodes-and-Edges-tp15486p15842.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
