Re: configuration needed to run twitter(25GB) dataset

When I use fewer partitions, (like 6) It seems that all the task will be assigned to the same machine, because the machine has more than 6 cores.But this will run out of memory. How to set fewer partitions number and use all the machine at the same time? -- View this message in context: http://

Re: Re:Re: [GraphX] The best way to construct a graph

Have you tried to write another similar function like edgeListFile in the same file, and then compile the project again? -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/GraphX-The-best-way-to-construct-a-graph-tp11122p11138.html Sent from the Apache Spark Us

Re: [GraphX] The best way to construct a graph

I think you can try GraphLoader.edgeListFile, and then use join to associate the attributes with each vertex -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/GraphX-The-best-way-to-construct-a-graph-tp11122p11127.html Sent from the Apache Spark User List mail

Re: configuration needed to run twitter(25GB) dataset

Is it possible to reduce the number of edge partitions and exploit parallelism fully at the same time? For example, one partition per node, and the threads in the same node share the same partition. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/configurati

VertexPartition and ShippableVertexPartition

There is a VertexPartition in the EdgePartition,which is created by EdgePartitionBuilder.toEdgePartition. and There is also a ShippableVertexPartition in the VertexRDD. These two Partitions have a lot of common things like index, data and Bitset, why is this necessary? -- View this message in co

Where is the "PowerGraph abstraction"

I download the spark 1.0.1, but I cannot find the "PowerGraph abstraction" mentioned in the GraphX paper. What I can find is the pregel abstraction. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Where-is-the-PowerGraph-abstraction-tp10457.html Sent from t