On Sun, Apr 20, 2014 at 6:18 PM, Qi Song <[email protected]> wrote:
> I was running some pagerank tests of GraphX in my 8 nodes cluster. I > allocated each worker 32G memory and 8 CPU cores. The LiveJournal dataset > used 370s, which in my mind is reasonable. But when I tried the > com-Friendster data ( http://snap.stanford.edu/data/com-Friendster.html ) > with 65608366 nodes and 1806067135 edges, it took more than 70 hours and is > still running. Maybe GraphX is running out of memory? You can check memory usage and total memory available to the workers by looking at the Executors tab in the Spark UI. Ankur <http://www.ankurdave.com/>
