Re: Checkpoint bugs in GraphX

2014-11-10 Thread GuoQiang Li
: "Xu Lijie";; Date: Tue, Nov 11, 2014 11:40 AM To: "GuoQiang Li"; Cc: "user"; "dev"; Subject: Re: Checkpoint bugs in GraphX Nice, we currently encounter a stackoverflow error caused by this bug. We also found that "val partitionsRDD: R

Re: Checkpoint bugs in GraphX

2014-11-10 Thread Xu Lijie
Nice, we currently encounter a stackoverflow error caused by this bug. We also found that "val partitionsRDD: RDD[(PartitionID, EdgePartition[ED, VD])], val targetStorageLevel: StorageLevel = StorageLevel.MEMORY_ONLY)" will not be serialized even without adding @transient. However, transient can

Re: Checkpoint bugs in GraphX

2014-11-10 Thread GuoQiang Li
I have been trying to fix this bug.‍ The related PR: https://github.com/apache/spark/pull/2631‍ -- Original -- From: "Xu Lijie";; Date: Tue, Nov 11, 2014 10:19 AM To: "user"; "dev"; Subject: Checkpoint bugs in GraphX Hi, all. I'm not sure whether someone