RDD checkpoint failures in case of insufficient memory

2018-03-14 Thread Sergey Zhemzhitsky
Hi there, A while ago running GraphX jobs I've discovered that PeriodicRDDCheckpointer fails with FileNotFoundException's in case of insufficient memory resources. I believe that any iterative job which uses PeriodicRDDCheckpointer (like ML) suffers from the same issue (but it's not visible enoug

RDD checkpoint

2015-07-13 Thread 牛兆捷
The checkpointed RDD computed twice, why not do the checkpoint for the RDD once it is computed? Is there any special reason for this? -- *Regards,* *Zhaojie*