Re: why one of Stage is into Skipped section instead of Completed

2015-12-27 Thread Prem Spark
Thank you Silvio for the update. On Sat, Dec 26, 2015 at 1:14 PM, Silvio Fiorito < silvio.fior...@granturing.com> wrote: > Skipped stages result from existing shuffle output of a stage when > re-running a transformation. The executors will have the output of the > stage in their local dirs and Sp

Re: why one of Stage is into Skipped section instead of Completed

2015-12-26 Thread Silvio Fiorito
Skipped stages result from existing shuffle output of a stage when re-running a transformation. The executors will have the output of the stage in their local dirs and Spark recognizes that, so rather than re-computing, it will start from the following stage. So, this is a good thing in that you