Re: context.runJob() was suspended in getPreferredLocations() function

2017-01-01 Thread Liang-Chi Hsieh
Hi, Simply said, you submit another Job in the event thread which will be blocked and can't receive the this job submission event. So your second job submission is never processed, and the getPreferredLocations method is never returned. Fei Hu wrote > Dear all, > > I tried to customize my own

context.runJob() was suspended in getPreferredLocations() function

2016-12-30 Thread Fei Hu
Dear all, I tried to customize my own RDD. In the getPreferredLocations() function, I used the following code to query anonter RDD, which was used as an input to initialize this customized RDD: * val results: Array[Array[DataChunkPartition]] = context.runJob(partitionsRDD, (con