Have you tried calling coalesce() before saveAsTextFile ? Cheers
On Wed, Dec 2, 2015 at 3:15 PM, Ram VISWANADHA < ram.viswana...@dailymotion.com> wrote: > JavaRDD.saveAsTextFile is taking a long time to succeed. There are 10 > tasks, the first 9 complete in a reasonable time but the last task is > taking a long time to complete. The last task contains the maximum number > of records like 90% of the total number of records. Is there any way to > parallelize the execution by increasing the number of tasks or evenly > distributing the number of records to different tasks? > > Thanks in advance. > > Best Regards, > Ram >
