Hey Peter,
I think that this is actually due to an error-handling issue: if you look
at the stack trace that you posted, the NPE is being thrown from an
error-handling branch of a `finally` block:
@Override public void write(scala.collection.Iterator<Product2<K, V>>
records) throws IOException { boolean success = false; try { while
(records.hasNext())
{ insertRecordIntoSorter(records.next()); } closeAndWriteOutput(); success =
true; } finally { if (!success) { sorter.cleanupAfterError(); // <---- this
is the line throwing the error } } }
I suspect that what's happening is that an exception is being thrown from
user / upstream code in the initial call to records.next(), but the
error-handling block is failing because sorter == null since we haven't
initialized it yet.
I'm going to file a JIRA for this and will try to add a set of regression
tests to the ShuffleSuite to make sure exceptions from user code aren't
swallowed like this.
On Fri, Jun 19, 2015 at 11:36 AM, Peter Rudenko <[email protected]>
wrote:
> Hi want to try new tungsten-sort shuffle manager, but on 1 stage
> executors start to die with NPE:
>
> 15/06/19 17:53:35 WARN TaskSetManager: Lost task 38.0 in stage 41.0 (TID
> 3176, ip-10-50-225-214.ec2.internal): java.lang.NullPointerException
> at
> org.apache.spark.shuffle.unsafe.UnsafeShuffleWriter.write(UnsafeShuffleWriter.java:151)
> at
> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:70)
> at
> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
> at org.apache.spark.scheduler.Task.run(Task.scala:70)
> at
> org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
>
>
> Any suggestions?
>
> Thanks,
> Peter Rudenko
>
