Catching executor exception from executor in driver

[Justin Yip]

Hello,

I would like to know if there is a way of catching exception throw from
executor exception from the driver program. Here is an example:

try {
  val x = sc.parallelize(Seq(1,2,3)).map(e => e / 0).collect
} catch {
  case e: SparkException => {
    println(s"ERROR: $e")
    println(s"CAUSE: ${e.getCause}")
  }
}

Output:
ERROR: org.apache.spark.SparkException: Job aborted due to stage failure:
Task 1 in stage 1.0 failed 4 times, most recent failure: Lost task 1.3 in
stage 1.0 (TID 15, pio1.c.ace-lotus-714.internal):
java.lang.ArithmeticException: / by zero
at
$line71.$read$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$anonfun$1.apply$mcII$sp(<console>:51)
...
CAUSE: null

The exception cause is a null value. Is there any way that I can catch the
ArithmeticException?

Thanks

Justin




--
View this message in context: 
http://apache-spark-user-list.1001560.n3.nabble.com/Catching-executor-exception-from-executor-in-driver-tp22495.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.