I think there might be yet another log to check -- the actual executor log.
I'm not quite sure where are stored but I believe it's under spark.local.dir (by default /tmp) and then look for folders like app-... and under there should be the executor number and logs. Might be easier to click through the UI (I think localhost:8080) click the worker and it will show you all the executors that were started on that worker. Click an executor that has failed at the error log there. I believe that log starts with the message: Spark Executor Command: ... I was wrestling with something similar this morning, I believe because of ports... -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Job-aborted-due-to-stage-failure-Master-removed-our-application-FAILED-tp12573p12586.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
