I am facing different kinds of java.lang.ClassNotFoundException when trying to
run spark on mesos. One error has to do with
org.apache.spark.executor.MesosExecutorBackend. Another has to do with
org.apache.spark.serializer.JavaSerializer. I see other people complaining
about similar issues.
I tried with different version of spark distribution - 0.9.0 and 1.0.0-SNAPSHOT
and faced the same problem. I think the reason for this is is related to the
error below.
$ jar -xf spark-assembly_2.10-0.9.0-incubating-hadoop2.2.0.jar
java.io.IOException: META-INF/license : could not create directory
at sun.tools.jar.Main.extractFile(Main.java:907)
at sun.tools.jar.Main.extract(Main.java:850)
at sun.tools.jar.Main.run(Main.java:240)
at sun.tools.jar.Main.main(Main.java:1147)
This error happens with all the jars that I created. But the classes that are
already generated is different in the different cases. If JavaSerializer is not
already extracted before encountering META-INF/license, then that class is not
found during execution. If MesosExecutorBackend is not found, then that class
shows up in the mesos slave error logs. Can someone confirm if this is a valid
cause for the problem I am seeing? Any way I can debug this further?
— Bharath
