This inevitably means the run-time classpath includes a different copy of the same library/class as something in your uber jar and the different version is taking precedence. Here it's Commons HttpComponents. Where exactly it's coming from is specific to your deployment, but that's the issue.
On Sat, Jun 21, 2014 at 9:30 AM, Peng Cheng <pc...@uow.edu.au> wrote: > I have a Spark application that runs perfectly in local mode with 8 threads, > but when deployed on a single-node cluster. It gives the following error: > > ROR TaskSchedulerImpl: Lost executor 0 on 192.168.42.202: Uncaught exception > Spark assembly has been built with Hive, including Datanucleus jars on > classpath > 14/06/21 04:18:53 ERROR TaskSetManager: Task 2.0:0 failed 3 times; aborting > job > Exception in thread "main" org.apache.spark.SparkException: Job aborted due > to stage failure: Task 2.0:0 failed 3 times, most recent failure: Exception > failure in TID 7 on host 192.168.42.202: java.lang.NoSuchFieldError: > INSTANCE > org.apache.http.entity.ContentType.parse(ContentType.java:229) > ... > > This is weird as this error is supposed to be caught by compiler but not jvm > (unless Spark has changed the content of a class internally, which is > impossible because the class is in the uber-jar but not closure). Also, I > can confirm that the class that contains INSTANCE as a property is in the > uber jar, so there is really no reason for Spark to throw it. > > Here is another independent question: I've also encounter several errors > that only appears in cluster mode, they are hard to fix because I cannot > debug them. Is there a local cluster simulation mode that can throw all > errors yet allows me to debug? > > Yours Peng > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Spark-throws-NoSuchFieldError-when-testing-on-cluster-mode-tp8064.html > Sent from the Apache Spark User List mailing list archive at Nabble.com.
