Hi, Patrick --
Java 7 on the development machines:
» java -version
1 ↵
java version "1.7.0_51"
Java(TM) SE Runtime Environment (build 1.7.0_51-b13)
Java HotSpot(TM) 64-Bit Server VM (build 24.51-b03, mixed mode)
And on the deployed boxes:
$ java -version
java version "1.7.0_55"
OpenJDK Runtime Environment (IcedTea 2.4.7) (7u55-2.4.7-1ubuntu1)
OpenJDK 64-Bit Server VM (build 24.51-b03, mixed mode)
Also, "unzip -l" in place of "jar tvf" gives the same results, so I don't
think it's an issue with jar not reporting the files. Also, the classes do
get correctly packaged into the uberjar:
unzip -l /target/[deleted]-driver.jar | grep 'rdd/RDD' | grep 'saveAs'
1519 06-08-14 12:05
org/apache/spark/rdd/RDD$anonfun$saveAsTextFile$1.class
1560 06-08-14 12:05
org/apache/spark/rdd/RDD$anonfun$saveAsTextFile$2.class
Best.
-- Paul
—
p...@mult.ifario.us | Multifarious, Inc. | http://mult.ifario.us/
On Sun, Jun 8, 2014 at 1:02 PM, Patrick Wendell <pwend...@gmail.com> wrote:
> Paul,
>
> Could you give the version of Java that you are building with and the
> version of Java you are running with? Are they the same?
>
> Just off the cuff, I wonder if this is related to:
> https://issues.apache.org/jira/browse/SPARK-1520
>
> If it is, it could appear that certain functions are not in the jar
> because they go beyond the extended zip boundary `jar tvf` won't list
> them.
>
> - Patrick
>
> On Sun, Jun 8, 2014 at 12:45 PM, Paul Brown <p...@mult.ifario.us> wrote:
> > Moving over to the dev list, as this isn't a user-scope issue.
> >
> > I just ran into this issue with the missing saveAsTestFile, and here's a
> > little additional information:
> >
> > - Code ported from 0.9.1 up to 1.0.0; works with local[n] in both cases.
> > - Driver built as an uberjar via Maven.
> > - Deployed to smallish EC2 cluster in standalone mode (S3 storage) with
> > Spark 1.0.0-hadoop1 downloaded from Apache.
> >
> > Given that it functions correctly in local mode but not in a standalone
> > cluster, this suggests to me that the issue is in a difference between
> the
> > Maven version and the hadoop1 version.
> >
> > In the spirit of taking the computer at its word, we can just have a look
> > in the JAR files. Here's what's in the Maven dep as of 1.0.0:
> >
> > jar tvf
> >
> ~/.m2/repository/org/apache/spark/spark-core_2.10/1.0.0/spark-core_2.10-1.0.0.jar
> > | grep 'rdd/RDD' | grep 'saveAs'
> > 1519 Mon May 26 13:57:58 PDT 2014
> > org/apache/spark/rdd/RDD$anonfun$saveAsTextFile$1.class
> > 1560 Mon May 26 13:57:58 PDT 2014
> > org/apache/spark/rdd/RDD$anonfun$saveAsTextFile$2.class
> >
> >
> > And here's what's in the hadoop1 distribution:
> >
> > jar tvf spark-assembly-1.0.0-hadoop1.0.4.jar| grep 'rdd/RDD' | grep
> 'saveAs'
> >
> >
> > I.e., it's not there. It is in the hadoop2 distribution:
> >
> > jar tvf spark-assembly-1.0.0-hadoop2.2.0.jar| grep 'rdd/RDD' | grep
> 'saveAs'
> > 1519 Mon May 26 07:29:54 PDT 2014
> > org/apache/spark/rdd/RDD$anonfun$saveAsTextFile$1.class
> > 1560 Mon May 26 07:29:54 PDT 2014
> > org/apache/spark/rdd/RDD$anonfun$saveAsTextFile$2.class
> >
> >
> > So something's clearly broken with the way that the distribution
> assemblies
> > are created.
> >
> > FWIW and IMHO, the "right" way to publish the hadoop1 and hadoop2 flavors
> > of Spark to Maven Central would be as *entirely different* artifacts
> > (spark-core-h1, spark-core-h2).
> >
> > Logged as SPARK-2075 <https://issues.apache.org/jira/browse/SPARK-2075>.
> >
> > Cheers.
> > -- Paul
> >
> >
> >
> > --
> > p...@mult.ifario.us | Multifarious, Inc. | http://mult.ifario.us/
> >
> >
> > On Fri, Jun 6, 2014 at 2:45 AM, HenriV <henri.vanh...@vdab.be> wrote:
> >
> >> I'm experiencing the same error while upgrading from 0.9.1 to 1.0.0.
> >> Im using google compute engine and cloud storage. but saveAsTextFile is
> >> returning errors while saving in the cloud or saving local. When i
> start a
> >> job in the cluster i do get an error but after this error it keeps on
> >> running fine untill the saveAsTextFile. ( I don't know if the two are
> >> connected)
> >>
> >> -----------Error at job startup-------
> >> ERROR metrics.MetricsSystem: Sink class
> >> org.apache.spark.metrics.sink.MetricsServlet cannot be instantialized
> >> java.lang.reflect.InvocationTargetException
> >> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> >> Method)
> >> at
> >>
> >>
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> >> at
> >>
> >>
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> >> at
> java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> >> at
> >>
> >>
> org.apache.spark.metrics.MetricsSystem$$anonfun$registerSinks$1.apply(MetricsSystem.scala:136)
> >> at
> >>
> >>
> org.apache.spark.metrics.MetricsSystem$$anonfun$registerSinks$1.apply(MetricsSystem.scala:130)
> >> at
> >>
> scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:98)
> >> at
> >>
> scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:98)
> >> at
> >>
> scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:226)
> >> at
> scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:39)
> >> at scala.collection.mutable.HashMap.foreach(HashMap.scala:98)
> >> at
> >>
> >>
> org.apache.spark.metrics.MetricsSystem.registerSinks(MetricsSystem.scala:130)
> >> at
> >> org.apache.spark.metrics.MetricsSystem.<init>(MetricsSystem.scala:84)
> >> at
> >>
> >>
> org.apache.spark.metrics.MetricsSystem$.createMetricsSystem(MetricsSystem.scala:167)
> >> at org.apache.spark.SparkEnv$.create(SparkEnv.scala:230)
> >> at org.apache.spark.SparkContext.<init>(SparkContext.scala:202)
> >> at Hello$.main(Hello.scala:101)
> >> at Hello.main(Hello.scala)
> >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >> at
> >>
> >>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> >> at
> >>
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >> at java.lang.reflect.Method.invoke(Method.java:606)
> >> at sbt.Run.invokeMain(Run.scala:72)
> >> at sbt.Run.run0(Run.scala:65)
> >> at sbt.Run.sbt$Run$$execute$1(Run.scala:54)
> >> at sbt.Run$$anonfun$run$1.apply$mcV$sp(Run.scala:58)
> >> at sbt.Run$$anonfun$run$1.apply(Run.scala:58)
> >> at sbt.Run$$anonfun$run$1.apply(Run.scala:58)
> >> at sbt.Logger$$anon$4.apply(Logger.scala:90)
> >> at sbt.TrapExit$App.run(TrapExit.scala:244)
> >> at java.lang.Thread.run(Thread.java:744)
> >> Caused by: java.lang.NoSuchMethodError:
> >> com.fasterxml.jackson.core.JsonFactory.requiresPropertyOrdering()Z
> >> at
> >>
> com.fasterxml.jackson.databind.ObjectMapper.<init>(ObjectMapper.java:445)
> >> at
> >>
> com.fasterxml.jackson.databind.ObjectMapper.<init>(ObjectMapper.java:366)
> >> at
> >>
> >>
> org.apache.spark.metrics.sink.MetricsServlet.<init>(MetricsServlet.scala:45)
> >> ... 31 more
> >>
> >> then it runs fine till i get to saveAsTextFile
> >>
> >> 14/06/06 09:05:12 INFO scheduler.TaskSetManager: Loss was due to
> >> java.lang.ClassNotFoundException:
> >> org.apache.spark.rdd.RDD$$anonfun$saveAsTextFile$1 [duplicate 17]
> >> 14/06/06 09:05:12 INFO scheduler.DAGScheduler: Failed to run
> saveAsTextFile
> >> at Hello.scala:123
> >> 14/06/06 09:05:12 INFO scheduler.TaskSchedulerImpl: Cancelling stage 0
> >> [error] (run-main-0) org.apache.spark.SparkException: Job aborted due to
> >> stage failure: Task 0.0:3 failed 4 times, most recent failure: Exception
> >> failure in TID 142 on host sparky-s1.c.quick-heaven-560.internal:
> >> java.lang.ClassNotFoundException:
> >> org.apache.spark.rdd.RDD$$anonfun$saveAsTextFile$1
> >> [error] java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> >> [error] java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> >> [error] java.security.AccessController.doPrivileged(Native
> Method)
> >> [error]
> java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> >> [error] java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> >> [error] java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> >> [error] java.lang.Class.forName0(Native Method)
> >> [error] java.lang.Class.forName(Class.java:270)
> >> [error]
> >>
> >>
> org.apache.spark.serializer.JavaDeserializationStream$$anon$1.resolveClass(JavaSerializer.scala:60)
> >> [error]
> >> java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1612)
> >> [error]
> >> java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1517)
> >> [error]
> >>
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)
> >> [error]
> >> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
> >> [error]
> >> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
> >> [error]
> >> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
> >> [error]
> >>
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
> >> [error]
> >> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
> >> [error]
> >> java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
> >> [error]
> >>
> >>
> org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:63)
> >> [error]
> >>
> org.apache.spark.scheduler.ResultTask$.deserializeInfo(ResultTask.scala:61)
> >> [error]
> >> org.apache.spark.scheduler.ResultTask.readExternal(ResultTask.scala:141)
> >> [error]
> >> java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:1837)
> >> [error]
> >>
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796)
> >> [error]
> >> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
> >> [error]
> >> java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
> >> [error]
> >>
> >>
> org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:63)
> >> [error]
> >>
> >>
> org.apache.spark.serializer.JavaSerializerInstance.deserialize(JavaSerializer.scala:85)
> >> [error]
> >> org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:169)
> >> [error]
> >>
> >>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> >> [error]
> >>
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> >> [error] java.lang.Thread.run(Thread.java:744)
> >>
> >> Thanks for any help or guidance.
> >>
> >>
> >>
> >>
> >>
> >> --
> >> View this message in context:
> >>
> http://apache-spark-user-list.1001560.n3.nabble.com/Strange-problem-with-saveAsTextFile-after-upgrade-Spark-0-9-0-1-0-0-tp6832p7122.html
> >> Sent from the Apache Spark User List mailing list archive at Nabble.com.
> >>
>
