You can specify these jars (joda-time-2.7.jar, joda-convert-1.7.jar) either as part of your build and assembly or via the --jars option to spark-submit.
HTH. On Fri, Feb 27, 2015 at 2:48 PM, Su She <suhsheka...@gmail.com> wrote: > Hello Everyone, > > I'm having some issues launching (non-spark) applications via the > spark-submit commands. The common error I am getting is c/p below. I am > able to submit a spark streaming/kafka spark application, but can't start a > dynamoDB java app. The common error is related to joda-time. > > 1) I realized spark-submit was pointing to joda-time-1.6 in the > hadoop/lib,so I deleted this and my error changed from NoSuchMethodFound to > NoClassDefFoundError. > > Instead of pointing to the other version of joda-time in the hadoop/lib, > it now pointed to the jars I set a path to in my spark-submit command (I > tried joda-time versions 2.2, 2.3, 2.6, 2.7), but still got the errors > > 2) My rudimentary theory is that spark-submit uses < joda-time-2.0, but > the applications I'm running need >2.0. > > Thank you for the help! > > Exception in thread "main" java.lang.NoClassDefFoundError: > org/joda/time/format/DateTimeFormat > at com.amazonaws.auth.AWS4Signer.<clinit>(AWS4Signer.java:44) > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:526) > at java.lang.Class.newInstance(Class.java:374) > at > com.amazonaws.auth.SignerFactory.createSigner(SignerFactory.java:119) > at > com.amazonaws.auth.SignerFactory.lookupAndCreateSigner(SignerFactory.java:105) > at > com.amazonaws.auth.SignerFactory.getSigner(SignerFactory.java:78) > at > com.amazonaws.AmazonWebServiceClient.computeSignerByServiceRegion(AmazonWebServiceClient.java:307) > at > com.amazonaws.AmazonWebServiceClient.computeSignerByURI(AmazonWebServiceClient.java:280) > at > com.amazonaws.AmazonWebServiceClient.setEndpoint(AmazonWebServiceClient.java:160) > at > com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.setEndpoint(AmazonDynamoDBClient.java:2946) > at > com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.init(AmazonDynamoDBClient.java:351) > at > com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.<init>(AmazonDynamoDBClient.java:273) > at > com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.<init>(AmazonDynamoDBClient.java:250) > at AmazonDynamoDBSample.init(AmazonDynamoDBSample.java:81) > at AmazonDynamoDBSample.main(AmazonDynamoDBSample.java:87) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:358) > at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:75) > at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) > Caused by: java.lang.ClassNotFoundException: > org.joda.time.format.DateTimeFormat > at java.net.URLClassLoader$1.run(URLClassLoader.java:366) > at java.net.URLClassLoader$1.run(URLClassLoader.java:355) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:354) > at java.lang.ClassLoader.loadClass(ClassLoader.java:425) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) > at java.lang.ClassLoader.loadClass(ClassLoader.java:358) > >
