Hi, Mans -- Both of those versions of Jackson are pretty ancient. Do you know which of the Spark dependencies is pulling them in? It would be good for us (the Jackson, Woodstox, etc., folks) to see if we can get people to upgrade to more recent versions of Jackson.
-- Paul — [email protected] | Multifarious, Inc. | http://mult.ifario.us/ On Fri, Jun 27, 2014 at 12:58 PM, M Singh <[email protected]> wrote: > Hi: > > I am using spark to stream data to cassandra and it works fine in local > mode. But when I execute the application in a standalone clustered env I > got exception included below (java.lang.NoClassDefFoundError: > org/codehaus/jackson/annotate/JsonClass). > > I think this is due to the jackson-core-asl dependency conflict > (jackson-core-asl 1.8.8 has the JsonClass but 1.9.x does not). The 1.9.x > version is being pulled in by spark-sql project. I tried adding > jackson-core-asl 1.8.8 with --jars argument while submitting the > application for execution but it did not work. So I created a custom spark > build excluding sql project. With this custom spark install I was able to > resolve the issue at least on a single node cluster (separate master and > worker). > > If there is an alternate way to resolve this conflicting jar issue without > a custom build (eg: configuration to use the user defined jars in the > executor class path first), please let me know. > > Also, is there a comprehensive list of configuration properties available > for spark ? > > Thanks > > Mans > > Exception trace > > TaskSetManager: Loss was due to java.lang.NoClassDefFoundError > java.lang.NoClassDefFoundError: org/codehaus/jackson/annotate/JsonClass > at > org.codehaus.jackson.map.introspect.JacksonAnnotationIntrospector.findDeserializationType(JacksonAnnotationIntrospector.java:524) > at > org.codehaus.jackson.map.deser.BasicDeserializerFactory.modifyTypeByAnnotation(BasicDeserializerFactory.java:732) > at > org.codehaus.jackson.map.deser.BasicDeserializerFactory.createCollectionDeserializer(BasicDeserializerFactory.java:229) > at > org.codehaus.jackson.map.deser.StdDeserializerProvider._createDeserializer(StdDeserializerProvider.java:386) > at > org.codehaus.jackson.map.deser.StdDeserializerProvider._createAndCache2(StdDeserializerProvider.java:307) > at > org.codehaus.jackson.map.deser.StdDeserializerProvider._createAndCacheValueDeserializer(StdDeserializerProvider.java:287) > at > org.codehaus.jackson.map.deser.StdDeserializerProvider.findValueDeserializer(StdDeserializerProvider.java:136) > at > org.codehaus.jackson.map.deser.StdDeserializerProvider.findTypedValueDeserializer(StdDeserializerProvider.java:157) > at > org.codehaus.jackson.map.ObjectMapper._findRootDeserializer(ObjectMapper.java:2468) > at > org.codehaus.jackson.map.ObjectMapper._readMapAndClose(ObjectMapper.java:2402) > at org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1602) > >
