spark-1.1.0-bin-hadoop2.4 java.lang.NoClassDefFoundError: org/codehaus/jackson/annotate/JsonClass

[Andy Davidson]

Hi I am new to spark.

I am trying to write a simple java program that process tweets that where
collected and stored in a file. I figured the simplest thing to do would be
to convert the JSON string into a java map. When I submit my jar file I keep
getting the following error

java.lang.NoClassDefFoundError: org/codehaus/jackson/annotate/JsonClass


For the life of me I can not figure out what the problem is. I am using
maven shade plugin and check to see that the missing class is my uber jar

Any suggestions would be greatly appreciated.

Andy

P.s. I should mention that I am running everything on my local machine.

<properties>

<jackson.version>1.8.8</jackson.version> <!-- 1.9.13 -->

<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>

</properties>



<dependencies>

<dependency>

<groupId>org.codehaus.jackson</groupId>

<artifactId>jackson-core-asl</artifactId>

<version>${jackson.version}</version>

</dependency>

<dependency>

<groupId>org.codehaus.jackson</groupId>

<artifactId>jackson-mapper-asl</artifactId>

<version>${jackson.version}</version>

</dependency>



I am using shade to build an uber jar



$jar ­tvf target/myUber.jar

Š

   580 Wed Sep 17 16:17:36 PDT 2014
org/codehaus/jackson/annotate/JsonClass.class

Š




14/09/17 16:13:24 ERROR Executor: Exception in task 0.0 in stage 0.0 (TID 0)

java.lang.NoClassDefFoundError: org/codehaus/jackson/annotate/JsonClass

at 
org.codehaus.jackson.map.introspect.JacksonAnnotationIntrospector.findDeseri
alizationType(JacksonAnnotationIntrospector.java:524)

at 
org.codehaus.jackson.map.deser.BasicDeserializerFactory.modifyTypeByAnnotati
on(BasicDeserializerFactory.java:732)

at 
org.codehaus.jackson.map.deser.BasicDeserializerFactory.createMapDeserialize
r(BasicDeserializerFactory.java:337)

at 
org.codehaus.jackson.map.deser.StdDeserializerProvider._createDeserializer(S
tdDeserializerProvider.java:377)

at 
org.codehaus.jackson.map.deser.StdDeserializerProvider._createAndCache2(StdD
eserializerProvider.java:307)

at 
org.codehaus.jackson.map.deser.StdDeserializerProvider._createAndCacheValueD
eserializer(StdDeserializerProvider.java:287)

at 
org.codehaus.jackson.map.deser.StdDeserializerProvider.findValueDeserializer
(StdDeserializerProvider.java:136)

at 
org.codehaus.jackson.map.deser.StdDeserializerProvider.findTypedValueDeseria
lizer(StdDeserializerProvider.java:157)

at 
org.codehaus.jackson.map.ObjectMapper._findRootDeserializer(ObjectMapper.jav
a:2468)

at 
org.codehaus.jackson.map.ObjectMapper._readMapAndClose(ObjectMapper.java:240
2)

at org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1609)

at com.santacruzintegration.spark.JSONHelper.toJsonMap(JSONHelper.java:40)





public class JSONHelper {

    public static final ObjectMapper mapper = new ObjectMapper();

    static {

        mapper.configure(Feature.FAIL_ON_UNKNOWN_PROPERTIES, false);

        //mapper.configure(Feature.USE_ANNOTATIONS, false);

    }

Š

   public static Map<String, String> toJsonMap(String json) {

        System.err.println("AEDWIP toJsonMap: " + json);

        try {

            TypeReference<HashMap<String, String>> typeRef = new
TypeReference<HashMap<String, String>>() {

            };

            //return mapper.readValue(json, new
TypeReference<HashMap<String,String>>(){});

            return mapper.readValue(json, typeRef);

            //return mapper.convertValue(json, typeRef);

            

//            HashMap<String,String> ret =

//                    new ObjectMapper().readValue(json, HashMap.class);

        } catch (Exception e) {

            throw new IOError(e);

        }

    }