Then maybe changing the following in pom.xml to 2.7.0 and rebuild Spark ?
<fasterxml.jackson.version>2.5.3</fasterxml.jackson.version>
On Tue, Jan 26, 2016 at 11:53 AM, asdf zxcv <benjamin.ha...@gmail.com>
wrote:
> Hmm, this did not seem to resolve the issue. I also tried adding a
> relocation for jackson as well.
>
> On Tue, Jan 26, 2016 at 10:09 AM, Ted Yu <yuzhih...@gmail.com> wrote:
>
>> I wonder if the following change would solve the problem you described
>> (by shading jackson.core):
>>
>> diff --git a/pom.xml b/pom.xml
>> index fb77506..32a3237 100644
>> --- a/pom.xml
>> +++ b/pom.xml
>> @@ -2177,6 +2177,7 @@
>> <include>org.eclipse.jetty:jetty-util</include>
>> <include>org.eclipse.jetty:jetty-server</include>
>> <include>com.google.guava:guava</include>
>> + <include>com.fasterxml.jackson.core</include>
>> </includes>
>> </artifactSet>
>> <relocations>
>>
>> Cheers
>>
>> On Tue, Jan 26, 2016 at 9:34 AM, asdf zxcv <benjamin.ha...@gmail.com>
>> wrote:
>>
>>> Hi all,
>>>
>>> I'm trying to run MaxMind GeoIP2 in a Spark task, but get a runtime
>>> error at init due to a NoSuchMethodError for ArrayNode from
>>> com.fasterxml.jackson.core:jackson-databind.
>>> This succeeds locally in unit tests, but fails in Spark tasks.
>>>
>>> I've excluded jackson-databind from all other dependencies, including
>>> Spark, since MaxMind is using the latest version of jackson-databind
>>> (2.7.0).
>>>
>>> `mvn dependency:tree` shows no conflicts for jackson-databind. There are
>>> numerous conflicts with other jackson utilities from org.codehaus, but this
>>> shouldn't be a problem since it is in a different namespace, right?
>>>
>>> I think this may be a problem with the shaded jar produced by
>>> maven-shade-plugin 2.4. Here is the config from pom.xml:
>>> pastebin.com/QzrhM5Ee
>>>
>>> `jar tvf shaded.jar` yields an entry for the missing class,
>>> com/fasterxml/jackson/databind/node/ArrayNode.class.
>>> META-INF/maven/com.fasterxml.jackson.core/jackson-databind/pom.xml has
>>> the correct version, 2.7.0. The shaded jar expanded jackson-databind into
>>> its constituent *.class files instead of providing the jar itself; is this
>>> normal?
>>>
>>> My other thought is there is some issue with the classpath. Is it
>>> sufficient that the shaded jar contains the jackson-databind *.class files?
>>> How else can I investigate this? I know that GeoIP databse init succeeds
>>> locally with the jackson-databind jar in ~/.m2/
>>>
>>> I've seen numerous issues with jackson-databind due to Spark using an
>>> old version and not shading this particular dependency, but this seems to
>>> have been resolved. I'm currently using version 1.3.1 and get the same
>>> result bumping the version to 1.6.0
>>>
>>> More detail here:
>>> https://stackoverflow.com/questions/34958520/nosuchmethodexception-in-maxmind-geoip-dependency-jackson-databind-built-with-mv
>>>
>>>
>>> Thanks,
>>>
>>> Ben
>>>
>>
>>
>
