Hi,
It is possible that one of the jars on the sharelib also contain an
embedded (and not correctly shaded) jackson 2.9.0. I had a similar problem
once with hive-exec.jar. You can check the content of the jars with a
simple unzip.
Best,
Sala
On Mon, Apr 15, 2019 at 8:51 AM Lian Jiang wrote:
>
Thanks guys for reply.
I am using jackson 2.6.7 which is in spark 2.3.1 library.
I deleted all files under /user/oozie/share/lib/lib_{TS}/oozie/ (except
oozie-hadoop-utils-hadoop-2-4.3.1.3.0.0.0-1634.jar and
oozie-sharelib-oozie-4.3.1.3.0.0.0-1634.jar). Also, I make
/user/oozie/share/lib/lib_{TS}
Hi,
Probably multiple jars in the spark sharelib contain the jackson library
and the system loads an incorrect one. If you turn on java verbose class
loading, you can find which jar is loaded and you can clean up the sharelib.
Best,
Sala
On Sat, Apr 13, 2019 at 8:05 PM Lian Jiang wrote:
> I tr
I tried spark.yarn.user.classpath.first = true and
oozie.launcher.mapreduce.user.classpath.first
= true but no luck.
On Wed, Apr 10, 2019 at 11:47 PM Lian Jiang wrote:
> Hi,
>
> I am using hortonwork HDP3.0 which has oozie 4.3.1 and spark 2.3.1. My
> spark job throws intermittent dependency erro
