I tried spark.yarn.user.classpath.first = true and oozie.launcher.mapreduce.user.classpath.first = true but no luck.
On Wed, Apr 10, 2019 at 11:47 PM Lian Jiang <jiangok2...@gmail.com> wrote: > Hi, > > I am using hortonwork HDP3.0 which has oozie 4.3.1 and spark 2.3.1. My > spark job throws intermittent dependency error: > > 2019-04-11 06:29:07,178 [Driver] ERROR > org.apache.spark.deploy.yarn.ApplicationMaster - User class threw > exception: java.lang.NoClassDefFoundError: Could not initialize class > com.fasterxml.jackson.databind.SerializationConfig > > java.lang.NoClassDefFoundError: Could not initialize class > com.fasterxml.jackson.databind.SerializationConfig > > at > com.fasterxml.jackson.databind.ObjectMapper.<init>(ObjectMapper.java:565) > > at > com.fasterxml.jackson.databind.ObjectMapper.<init>(ObjectMapper.java:480) > > > > Caused by: com.fasterxml.jackson.databind.JsonMappingException: > Incompatible Jackson version: 2.9.0 > > at > com.fasterxml.jackson.module.scala.JacksonModule$class.setupModule(JacksonModule.scala:64) > > at > com.fasterxml.jackson.module.scala.DefaultScalaModule.setupModule(DefaultScalaModule.scala:19) > > at > com.fasterxml.jackson.databind.ObjectMapper.registerModule(ObjectMapper.java:751) > > at org.apache.spark.util.JsonProtocol$.<init>(JsonProtocol.scala:59) > > at org.apache.spark.util.JsonProtocol$.<clinit>(JsonProtocol.scala) > > > > The same spark job running via spark-submit always pass. Why spark action > has such issue? Thanks for any clue. >
