Thanks Sean/Martin, my bad, Spark version was 3.0.1 so after using json 3.6.6 it fixed the issue.
Thanks Amit On Fri, Feb 4, 2022 at 3:37 PM Sean Owen <sro...@gmail.com> wrote: > My guess is that something else you depend on is actually bringing in a > different json4s, or you're otherwise mixing library/Spark versions. Use > mvn dependency:tree or equivalent on your build to see what you actually > build in. You probably do not need to include json4s at all as it is in > Spark anway > > On Fri, Feb 4, 2022 at 2:35 PM Amit Sharma <resolve...@gmail.com> wrote: > >> Martin Sean, changed it to 3.7.0-MS still getting the below error. >> I am still getting the same issue >> Exception in thread "streaming-job-executor-0" >> java.lang.NoSuchMethodError: >> org.json4s.ShortTypeHints$.apply$default$2()Ljava/lang/String; >> >> >> Thanks >> Amit >> >> On Fri, Feb 4, 2022 at 9:03 AM Martin Grigorov <mgrigo...@apache.org> >> wrote: >> >>> Hi, >>> >>> Amit said that he uses Spark 3.1, so the link should be >>> https://github.com/apache/spark/blob/branch-3.1/pom.xml#L879 (3.7.0-M5) >>> >>> @Amit: check your classpath. Maybe there are more jars of this >>> dependency. >>> >>> On Thu, Feb 3, 2022 at 10:53 PM Sean Owen <sro...@gmail.com> wrote: >>> >>>> You can look it up: >>>> https://github.com/apache/spark/blob/branch-3.2/pom.xml#L916 >>>> 3.7.0-M11 >>>> >>>> On Thu, Feb 3, 2022 at 1:57 PM Amit Sharma <resolve...@gmail.com> >>>> wrote: >>>> >>>>> Hello, everyone. I am migrating my spark stream to spark version 3.1. >>>>> I also upgraded json version as below >>>>> >>>>> libraryDependencies += "org.json4s" %% "json4s-native" % "3.7.0-M5" >>>>> >>>>> >>>>> While running the job I getting an error for the below code where I am >>>>> serializing the given inputs. >>>>> >>>>> implicit val formats = >>>>> Serialization.formats(ShortTypeHints(List(classOf[ForecastResponse], >>>>> classOf[OverlayRequest], >>>>> classOf[FTEResponseFromSpark], classOf[QuotaResponse], >>>>> classOf[CloneResponse] >>>>> >>>>> ))) >>>>> >>>>> >>>>> Exception in thread "streaming-job-executor-4" >>>>> java.lang.NoSuchMethodError: >>>>> org.json4s.ShortTypeHints$.apply$default$2()Ljava/lang/String; >>>>> >>>>> It seems to me jar issue, not sure which version of json4s-native should >>>>> I use with spark 3.1. >>>>> >>>>>
