Scala 2.12.8 broke binary compatibility with 2.12.7 which Flink
currently is compiled against.
As a result you must either stay at 2.12.7, or recompile Flink yourself
against 2.12.12 as shown here
<https://ci.apache.org/projects/flink/flink-docs-release-1.12/flinkDev/building.html#scala-versions>.
On 1/28/2021 2:34 AM, Sourabh Mokhasi wrote:
Hi,
We have several flink applications written with Flink 1.9.1 and
Scala 2.11.12 and we are in the process of upgrading to Flink 1.11.2
and Scala 2.12.12. We are using maven to manage our application
dependencies.
After updating the pom.xml file to use the upgraded versions of Scala
and Flink as mentioned above, all the unit tests written with
Scalatest 3.0.5(We are using flatspec style) fail with the following
exception.
org.apache.flink.shaded.guava18.com.google.common.util.concurrent.ExecutionError:
java.lang.NoClassDefFoundError: scala/math/Ordering$$anon$9
at
org.apache.flink.shaded.guava18.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2201)
at
org.apache.flink.shaded.guava18.com.google.common.cache.LocalCache.get(LocalCache.java:3937)
at
org.apache.flink.shaded.guava18.com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4739)
at
org.apache.flink.api.scala.typeutils.TraversableSerializer$.compileCbf(TraversableSerializer.scala:184)
at
org.apache.flink.api.scala.typeutils.TraversableSerializer.compileCbf(TraversableSerializer.scala:51)
at
org.apache.flink.api.scala.typeutils.TraversableSerializer.<init>(TraversableSerializer.scala:41)
Cause: java.lang.NoClassDefFoundError: scala/math/Ordering$$anon$9
at
scala.tools.nsc.transform.LambdaLift$LambdaLifter.<init>(LambdaLift.scala:67)
at
scala.tools.nsc.transform.LambdaLift.newTransformer(LambdaLift.scala:49)
at scala.tools.nsc.transform.Transform$Phase.apply(Transform.scala:30)
at scala.tools.nsc.Global$GlobalPhase.applyPhase(Global.scala:441)
at scala.tools.nsc.Global$GlobalPhase.run(Global.scala:392)
at scala.tools.nsc.Global$Run.compileUnitsInternal(Global.scala:1467)
at scala.tools.nsc.Global$Run.compileUnits(Global.scala:1451)
at
scala.tools.reflect.ToolBoxFactory$ToolBoxImpl$ToolBoxGlobal.wrapInPackageAndCompile(ToolBoxFactory.scala:201)
at
scala.tools.reflect.ToolBoxFactory$ToolBoxImpl$ToolBoxGlobal.compile(ToolBoxFactory.scala:256)
at
scala.tools.reflect.ToolBoxFactory$ToolBoxImpl.$anonfun$compile$13(ToolBoxFactory.scala:433)
...
Cause: java.lang.ClassNotFoundException: scala.math.Ordering$$anon$9
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
at
scala.tools.nsc.transform.LambdaLift$LambdaLifter.<init>(LambdaLift.scala:67)
[flink-akka.actor.default-dispatcher-3] INFO
org.apache.flink.runtime.taskexecutor.DefaultJobLeaderService - Stop
job leader service.
at
scala.tools.nsc.transform.LambdaLift.newTransformer(LambdaLift.scala:49)
[flink-akka.actor.default-dispatcher-3] INFO
org.apache.flink.runtime.state.TaskExecutorLocalStateStoresManager -
Shutting down TaskExecutorLocalStateStoresManager.
at scala.tools.nsc.transform.Transform$Phase.apply(Transform.scala:30)
at scala.tools.nsc.Global$GlobalPhase.applyPhase(Global.scala:441)
at scala.tools.nsc.Global$GlobalPhase.run(Global.scala:392)
at scala.tools.nsc.Global$Run.compileUnitsInternal(Global.scala:1467)
The app itself compiles without issues and this can be verified by
running mvn clean package `mvn clean package -DskipTests`
In order to troubleshoot/narrow down the issue, I upgraded the flink
package from 1.9.1 to to 1.11.2 while keeping the scala version the
same i.e 2.11.12 instead of 2.12.12 and this seems to have resolved
the issue.
The app compiles and the test cases pass as well.
Is this a known compatibility issue between Flink 1.11.2 and Scala
2.12.12?
Thanks,
Sourabh
