Hi,

I think I'm running into an Akka version conflict when running a Flink job
on a cluster.

The current situation:
- Flink cluster on Flink 1.4.2 (using Docker)
- Flink job which uses twitter4s [1] library and Akka version 2.5.8

In my Flink job I try to 'shutdown' an Akka actor from the twitter4s
library.
This results in a whole taskmanager crashing with the following stacktrace:

taskrunner_1  | 2018-06-11 09:03:14,454 INFO
org.apache.flink.runtime.taskmanager.TaskManager              -
Un-registering task and sending final execution state CANCELED to
JobManager for task Source: Custom Source -> Sink: Unnamed
(0ba7f7f259eee06fe2f7d783c868179b)
taskrunner_1  | Uncaught error from thread
[twitter4s-streaming-akka.actor.default-dispatcher-288]: loader constraint
violation: when resolving method
"akka.actor.ActorCell$$anonfun$3.<init>(Lakka/actor/ActorCell;)V" the class
loader (instance of
org/apache/flink/runtime/execution/librarycache/FlinkUserCodeClassLoaders$ChildFirstClassLoader)
of the current class, akka/actor/ActorCell, and the class loader (instance
of sun/misc/Launcher$AppClassLoader) for the method's defining class,
akka/actor/ActorCell$$anonfun$3, have different Class objects for the type
akka/actor/ActorCell used in the signature, shutting down JVM since
'akka.jvm-exit-on-fatal-error' is enabled for for
ActorSystem[twitter4s-streaming]
taskrunner_1  | java.lang.LinkageError: loader constraint violation: when
resolving method
"akka.actor.ActorCell$$anonfun$3.<init>(Lakka/actor/ActorCell;)V" the class
loader (instance of
org/apache/flink/runtime/execution/librarycache/FlinkUserCodeClassLoaders$ChildFirstClassLoader)
of the current class, akka/actor/ActorCell, and the class loader (instance
of sun/misc/Launcher$AppClassLoader) for the method's defining class,
akka/actor/ActorCell$$anonfun$3, have different Class objects for the type
akka/actor/ActorCell used in the signature
taskrunner_1  |     at akka.actor.ActorCell.invoke(ActorCell.scala:499)
taskrunner_1  |     at
akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
taskrunner_1  |     at akka.dispatch.Mailbox.run(Mailbox.scala:224)
taskrunner_1  |     at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
taskrunner_1  |     at
akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
taskrunner_1  |     at
akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
taskrunner_1  |     at
akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
taskrunner_1  |     at
akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
taskrunner_1  | 2018-06-11 09:03:14,984 INFO
org.apache.flink.runtime.blob.PermanentBlobCache              - Shutting
down BLOB cache
taskrunner_1  | 2018-06-11 09:03:14,985 INFO
org.apache.flink.runtime.blob.TransientBlobCache              - Shutting
down BLOB cache
taskrunner_1  | Exception in thread "twitter4s-streaming-shutdown-hook-1"
java.lang.NoClassDefFoundError:
akka/actor/CoordinatedShutdown$$anonfun$totalTimeout$1
taskrunner_1  |     at
akka.actor.CoordinatedShutdown.totalTimeout(CoordinatedShutdown.scala:515)
taskrunner_1  |     at
akka.actor.CoordinatedShutdown$$anonfun$initJvmHook$1.apply(CoordinatedShutdown.scala:217)
taskrunner_1  |     at
akka.actor.CoordinatedShutdown$$anon$2.run(CoordinatedShutdown.scala:547)
taskrunner_1  | Caused by: java.lang.ClassNotFoundException:
akka.actor.CoordinatedShutdown$$anonfun$totalTimeout$1
taskrunner_1  |     at
java.net.URLClassLoader.findClass(URLClassLoader.java:381)
taskrunner_1  |     at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
taskrunner_1  |     at
org.apache.flink.runtime.execution.librarycache.FlinkUserCodeClassLoaders$ChildFirstClassLoader.loadClass(FlinkUserCodeClassLoaders.java:128)
taskrunner_1  |     at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
taskrunner_1  |     ... 3 more

To me, it looks like an version conflict. Any suggestions how to solve this?

Thanks!
Wouter

[1] - Twitter4s:
https://github.com/DanielaSfregola/twitter4s/blob/master/build.sbt

Reply via email to