Hi, Please take a look on this thread first: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Akka-Http-used-in-custom-RichSourceFunction-td20314.html <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Akka-Http-used-in-custom-RichSourceFunction-td20314.html>
Piotrek > On 11 Jun 2018, at 11:16, Wouter Zorgdrager <[email protected]> wrote: > > Hi, > > I think I'm running into an Akka version conflict when running a Flink job on > a cluster. > > The current situation: > - Flink cluster on Flink 1.4.2 (using Docker) > - Flink job which uses twitter4s [1] library and Akka version 2.5.8 > > In my Flink job I try to 'shutdown' an Akka actor from the twitter4s library. > This results in a whole taskmanager crashing with the following stacktrace: > > taskrunner_1 | 2018-06-11 09:03:14,454 INFO > org.apache.flink.runtime.taskmanager.TaskManager - > Un-registering task and sending final execution state CANCELED to JobManager > for task Source: Custom Source -> Sink: Unnamed > (0ba7f7f259eee06fe2f7d783c868179b) > taskrunner_1 | Uncaught error from thread > [twitter4s-streaming-akka.actor.default-dispatcher-288]: loader constraint > violation: when resolving method > "akka.actor.ActorCell$$anonfun$3.<init>(Lakka/actor/ActorCell;)V" the class > loader (instance of > org/apache/flink/runtime/execution/librarycache/FlinkUserCodeClassLoaders$ChildFirstClassLoader) > of the current class, akka/actor/ActorCell, and the class loader (instance > of sun/misc/Launcher$AppClassLoader) for the method's defining class, > akka/actor/ActorCell$$anonfun$3, have different Class objects for the type > akka/actor/ActorCell used in the signature, shutting down JVM since > 'akka.jvm-exit-on-fatal-error' is enabled for for > ActorSystem[twitter4s-streaming] > taskrunner_1 | java.lang.LinkageError: loader constraint violation: when > resolving method > "akka.actor.ActorCell$$anonfun$3.<init>(Lakka/actor/ActorCell;)V" the class > loader (instance of > org/apache/flink/runtime/execution/librarycache/FlinkUserCodeClassLoaders$ChildFirstClassLoader) > of the current class, akka/actor/ActorCell, and the class loader (instance > of sun/misc/Launcher$AppClassLoader) for the method's defining class, > akka/actor/ActorCell$$anonfun$3, have different Class objects for the type > akka/actor/ActorCell used in the signature > taskrunner_1 | at akka.actor.ActorCell.invoke(ActorCell.scala:499) > taskrunner_1 | at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257) > taskrunner_1 | at akka.dispatch.Mailbox.run(Mailbox.scala:224) > taskrunner_1 | at akka.dispatch.Mailbox.exec(Mailbox.scala:234) > taskrunner_1 | at > akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) > taskrunner_1 | at > akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) > taskrunner_1 | at > akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) > taskrunner_1 | at > akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) > taskrunner_1 | 2018-06-11 09:03:14,984 INFO > org.apache.flink.runtime.blob.PermanentBlobCache - Shutting down > BLOB cache > taskrunner_1 | 2018-06-11 09:03:14,985 INFO > org.apache.flink.runtime.blob.TransientBlobCache - Shutting down > BLOB cache > taskrunner_1 | Exception in thread "twitter4s-streaming-shutdown-hook-1" > java.lang.NoClassDefFoundError: > akka/actor/CoordinatedShutdown$$anonfun$totalTimeout$1 > taskrunner_1 | at > akka.actor.CoordinatedShutdown.totalTimeout(CoordinatedShutdown.scala:515) > taskrunner_1 | at > akka.actor.CoordinatedShutdown$$anonfun$initJvmHook$1.apply(CoordinatedShutdown.scala:217) > taskrunner_1 | at > akka.actor.CoordinatedShutdown$$anon$2.run(CoordinatedShutdown.scala:547) > taskrunner_1 | Caused by: java.lang.ClassNotFoundException: > akka.actor.CoordinatedShutdown$$anonfun$totalTimeout$1 > taskrunner_1 | at > java.net.URLClassLoader.findClass(URLClassLoader.java:381) > taskrunner_1 | at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > taskrunner_1 | at > org.apache.flink.runtime.execution.librarycache.FlinkUserCodeClassLoaders$ChildFirstClassLoader.loadClass(FlinkUserCodeClassLoaders.java:128) > taskrunner_1 | at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > taskrunner_1 | ... 3 more > > To me, it looks like an version conflict. Any suggestions how to solve this? > > Thanks! > Wouter > > [1] - Twitter4s: > https://github.com/DanielaSfregola/twitter4s/blob/master/build.sbt > <https://github.com/DanielaSfregola/twitter4s/blob/master/build.sbt> >
