Can you see is there is anything unusual in the JobManager logs? Am 11.10.2015 18:56 schrieb "Matthias J. Sax" <mj...@apache.org>:
> Hi, > > I was just playing arround with Flink. After submitting my job, it runs > for multiple minutes, until I get the following Exception in one if the > TaskManager logs and the job fails. > > I have no clue what's going on... > > -Matthias > > > > 18:43:23,567 WARN akka.remote.RemoteWatcher > - Detected unreachable: [akka.tcp://flink@192.168.127.11:6123] > > 18:43:23,864 WARN akka.remote.ReliableDeliverySupervisor > - Association with remote system [akka.tcp:// > flink@192.168.127.11:6123] has failed, address is now gated for [5000] > ms. Reason is: [Disassociated]. > > 18:43:23,866 INFO org.apache.flink.runtime.taskmanager.TaskManager > - TaskManager akka://flink/user/taskmanager disconnects from > JobManager akka.tcp://flink@192.168.127.11:6123/user/jobmanager: > JobManager is no longer reachable > > 18:43:23,867 INFO org.apache.flink.runtime.taskmanager.TaskManager > - Cancelling all computations and discarding all cached data. > > 18:43:23,870 INFO org.apache.flink.runtime.taskmanager.Task > - Attempting to fail task externally Keyed Aggregation -> Sink: > Unnamed (1/28) > > 18:43:23,870 INFO org.apache.flink.runtime.taskmanager.Task > - Keyed Aggregation -> Sink: Unnamed (1/28) switched to FAILED > with exception. > > java.lang.Exception: TaskManager akka://flink/user/taskmanager > disconnects from JobManager akka.tcp:// > flink@192.168.127.11:6123/user/jobmanager: JobManager is no longer > reachable > > at > org.apache.flink.runtime.taskmanager.TaskManager.handleJobManagerDisconnect(TaskManager.scala:826) > > at > org.apache.flink.runtime.taskmanager.TaskManager$$anonfun$handleMessage$1.applyOrElse(TaskManager.scala:297) > > at > scala.runtime.AbstractPartialFunction$mcVL$sp.apply$mcVL$sp(AbstractPartialFunction.scala:33) > > at > scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:33) > > at > scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:25) > > at > org.apache.flink.runtime.LeaderSessionMessageFilter$$anonfun$receive$1.applyOrElse(LeaderSessionMessageFilter.scala:44) > > at > scala.runtime.AbstractPartialFunction$mcVL$sp.apply$mcVL$sp(AbstractPartialFunction.scala:33) > > at > scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:33) > > at > scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:25) > > at > org.apache.flink.runtime.LogMessages$$anon$1.apply(LogMessages.scala:33) > > at > org.apache.flink.runtime.LogMessages$$anon$1.apply(LogMessages.scala:28) > > at > scala.PartialFunction$class.applyOrElse(PartialFunction.scala:118) > > at > org.apache.flink.runtime.LogMessages$$anon$1.applyOrElse(LogMessages.scala:28) > > at akka.actor.Actor$class.aroundReceive(Actor.scala:465) > > at > org.apache.flink.runtime.taskmanager.TaskManager.aroundReceive(TaskManager.scala:119) > > at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516) > > at > akka.actor.dungeon.DeathWatch$class.receivedTerminated(DeathWatch.scala:46) > > at akka.actor.ActorCell.receivedTerminated(ActorCell.scala:369) > > at akka.actor.ActorCell.autoReceiveMessage(ActorCell.scala:501) > > at akka.actor.ActorCell.invoke(ActorCell.scala:486) > > at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:254) > > at akka.dispatch.Mailbox.run(Mailbox.scala:221) > > at akka.dispatch.Mailbox.exec(Mailbox.scala:231) > > at > scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) > > at > scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) > > at > scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) > > at > scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) > > 18:43:23,879 INFO org.apache.flink.runtime.taskmanager.Task > - Triggering cancellation of task code Keyed Aggregation -> Sink: > Unnamed (1/28) (4b1f979a60c30f84ee60553730a4e99a). > > 18:43:23,880 INFO org.apache.flink.runtime.taskmanager.Task > - Attempting to fail task externally Source: Custom Source -> Flat > Map (1/28) > > 18:43:23,880 INFO org.apache.flink.runtime.taskmanager.Task > - Source: Custom Source -> Flat Map (1/28) switched to FAILED with > exception. > > java.lang.Exception: TaskManager akka://flink/user/taskmanager > disconnects from JobManager akka.tcp:// > flink@192.168.127.11:6123/user/jobmanager: JobManager is no longer > reachable > > at > org.apache.flink.runtime.taskmanager.TaskManager.handleJobManagerDisconnect(TaskManager.scala:826) > > at > org.apache.flink.runtime.taskmanager.TaskManager$$anonfun$handleMessage$1.applyOrElse(TaskManager.scala:297) > > at > scala.runtime.AbstractPartialFunction$mcVL$sp.apply$mcVL$sp(AbstractPartialFunction.scala:33) > > at > scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:33) > > at > scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:25) > > at > org.apache.flink.runtime.LeaderSessionMessageFilter$$anonfun$receive$1.applyOrElse(LeaderSessionMessageFilter.scala:44) > > at > scala.runtime.AbstractPartialFunction$mcVL$sp.apply$mcVL$sp(AbstractPartialFunction.scala:33) > > at > scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:33) > > at > scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:25) > > at > org.apache.flink.runtime.LogMessages$$anon$1.apply(LogMessages.scala:33) > > at > org.apache.flink.runtime.LogMessages$$anon$1.apply(LogMessages.scala:28) > > at > scala.PartialFunction$class.applyOrElse(PartialFunction.scala:118) > > at > org.apache.flink.runtime.LogMessages$$anon$1.applyOrElse(LogMessages.scala:28) > > at akka.actor.Actor$class.aroundReceive(Actor.scala:465) > > at > org.apache.flink.runtime.taskmanager.TaskManager.aroundReceive(TaskManager.scala:119) > > at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516) > > at > akka.actor.dungeon.DeathWatch$class.receivedTerminated(DeathWatch.scala:46) > > at akka.actor.ActorCell.receivedTerminated(ActorCell.scala:369) > > at akka.actor.ActorCell.autoReceiveMessage(ActorCell.scala:501) > > at akka.actor.ActorCell.invoke(ActorCell.scala:486) > > at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:254) > > at akka.dispatch.Mailbox.run(Mailbox.scala:221) > > at akka.dispatch.Mailbox.exec(Mailbox.scala:231) > > at > scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) > > at > scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) > > at > scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) > > at > scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) > >
