[
https://issues.apache.org/jira/browse/FLINK-26501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17502149#comment-17502149
]
Yun Gao commented on FLINK-26501:
---------------------------------
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=32569&view=logs&j=68a897ab-3047-5660-245a-cce8f83859f6&t=89ed5489-a970-5ff2-67f7-d7391de0165f&l=19179
> Quickstarts Scala nightly end-to-end test failed on azure due to checkponts
> failed and logs contains exceptions
> ---------------------------------------------------------------------------------------------------------------
>
> Key: FLINK-26501
> URL: https://issues.apache.org/jira/browse/FLINK-26501
> Project: Flink
> Issue Type: Bug
> Components: API / Scala, Runtime / Checkpointing
> Affects Versions: 1.14.3
> Reporter: Yun Gao
> Priority: Critical
> Labels: test-stability
>
> {code:java}
> 2022-03-05T02:35:36.4040037Z Mar 05 02:35:36 2022-03-05 02:35:34,334 INFO
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Triggering
> checkpoint 1 (type=CHECKPOINT) @ 1646447734295 for job
> b236087395260dc34648b84c2b86d6e8.
> 2022-03-05T02:35:36.4041701Z Mar 05 02:35:36 2022-03-05 02:35:34,387 INFO
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Decline
> checkpoint 1 by task e8a324cae6bf452d32db6797bbbafad0 of job
> b236087395260dc34648b84c2b86d6e8 at 127.0.0.1:45911-0a50f5 @ localhost
> (dataPort=44047).
> 2022-03-05T02:35:36.4043279Z Mar 05 02:35:36
> org.apache.flink.util.SerializedThrowable: Task name with subtask : Source:
> Sequence Source (Deprecated) -> Map -> Sink: Unnamed (1/1)#0 Failure reason:
> Checkpoint was declined (task is closing)
> 2022-03-05T02:35:36.4044531Z Mar 05 02:35:36 at
> org.apache.flink.runtime.taskmanager.Task.declineCheckpoint(Task.java:1389)
> ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
> 2022-03-05T02:35:36.4045729Z Mar 05 02:35:36 at
> org.apache.flink.runtime.taskmanager.Task.declineCheckpoint(Task.java:1382)
> ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
> 2022-03-05T02:35:36.4047172Z Mar 05 02:35:36 at
> org.apache.flink.runtime.taskmanager.Task.triggerCheckpointBarrier(Task.java:1348)
> ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
> 2022-03-05T02:35:36.4049092Z Mar 05 02:35:36 at
> org.apache.flink.runtime.taskexecutor.TaskExecutor.triggerCheckpoint(TaskExecutor.java:956)
> ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
> 2022-03-05T02:35:36.4050158Z Mar 05 02:35:36 at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_322]
> 2022-03-05T02:35:36.4050929Z Mar 05 02:35:36 at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:1.8.0_322]
> 2022-03-05T02:35:36.4051776Z Mar 05 02:35:36 at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_322]
> 2022-03-05T02:35:36.4052559Z Mar 05 02:35:36 at
> java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_322]
> 2022-03-05T02:35:36.4053373Z Mar 05 02:35:36 at
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.lambda$handleRpcInvocation$1(AkkaRpcActor.java:316)
> ~[?:?]
> 2022-03-05T02:35:36.4054849Z Mar 05 02:35:36 at
> org.apache.flink.runtime.concurrent.akka.ClassLoadingUtils.runWithContextClassLoader(ClassLoadingUtils.java:83)
> ~[?:?]
> 2022-03-05T02:35:36.4055685Z Mar 05 02:35:36 at
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcInvocation(AkkaRpcActor.java:314)
> ~[?:?]
> 2022-03-05T02:35:36.4056461Z Mar 05 02:35:36 at
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:217)
> ~[?:?]
> 2022-03-05T02:35:36.4057219Z Mar 05 02:35:36 at
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:163)
> ~[?:?]
> 2022-03-05T02:35:36.4057899Z Mar 05 02:35:36 at
> akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) ~[?:?]
> 2022-03-05T02:35:36.4059666Z Mar 05 02:35:36 at
> akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) ~[?:?]
> 2022-03-05T02:35:36.4061005Z Mar 05 02:35:36 at
> scala.PartialFunction.applyOrElse(PartialFunction.scala:123)
> ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
> 2022-03-05T02:35:36.4062324Z Mar 05 02:35:36 at
> scala.PartialFunction.applyOrElse$(PartialFunction.scala:122)
> ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
> 2022-03-05T02:35:36.4063941Z Mar 05 02:35:36 at
> akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) ~[?:?]
> 2022-03-05T02:35:36.4065009Z Mar 05 02:35:36 at
> scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)
> ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
> 2022-03-05T02:35:36.4066205Z Mar 05 02:35:36 at
> scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172)
> ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
> 2022-03-05T02:35:36.4067514Z Mar 05 02:35:36 at
> scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172)
> ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
> 2022-03-05T02:35:36.4068255Z Mar 05 02:35:36 at
> akka.actor.Actor.aroundReceive(Actor.scala:537) ~[?:?]
> 2022-03-05T02:35:36.4069019Z Mar 05 02:35:36 at
> akka.actor.Actor.aroundReceive$(Actor.scala:535) ~[?:?]
> 2022-03-05T02:35:36.4069638Z Mar 05 02:35:36 at
> akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:220) ~[?:?]
> 2022-03-05T02:35:36.4070271Z Mar 05 02:35:36 at
> akka.actor.ActorCell.receiveMessage(ActorCell.scala:580) ~[?:?]
> 2022-03-05T02:35:36.4070862Z Mar 05 02:35:36 at
> akka.actor.ActorCell.invoke(ActorCell.scala:548) ~[?:?]
> 2022-03-05T02:35:36.4071453Z Mar 05 02:35:36 at
> akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) ~[?:?]
> 2022-03-05T02:35:36.4072430Z Mar 05 02:35:36 at
> akka.dispatch.Mailbox.run(Mailbox.scala:231) ~[?:?]
> 2022-03-05T02:35:36.4073023Z Mar 05 02:35:36 at
> akka.dispatch.Mailbox.exec(Mailbox.scala:243) ~[?:?]
> 2022-03-05T02:35:36.4073687Z Mar 05 02:35:36 at
> java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) ~[?:1.8.0_322]
> 2022-03-05T02:35:36.4074596Z Mar 05 02:35:36 at
> java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
> ~[?:1.8.0_322]
> 2022-03-05T02:35:36.4075712Z Mar 05 02:35:36 at
> java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
> ~[?:1.8.0_322]
> 2022-03-05T02:35:36.4076437Z Mar 05 02:35:36 at
> java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175)
> ~[?:1.8.0_322]
> 2022-03-05T02:35:36.4077754Z Mar 05 02:35:36 2022-03-05 02:35:34,410 WARN
> org.apache.flink.runtime.checkpoint.CheckpointFailureManager [] - Failed to
> trigger checkpoint 1 for job b236087395260dc34648b84c2b86d6e8. (0 consecutive
> failed attempts so far)
> 2022-03-05T02:35:36.4078865Z Mar 05 02:35:36
> org.apache.flink.runtime.checkpoint.CheckpointException: Checkpoint was
> declined (task is closing)
> 2022-03-05T02:35:36.4080161Z Mar 05 02:35:36 at
> org.apache.flink.runtime.messages.checkpoint.SerializedCheckpointException.unwrap(SerializedCheckpointException.java:51)
> ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
> 2022-03-05T02:35:36.4081619Z Mar 05 02:35:36 at
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator.receiveDeclineMessage(CheckpointCoordinator.java:988)
> ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
> 2022-03-05T02:35:36.4083063Z Mar 05 02:35:36 at
> org.apache.flink.runtime.scheduler.ExecutionGraphHandler.lambda$declineCheckpoint$2(ExecutionGraphHandler.java:103)
> ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
> 2022-03-05T02:35:36.4085407Z Mar 05 02:35:36 at
> org.apache.flink.runtime.scheduler.ExecutionGraphHandler.lambda$processCheckpointCoordinatorMessage$3(ExecutionGraphHandler.java:119)
> ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
> 2022-03-05T02:35:36.4086635Z Mar 05 02:35:36 at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [?:1.8.0_322]
> 2022-03-05T02:35:36.4087419Z Mar 05 02:35:36 at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [?:1.8.0_322]
> 2022-03-05T02:35:36.4088438Z Mar 05 02:35:36 at
> java.lang.Thread.run(Thread.java:750) [?:1.8.0_322]
> 2022-03-05T02:35:36.4089614Z Mar 05 02:35:36 Caused by:
> org.apache.flink.util.SerializedThrowable: Task name with subtask : Source:
> Sequence Source (Deprecated) -> Map -> Sink: Unnamed (1/1)#0 Failure reason:
> Checkpoint was declined (task is closing)
> 2022-03-05T02:35:36.4090937Z Mar 05 02:35:36 at
> org.apache.flink.runtime.taskmanager.Task.declineCheckpoint(Task.java:1389)
> ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
> 2022-03-05T02:35:36.4092177Z Mar 05 02:35:36 at
> org.apache.flink.runtime.taskmanager.Task.declineCheckpoint(Task.java:1382)
> ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
> 2022-03-05T02:35:36.4093430Z Mar 05 02:35:36 at
> org.apache.flink.runtime.taskmanager.Task.triggerCheckpointBarrier(Task.java:1348)
> ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
> 2022-03-05T02:35:36.4094740Z Mar 05 02:35:36 at
> org.apache.flink.runtime.taskexecutor.TaskExecutor.triggerCheckpoint(TaskExecutor.java:956)
> ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
> 2022-03-05T02:35:36.4095836Z Mar 05 02:35:36 at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_322]
> 2022-03-05T02:35:36.4096579Z Mar 05 02:35:36 at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:1.8.0_322]
> 2022-03-05T02:35:36.4097766Z Mar 05 02:35:36 at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_322]
> 2022-03-05T02:35:36.4098684Z Mar 05 02:35:36 at
> java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_322]
> 2022-03-05T02:35:36.4101381Z Mar 05 02:35:36 at
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.lambda$handleRpcInvocation$1(AkkaRpcActor.java:316)
> ~[?:?]
> 2022-03-05T02:35:36.4102353Z Mar 05 02:35:36 at
> org.apache.flink.runtime.concurrent.akka.ClassLoadingUtils.runWithContextClassLoader(ClassLoadingUtils.java:83)
> ~[?:?]
> 2022-03-05T02:35:36.4103218Z Mar 05 02:35:36 at
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcInvocation(AkkaRpcActor.java:314)
> ~[?:?]
> 2022-03-05T02:35:36.4104019Z Mar 05 02:35:36 at
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:217)
> ~[?:?]
> 2022-03-05T02:35:36.4104801Z Mar 05 02:35:36 at
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:163)
> ~[?:?]
> 2022-03-05T02:35:36.4105719Z Mar 05 02:35:36 at
> akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) ~[?:?]
> 2022-03-05T02:35:36.4108356Z Mar 05 02:35:36 at
> akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) ~[?:?]
> 2022-03-05T02:35:36.4110333Z Mar 05 02:35:36 at
> scala.PartialFunction.applyOrElse(PartialFunction.scala:123)
> ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
> 2022-03-05T02:35:36.4112523Z Mar 05 02:35:36 at
> scala.PartialFunction.applyOrElse$(PartialFunction.scala:122)
> ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
> 2022-03-05T02:35:36.4113601Z Mar 05 02:35:36 at
> akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) ~[?:?]
> 2022-03-05T02:35:36.4114790Z Mar 05 02:35:36 at
> scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)
> ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
> 2022-03-05T02:35:36.4116110Z Mar 05 02:35:36 at
> scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172)
> ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
> 2022-03-05T02:35:36.4117636Z Mar 05 02:35:36 at
> scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172)
> ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
> 2022-03-05T02:35:36.4118641Z Mar 05 02:35:36 at
> akka.actor.Actor.aroundReceive(Actor.scala:537) ~[?:?]
> 2022-03-05T02:35:36.4119307Z Mar 05 02:35:36 at
> akka.actor.Actor.aroundReceive$(Actor.scala:535) ~[?:?]
> 2022-03-05T02:35:36.4120161Z Mar 05 02:35:36 at
> akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:220) ~[?:?]
> 2022-03-05T02:35:36.4120842Z Mar 05 02:35:36 at
> akka.actor.ActorCell.receiveMessage(ActorCell.scala:580) ~[?:?]
> 2022-03-05T02:35:36.4121482Z Mar 05 02:35:36 at
> akka.actor.ActorCell.invoke(ActorCell.scala:548) ~[?:?]
> 2022-03-05T02:35:36.4122113Z Mar 05 02:35:36 at
> akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) ~[?:?]
> 2022-03-05T02:35:36.4122736Z Mar 05 02:35:36 at
> akka.dispatch.Mailbox.run(Mailbox.scala:231) ~[?:?]
> 2022-03-05T02:35:36.4123332Z Mar 05 02:35:36 at
> akka.dispatch.Mailbox.exec(Mailbox.scala:243) ~[?:?]
> 2022-03-05T02:35:36.4123984Z Mar 05 02:35:36 at
> java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) ~[?:1.8.0_322]
> 2022-03-05T02:35:36.4124749Z Mar 05 02:35:36 at
> java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
> ~[?:1.8.0_322]
> 2022-03-05T02:35:36.4125750Z Mar 05 02:35:36 at
> java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
> ~[?:1.8.0_322]
> 2022-03-05T02:35:36.4126591Z Mar 05 02:35:36 at
> java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175)
> ~[?:1.8.0_322]
> 2022-03-05T02:35:36.4128133Z Mar 05 02:35:36 2022-03-05 02:35:34,430 INFO
> org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source:
> Sequence Source (Deprecated) -> Map -> Sink: Unnamed (1/1)
> (e8a324cae6bf452d32db6797bbbafad0) switched from RUNNING to FINISHED. {code}
> [https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=32553&view=logs&j=91bf6583-3fb2-592f-e4d4-d79d79c3230a&t=cc5499f8-bdde-5157-0d76-b6528ecd808e&l=18735]
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
