Hello thanks for your response,
I just want to know why you suggest that i should use the native Kubernetes HA.
Im my case I need to use the standalone application mode.
On 2021/10/25 09:25:21, Xintong Song <tonysong...@gmail.com> wrote:
> You should be using the native Kubernetes HA. This error message suggests
> Flink's dispatcher cannot successfully renew it's leadership within 120s,
> so according to the leader election protocol it loses the leadership and
> thus shutted down.
>
> This is usually caused by:
> 1) Long time GC pause in Flink's master process. You can check the
> jobmanager GC logs to verify that. And if it is indeed the problem, you can
> try to increase jobmanager's heap memory size.
> 2) Kubernetes API server is temporarily not accessible. You would need to
> check the Kubernetes logs, and if it is indeed the problem, you can either
> try to fix the API server accessibility or increase the leadership renew
> timeout (high-availability.kubernetes.leader-election.renew-deadline).
>
> Thank you~
>
> Xintong Song
>
>
>
> On Mon, Oct 25, 2021 at 5:08 PM marco <mejrihousse...@gmail.com> wrote:
>
> > Any suggestions would be appreciated.
> >
> > On 2021/10/20 16:18:39, marco <mejrihousse...@gmail.com> wrote:
> > >
> > >
> > > Hello flink community::
> > >
> > > I am deploying flink application cluster standalone mode on kubernetes,
> > but i am facing some problems
> > >
> > > the job starts normally and it continues to run but at some point in
> > time it crushes and gets restarted.
> > >
> > > Does anyone facing the same problem or know how to resolve it.
> > >
> > > I am using flink 1.12.5 version.
> > >
> > > here's the log of the JM:
> > >
> > >
> > {"@timestamp":"2021-10-19T04:47:29.887+02:00","@version":"1","message":"WebSocket
> > successfully
> > opened","logger_name":"io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager","thread_name":"OkHttp
> > https://172.31.64.1/...","level":"DEBUG","level_value":10000}
> > >
> > {"@timestamp":"2021-10-19T04:49:06.833+02:00","@version":"1","message":"Triggering
> > checkpoint 2930 (type=CHECKPOINT) @ 1634611746678 for job
> > 00000000000000000000000000000000.","logger_name":"org.apache.flink.runtime.checkpoint.CheckpointCoordinator","thread_name":"Checkpoint
> > Timer","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:49:07.000+02:00","@version":"1","message":"Completed
> > checkpoint 2930 for job 00000000000000000000000000000000 (474 bytes in 229
> > ms).","logger_name":"org.apache.flink.runtime.checkpoint.CheckpointCoordinator","thread_name":"jobmanager-future-thread-1","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:51:40.861+02:00","@version":"1","message":"WebSocket
> > close received. code: 1000, reason:
> > ","logger_name":"io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager","thread_name":"OkHttp
> > WebSocket https://172.31.64.1/...","level":"DEBUG","level_value":10000}
> > >
> > {"@timestamp":"2021-10-19T04:51:40.862+02:00","@version":"1","message":"Submitting
> > reconnect task to the
> > executor","logger_name":"io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager","thread_name":"OkHttp
> > WebSocket https://172.31.64.1/...","level":"DEBUG","level_value":10000}
> > >
> > {"@timestamp":"2021-10-19T04:51:40.862+02:00","@version":"1","message":"Scheduling
> > reconnect
> > task","logger_name":"io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager","thread_name":"scheduleReconnect|Executor
> > for Watch 1899640261","level":"DEBUG","level_value":10000}
> > >
> > {"@timestamp":"2021-10-19T04:51:40.862+02:00","@version":"1","message":"Current
> > reconnect backoff is 1000 milliseconds
> > (T0)","logger_name":"io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager","thread_name":"scheduleReconnect|Executor
> > for Watch 1899640261","level":"DEBUG","level_value":10000}
> > >
> > {"@timestamp":"2021-10-19T04:51:41.863+02:00","@version":"1","message":"Connecting
> > websocket ...
> > io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager@713a35c5","logger_name":"io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager","thread_name":"reconnectAttempt|Executor
> > for Watch 1899640261","level":"DEBUG","level_value":10000}
> > >
> > {"@timestamp":"2021-10-19T04:51:41.864+02:00","@version":"1","message":"WebSocket
> > successfully
> > opened","logger_name":"io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager","thread_name":"OkHttp
> > https://172.31.64.1/...","level":"DEBUG","level_value":10000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.916+02:00","@version":"1","message":"Renew
> > deadline reached after 120 seconds while renewing lock ConfigMapLock:
> > flink-dev - job1-dispatcher-leader
> > (8f7dddbb-236a-43fe-a706-1795b494838d)","logger_name":"io.fabric8.kubernetes.client.extended.leaderelection.LeaderElector","thread_name":"pool-19-thread-1","level":"DEBUG","level_value":10000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.918+02:00","@version":"1","message":"Stopping
> > SessionDispatcherLeaderProcess.","logger_name":"org.apache.flink.runtime.dispatcher.runner.SessionDispatcherLeaderProcess","thread_name":"KubernetesLeaderElector-ExecutorService-thread-1","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.919+02:00","@version":"1","message":"Stopping
> > dispatcher akka.tcp://flink@172.24.10.160:6123/user/rpc/dispatcher_1
> > .","logger_name":"org.apache.flink.runtime.dispatcher.StandaloneDispatcher","thread_name":"flink-akka.actor.default-dispatcher-61","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.919+02:00","@version":"1","message":"Stopping
> > all currently running jobs of dispatcher akka.tcp://
> > flink@172.24.10.160:6123/user/rpc/dispatcher_1
> > .","logger_name":"org.apache.flink.runtime.dispatcher.StandaloneDispatcher","thread_name":"flink-akka.actor.default-dispatcher-61","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.920+02:00","@version":"1","message":"Stopping
> > the JobMaster for job
> > flinkJob(00000000000000000000000000000000).","logger_name":"org.apache.flink.runtime.jobmaster.JobMaster","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.920+02:00","@version":"1","message":"Stopping
> > DefaultLeaderRetrievalService.","logger_name":"org.apache.flink.runtime.leaderretrieval.DefaultLeaderRetrievalService","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.921+02:00","@version":"1","message":"Stopping
> > KubernetesLeaderRetrievalDriver{configMapName='job1-resourcemanager-leader'}.","logger_name":"org.apache.flink.kubernetes.highavailability.KubernetesLeaderRetrievalDriver","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.921+02:00","@version":"1","message":"Force
> > closing the watch
> > io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager@367fad26
> > ","logger_name":"io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager","thread_name":"flink-akka.actor.default-dispatcher-68","level":"DEBUG","level_value":10000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.921+02:00","@version":"1","message":"The
> > watcher is
> > closing.","logger_name":"org.apache.flink.kubernetes.kubeclient.resources.KubernetesConfigMapWatcher","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.921+02:00","@version":"1","message":"Closing
> > websocket org.apache.flink.kubernetes.shaded.okhttp3.internal.ws
> > .RealWebSocket@28d6d80e
> > ","logger_name":"io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager","thread_name":"flink-akka.actor.default-dispatcher-68","level":"DEBUG","level_value":10000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.922+02:00","@version":"1","message":"WebSocket
> > close received. code: 1000, reason:
> > ","logger_name":"io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager","thread_name":"OkHttp
> > https://172.31.64.1/...","level":"DEBUG","level_value":10000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.922+02:00","@version":"1","message":"Ignoring
> > onClose for already closed/closing
> > websocket","logger_name":"io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager","thread_name":"OkHttp
> > https://172.31.64.1/...","level":"DEBUG","level_value":10000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.922+02:00","@version":"1","message":"Job
> > flinkJob (00000000000000000000000000000000) switched from state RUNNING to
> > SUSPENDED.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000,"stack_trace":"org.apache.flink.util.FlinkException:
> > Stopping JobMaster for job flinkJob(00000000000000000000000000000000).
> > > org.apache.flink.runtime.jobmaster.JobMaster.onStop(JobMaster.java:413)
> > >
> > org.apache.flink.runtime.rpc.RpcEndpoint.internalCallOnStop(RpcEndpoint.java:214)
> > >
> > org.apache.flink.runtime.rpc.akka.AkkaRpcActor$StartedState.terminate(AkkaRpcActor.java:563)
> > >
> > org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleControlMessage(AkkaRpcActor.java:186)
> > > akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26)
> > > akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21)
> > > scala.PartialFunction.applyOrElse(PartialFunction.scala:123)
> > > scala.PartialFunction.applyOrElse$(PartialFunction.scala:122)
> > > akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21)
> > > scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)
> > > scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172)
> > > akka.actor.Actor.aroundReceive(Actor.scala:517)
> > > akka.actor.Actor.aroundReceive$(Actor.scala:515)
> > > akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225)
> > > akka.actor.ActorCell.receiveMessage(ActorCell.scala:592)
> > > akka.actor.ActorCell.invoke(ActorCell.scala:561)
> > > akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258)
> > > akka.dispatch.Mailbox.run(Mailbox.scala:225)
> > > akka.dispatch.Mailbox.exec(Mailbox.scala:235)
> > > akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
> > >
> > akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
> > > akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
> > >
> > akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
> > > "}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.925+02:00","@version":"1","message":"Source:
> > Rabbit MQ (1/1) (998a108c41f221566faca1d0e27530a6) switched from RUNNING to
> > CANCELING.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.926+02:00","@version":"1","message":"Source:
> > Rabbit MQ (1/1) (998a108c41f221566faca1d0e27530a6) switched from CANCELING
> > to
> > CANCELED.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.927+02:00","@version":"1","message":"Discarding
> > the results produced by task execution
> > 998a108c41f221566faca1d0e27530a6.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.928+02:00","@version":"1","message":"Discarding
> > the results produced by task execution
> > 998a108c41f221566faca1d0e27530a6.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.930+02:00","@version":"1","message":"Json
> > conversion (1/2) (e7779a5274b0d154b9ce1a6db8d03912) switched from RUNNING
> > to
> > CANCELING.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.930+02:00","@version":"1","message":"Json
> > conversion (1/2) (e7779a5274b0d154b9ce1a6db8d03912) switched from CANCELING
> > to
> > CANCELED.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.930+02:00","@version":"1","message":"Discarding
> > the results produced by task execution
> > e7779a5274b0d154b9ce1a6db8d03912.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.930+02:00","@version":"1","message":"Discarding
> > the results produced by task execution
> > e7779a5274b0d154b9ce1a6db8d03912.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.931+02:00","@version":"1","message":"Json
> > conversion (2/2) (31a2506d16e7e5057d01c5b115e280b8) switched from RUNNING
> > to
> > CANCELING.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.931+02:00","@version":"1","message":"Json
> > conversion (2/2) (31a2506d16e7e5057d01c5b115e280b8) switched from CANCELING
> > to
> > CANCELED.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.931+02:00","@version":"1","message":"Discarding
> > the results produced by task execution
> > 31a2506d16e7e5057d01c5b115e280b8.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.931+02:00","@version":"1","message":"Discarding
> > the results produced by task execution
> > 31a2506d16e7e5057d01c5b115e280b8.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.931+02:00","@version":"1","message":"Filter
> > with avro schema (1/2) (1190c97468ad43833ca0cfa7f46c4c50) switched from
> > RUNNING to
> > CANCELING.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.932+02:00","@version":"1","message":"Filter
> > with avro schema (1/2) (1190c97468ad43833ca0cfa7f46c4c50) switched from
> > CANCELING to
> > CANCELED.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.932+02:00","@version":"1","message":"Discarding
> > the results produced by task execution
> > 1190c97468ad43833ca0cfa7f46c4c50.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.932+02:00","@version":"1","message":"Discarding
> > the results produced by task execution
> > 1190c97468ad43833ca0cfa7f46c4c50.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.932+02:00","@version":"1","message":"Filter
> > with avro schema (2/2) (923e11bd4045c9d5321a28507c640dc9) switched from
> > RUNNING to
> > CANCELING.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.932+02:00","@version":"1","message":"Filter
> > with avro schema (2/2) (923e11bd4045c9d5321a28507c640dc9) switched from
> > CANCELING to
> > CANCELED.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.933+02:00","@version":"1","message":"Discarding
> > the results produced by task execution
> > 923e11bd4045c9d5321a28507c640dc9.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.933+02:00","@version":"1","message":"Discarding
> > the results produced by task execution
> > 923e11bd4045c9d5321a28507c640dc9.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.933+02:00","@version":"1","message":"Filter
> > Strategy Pattern (1/2) (62b200109f0b1395543c496a9c8c1790) switched from
> > RUNNING to
> > CANCELING.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.933+02:00","@version":"1","message":"Filter
> > Strategy Pattern (1/2) (62b200109f0b1395543c496a9c8c1790) switched from
> > CANCELING to
> > CANCELED.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.933+02:00","@version":"1","message":"Discarding
> > the results produced by task execution
> > 62b200109f0b1395543c496a9c8c1790.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.934+02:00","@version":"1","message":"Discarding
> > the results produced by task execution
> > 62b200109f0b1395543c496a9c8c1790.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.934+02:00","@version":"1","message":"Filter
> > Strategy Pattern (2/2) (f47da1a29252a9d2bff505953699c012) switched from
> > RUNNING to
> > CANCELING.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.934+02:00","@version":"1","message":"Filter
> > Strategy Pattern (2/2) (f47da1a29252a9d2bff505953699c012) switched from
> > CANCELING to
> > CANCELED.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.934+02:00","@version":"1","message":"Discarding
> > the results produced by task execution
> > f47da1a29252a9d2bff505953699c012.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.935+02:00","@version":"1","message":"Discarding
> > the results produced by task execution
> > f47da1a29252a9d2bff505953699c012.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.935+02:00","@version":"1","message":"Get
> > Subscriptions (1/2) (a7fdf13cf85ebc37be56b4990aacb980) switched from
> > RUNNING to
> > CANCELING.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.935+02:00","@version":"1","message":"Get
> > Subscriptions (1/2) (a7fdf13cf85ebc37be56b4990aacb980) switched from
> > CANCELING to
> > CANCELED.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.935+02:00","@version":"1","message":"Discarding
> > the results produced by task execution
> > a7fdf13cf85ebc37be56b4990aacb980.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.936+02:00","@version":"1","message":"Discarding
> > the results produced by task execution
> > a7fdf13cf85ebc37be56b4990aacb980.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.936+02:00","@version":"1","message":"Get
> > Subscriptions (2/2) (27b64dbb773ec86cc0d350ffd57da1d1) switched from
> > RUNNING to
> > CANCELING.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.936+02:00","@version":"1","message":"Get
> > Subscriptions (2/2) (27b64dbb773ec86cc0d350ffd57da1d1) switched from
> > CANCELING to
> > CANCELED.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.936+02:00","@version":"1","message":"Discarding
> > the results produced by task execution
> > 27b64dbb773ec86cc0d350ffd57da1d1.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.936+02:00","@version":"1","message":"Discarding
> > the results produced by task execution
> > 27b64dbb773ec86cc0d350ffd57da1d1.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.936+02:00","@version":"1","message":"httpAd
> > call (1/2) (533763bc8108d3bcf2935aeb35970f0c) switched from RUNNING to
> > CANCELING.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.937+02:00","@version":"1","message":"httpAd
> > call (1/2) (533763bc8108d3bcf2935aeb35970f0c) switched from CANCELING to
> > CANCELED.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.937+02:00","@version":"1","message":"Discarding
> > the results produced by task execution
> > 533763bc8108d3bcf2935aeb35970f0c.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.937+02:00","@version":"1","message":"Discarding
> > the results produced by task execution
> > 533763bc8108d3bcf2935aeb35970f0c.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.937+02:00","@version":"1","message":"httpAd
> > call (2/2) (601e7e96fc3a1af7a5d79421c75c9bc7) switched from RUNNING to
> > CANCELING.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.937+02:00","@version":"1","message":"httpAd
> > call (2/2) (601e7e96fc3a1af7a5d79421c75c9bc7) switched from CANCELING to
> > CANCELED.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.937+02:00","@version":"1","message":"Discarding
> > the results produced by task execution
> > 601e7e96fc3a1af7a5d79421c75c9bc7.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.938+02:00","@version":"1","message":"Discarding
> > the results produced by task execution
> > 601e7e96fc3a1af7a5d79421c75c9bc7.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.938+02:00","@version":"1","message":"Update
> > Main Record (1/2) (2004748b8a985c997bb1e96c21d7b7ea) switched from RUNNING
> > to
> > CANCELING.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.938+02:00","@version":"1","message":"Update
> > Main Record (1/2) (2004748b8a985c997bb1e96c21d7b7ea) switched from
> > CANCELING to
> > CANCELED.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.938+02:00","@version":"1","message":"Discarding
> > the results produced by task execution
> > 2004748b8a985c997bb1e96c21d7b7ea.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.938+02:00","@version":"1","message":"Discarding
> > the results produced by task execution
> > 2004748b8a985c997bb1e96c21d7b7ea.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.938+02:00","@version":"1","message":"Update
> > Main Record (2/2) (13f2a23c52db212bcc531d1f9f7a41b8) switched from RUNNING
> > to
> > CANCELING.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.939+02:00","@version":"1","message":"Update
> > Main Record (2/2) (13f2a23c52db212bcc531d1f9f7a41b8) switched from
> > CANCELING to
> > CANCELED.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.939+02:00","@version":"1","message":"Discarding
> > the results produced by task execution
> > 13f2a23c52db212bcc531d1f9f7a41b8.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.982+02:00","@version":"1","message":"Discarding
> > the results produced by task execution
> > 13f2a23c52db212bcc531d1f9f7a41b8.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.983+02:00","@version":"1","message":"Sink:
> > Send to Notifier (1/2) (4467a95a23f6a5e983f006fb12832b10) switched from
> > RUNNING to
> > CANCELING.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.983+02:00","@version":"1","message":"Sink:
> > Send to Notifier (1/2) (4467a95a23f6a5e983f006fb12832b10) switched from
> > CANCELING to
> > CANCELED.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.984+02:00","@version":"1","message":"Discarding
> > the results produced by task execution
> > 4467a95a23f6a5e983f006fb12832b10.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.985+02:00","@version":"1","message":"Sink:
> > Send to Notifier (2/2) (cf5ce6a12a584da4651a7d341cb1aeef) switched from
> > RUNNING to
> > CANCELING.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.985+02:00","@version":"1","message":"Sink:
> > Send to Notifier (2/2) (cf5ce6a12a584da4651a7d341cb1aeef) switched from
> > CANCELING to
> > CANCELED.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.985+02:00","@version":"1","message":"Discarding
> > the results produced by task execution
> > cf5ce6a12a584da4651a7d341cb1aeef.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.985+02:00","@version":"1","message":"Stopping
> > checkpoint coordinator for job
> > 00000000000000000000000000000000.","logger_name":"org.apache.flink.runtime.checkpoint.CheckpointCoordinator","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.986+02:00","@version":"1","message":"Suspending","logger_name":"org.apache.flink.runtime.checkpoint.DefaultCompletedCheckpointStore","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.986+02:00","@version":"1","message":"Shutting
> > down.","logger_name":"org.apache.flink.kubernetes.highavailability.KubernetesCheckpointIDCounter","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.986+02:00","@version":"1","message":"Job
> > 00000000000000000000000000000000 has been
> > suspended.","logger_name":"org.apache.flink.runtime.executiongraph.ExecutionGraph","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.990+02:00","@version":"1","message":"Suspending
> > SlotPool.","logger_name":"org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.990+02:00","@version":"1","message":"Close
> > ResourceManager connection 81a0f3b34590e11df305b58878a355b1: Stopping
> > JobMaster for job
> > flinkJob(00000000000000000000000000000000)..","logger_name":"org.apache.flink.runtime.jobmaster.JobMaster","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.991+02:00","@version":"1","message":"Stopping
> > SlotPool.","logger_name":"org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.991+02:00","@version":"1","message":"Disconnect
> > job manager bc901e3d25d406f70027b81673c74...@akka.tcp://
> > flink@172.24.10.160:6123/user/rpc/jobmanager_2 for job
> > 00000000000000000000000000000000 from the resource
> > manager.","logger_name":"org.apache.flink.runtime.resourcemanager.StandaloneResourceManager","thread_name":"flink-akka.actor.default-dispatcher-61","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.993+02:00","@version":"1","message":"Leader
> > election
> > started","logger_name":"io.fabric8.kubernetes.client.extended.leaderelection.LeaderElector","thread_name":"KubernetesLeaderElector-ExecutorService-thread-1","level":"DEBUG","level_value":10000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.993+02:00","@version":"1","message":"Stopping
> > DefaultLeaderElectionService.","logger_name":"org.apache.flink.runtime.leaderelection.DefaultLeaderElectionService","thread_name":"AkkaRpcService-Supervisor-Termination-Future-Executor-thread-1","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.993+02:00","@version":"1","message":"Attempting
> > to acquire leader lease 'ConfigMapLock: flink-dev - job1-dispatcher-leader
> > (8f7dddbb-236a-43fe-a706-1795b494838d)'...","logger_name":"io.fabric8.kubernetes.client.extended.leaderelection.LeaderElector","thread_name":"KubernetesLeaderElector-ExecutorService-thread-1","level":"DEBUG","level_value":10000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.994+02:00","@version":"1","message":"Closing
> > KubernetesLeaderElectionDriver{configMapName='job1-00000000000000000000000000000000-jobmanager-leader'}.","logger_name":"org.apache.flink.kubernetes.highavailability.KubernetesLeaderElectionDriver","thread_name":"AkkaRpcService-Supervisor-Termination-Future-Executor-thread-1","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.994+02:00","@version":"1","message":"Force
> > closing the watch
> > io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager@4d181edd
> > ","logger_name":"io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager","thread_name":"AkkaRpcService-Supervisor-Termination-Future-Executor-thread-1","level":"DEBUG","level_value":10000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.994+02:00","@version":"1","message":"The
> > watcher is
> > closing.","logger_name":"org.apache.flink.kubernetes.kubeclient.resources.KubernetesConfigMapWatcher","thread_name":"AkkaRpcService-Supervisor-Termination-Future-Executor-thread-1","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.994+02:00","@version":"1","message":"Closing
> > websocket org.apache.flink.kubernetes.shaded.okhttp3.internal.ws
> > .RealWebSocket@5efd28bd
> > ","logger_name":"io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager","thread_name":"AkkaRpcService-Supervisor-Termination-Future-Executor-thread-1","level":"DEBUG","level_value":10000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.994+02:00","@version":"1","message":"Loop
> > thread
> > interrupted","logger_name":"io.fabric8.kubernetes.client.extended.leaderelection.LeaderElector","thread_name":"KubernetesLeaderElector-ExecutorService-thread-1","level":"DEBUG","level_value":10000,"stack_trace":"java.lang.InterruptedException:
> > null
> > >
> > java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:998)
> > >
> > java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
> > > java.util.concurrent.CountDownLatch.await(CountDownLatch.java:231)
> > >
> > io.fabric8.kubernetes.client.extended.leaderelection.LeaderElector.loop(LeaderElector.java:200)
> > >
> > io.fabric8.kubernetes.client.extended.leaderelection.LeaderElector.renewWithTimeout(LeaderElector.java:100)
> > >
> > io.fabric8.kubernetes.client.extended.leaderelection.LeaderElector.run(LeaderElector.java:71)
> > >
> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > >
> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > > java.lang.Thread.run(Thread.java:748)
> > > "}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.995+02:00","@version":"1","message":"WebSocket
> > close received. code: 1000, reason:
> > ","logger_name":"io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager","thread_name":"OkHttp
> > https://172.31.64.1/...","level":"DEBUG","level_value":10000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.995+02:00","@version":"1","message":"Job
> > 00000000000000000000000000000000 reached terminal state
> > SUSPENDED.","logger_name":"org.apache.flink.runtime.dispatcher.StandaloneDispatcher","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.995+02:00","@version":"1","message":"Ignoring
> > onClose for already closed/closing
> > websocket","logger_name":"io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager","thread_name":"OkHttp
> > https://172.31.64.1/...","level":"DEBUG","level_value":10000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.996+02:00","@version":"1","message":"Released
> > job graph 00000000000000000000000000000000 from
> > KubernetesStateHandleStore{configMapName='job1-dispatcher-leader'}.","logger_name":"org.apache.flink.runtime.jobmanager.DefaultJobGraphStore","thread_name":"cluster-io-thread-4","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.997+02:00","@version":"1","message":"Application
> > failed unexpectedly:
> > ","logger_name":"org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap","thread_name":"cluster-io-thread-4","level":"WARN","level_value":30000,"stack_trace":"java.util.concurrent.CancellationException:
> > null
> > >
> > java.util.concurrent.CompletableFuture.cancel(CompletableFuture.java:2276)
> > >
> > org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.stop(ApplicationDispatcherBootstrap.java:120)
> > >
> > org.apache.flink.runtime.dispatcher.Dispatcher.lambda$onStop$0(Dispatcher.java:274)
> > >
> > org.apache.flink.runtime.concurrent.FutureUtils.lambda$runAfterwardsAsync$18(FutureUtils.java:687)
> > >
> > java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
> > >
> > java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
> > >
> > java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:456)
> > >
> > org.apache.flink.runtime.concurrent.DirectExecutorService.execute(DirectExecutorService.java:217)
> > >
> > java.util.concurrent.CompletableFuture$UniCompletion.claim(CompletableFuture.java:543)
> > >
> > java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:765)
> > >
> > java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
> > >
> > java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
> > >
> > java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
> > >
> > org.apache.flink.runtime.concurrent.FutureUtils$CompletionConjunctFuture.completeFuture(FutureUtils.java:971)
> > >
> > java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
> > >
> > java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
> > >
> > java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
> > >
> > java.util.concurrent.CompletableFuture.postFire(CompletableFuture.java:575)
> > >
> > java.util.concurrent.CompletableFuture$UniRun.tryFire(CompletableFuture.java:704)
> > >
> > java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:456)
> > >
> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > >
> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > > java.lang.Thread.run(Thread.java:748)
> > > "}
> > >
> > {"@timestamp":"2021-10-19T04:53:16.998+02:00","@version":"1","message":"Fatal
> > error occurred in the cluster
> > entrypoint.","logger_name":"org.apache.flink.runtime.entrypoint.ClusterEntrypoint","thread_name":"cluster-io-thread-4","level":"ERROR","level_value":40000,"stack_trace":"org.apache.flink.util.FlinkException:
> > Application failed unexpectedly.
> > >
> > org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.lambda$runApplicationAndShutdownClusterAsync$0(ApplicationDispatcherBootstrap.java:170)
> > >
> > java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:836)
> > >
> > java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:811)
> > >
> > java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
> > >
> > java.util.concurrent.CompletableFuture.cancel(CompletableFuture.java:2278)
> > >
> > org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.stop(ApplicationDispatcherBootstrap.java:120)
> > >
> > org.apache.flink.runtime.dispatcher.Dispatcher.lambda$onStop$0(Dispatcher.java:274)
> > >
> > org.apache.flink.runtime.concurrent.FutureUtils.lambda$runAfterwardsAsync$18(FutureUtils.java:687)
> > >
> > java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
> > >
> > java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
> > >
> > java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:456)
> > >
> > org.apache.flink.runtime.concurrent.DirectExecutorService.execute(DirectExecutorService.java:217)
> > >
> > java.util.concurrent.CompletableFuture$UniCompletion.claim(CompletableFuture.java:543)
> > >
> > java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:765)
> > >
> > java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
> > >
> > java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
> > >
> > java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
> > >
> > org.apache.flink.runtime.concurrent.FutureUtils$CompletionConjunctFuture.completeFuture(FutureUtils.java:971)
> > >
> > java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
> > >
> > java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
> > >
> > java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
> > >
> > java.util.concurrent.CompletableFuture.postFire(CompletableFuture.java:575)
> > >
> > java.util.concurrent.CompletableFuture$UniRun.tryFire(CompletableFuture.java:704)
> > >
> > java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:456)
> > >
> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > >
> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > > java.lang.Thread.run(Thread.java:748)
> > > Caused by: java.util.concurrent.CancellationException: null
> > >
> > java.util.concurrent.CompletableFuture.cancel(CompletableFuture.java:2276)
> > > \t... 22 common frames omitted
> > > "}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.084+02:00","@version":"1","message":"Shutting
> > StandaloneApplicationClusterEntryPoint down with application status
> > UNKNOWN. Diagnostics Cluster entrypoint has been closed
> > externally..","logger_name":"org.apache.flink.runtime.entrypoint.ClusterEntrypoint","thread_name":"StandaloneApplicationClusterEntryPoint
> > shutdown hook","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.085+02:00","@version":"1","message":"Shutting
> > down rest
> > endpoint.","logger_name":"org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint","thread_name":"StandaloneApplicationClusterEntryPoint
> > shutdown hook","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.086+02:00","@version":"1","message":"Stopped
> > BLOB server at
> > 0.0.0.0:6124","logger_name":"org.apache.flink.runtime.blob.BlobServer","thread_name":"BlobServer
> > shutdown hook","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.090+02:00","@version":"1","message":"Successfully
> > Acquired leader lease 'ConfigMapLock: flink-dev - job1-dispatcher-leader
> > (8f7dddbb-236a-43fe-a706-1795b494838d)'","logger_name":"io.fabric8.kubernetes.client.extended.leaderelection.LeaderElector","thread_name":"pool-25081-thread-1","level":"DEBUG","level_value":10000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.090+02:00","@version":"1","message":"Attempting
> > to renew leader lease 'ConfigMapLock: flink-dev - job1-dispatcher-leader
> > (8f7dddbb-236a-43fe-a706-1795b494838d)'...","logger_name":"io.fabric8.kubernetes.client.extended.leaderelection.LeaderElector","thread_name":"KubernetesLeaderElector-ExecutorService-thread-1","level":"DEBUG","level_value":10000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.185+02:00","@version":"1","message":"Removing
> > cache directory
> > /tmp/flink-web-682b3e4f-ed72-4b00-96a0-da5c8248b62c/flink-web-ui","logger_name":"org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint","thread_name":"Thread-29","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.186+02:00","@version":"1","message":"Stopping
> > DefaultLeaderElectionService.","logger_name":"org.apache.flink.runtime.leaderelection.DefaultLeaderElectionService","thread_name":"Thread-29","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.186+02:00","@version":"1","message":"Closing
> > KubernetesLeaderElectionDriver{configMapName='job1-restserver-leader'}.","logger_name":"org.apache.flink.kubernetes.highavailability.KubernetesLeaderElectionDriver","thread_name":"Thread-29","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.186+02:00","@version":"1","message":"Force
> > closing the watch
> > io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager@6f9e08d4
> > ","logger_name":"io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager","thread_name":"Thread-29","level":"DEBUG","level_value":10000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.186+02:00","@version":"1","message":"The
> > watcher is
> > closing.","logger_name":"org.apache.flink.kubernetes.kubeclient.resources.KubernetesConfigMapWatcher","thread_name":"Thread-29","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.186+02:00","@version":"1","message":"Closing
> > websocket org.apache.flink.kubernetes.shaded.okhttp3.internal.ws
> > .RealWebSocket@42c1c89c
> > ","logger_name":"io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager","thread_name":"Thread-29","level":"DEBUG","level_value":10000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.186+02:00","@version":"1","message":"Loop
> > thread
> > interrupted","logger_name":"io.fabric8.kubernetes.client.extended.leaderelection.LeaderElector","thread_name":"KubernetesLeaderElector-ExecutorService-thread-1","level":"DEBUG","level_value":10000,"stack_trace":"java.lang.InterruptedException:
> > null
> > >
> > java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:998)
> > >
> > java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
> > > java.util.concurrent.CountDownLatch.await(CountDownLatch.java:231)
> > >
> > io.fabric8.kubernetes.client.extended.leaderelection.LeaderElector.loop(LeaderElector.java:200)
> > >
> > io.fabric8.kubernetes.client.extended.leaderelection.LeaderElector.renewWithTimeout(LeaderElector.java:100)
> > >
> > io.fabric8.kubernetes.client.extended.leaderelection.LeaderElector.run(LeaderElector.java:71)
> > >
> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > >
> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > > java.lang.Thread.run(Thread.java:748)
> > > "}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.186+02:00","@version":"1","message":"Shut
> > down
> > complete.","logger_name":"org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint","thread_name":"Thread-29","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.186+02:00","@version":"1","message":"Closing
> > components.","logger_name":"org.apache.flink.runtime.entrypoint.component.DispatcherResourceManagerComponent","thread_name":"Thread-29","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.187+02:00","@version":"1","message":"Stopping
> > DefaultLeaderRetrievalService.","logger_name":"org.apache.flink.runtime.leaderretrieval.DefaultLeaderRetrievalService","thread_name":"Thread-29","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.187+02:00","@version":"1","message":"Stopping
> > KubernetesLeaderRetrievalDriver{configMapName='job1-dispatcher-leader'}.","logger_name":"org.apache.flink.kubernetes.highavailability.KubernetesLeaderRetrievalDriver","thread_name":"Thread-29","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.187+02:00","@version":"1","message":"WebSocket
> > close received. code: 1000, reason:
> > ","logger_name":"io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager","thread_name":"OkHttp
> > https://172.31.64.1/...","level":"DEBUG","level_value":10000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.187+02:00","@version":"1","message":"Force
> > closing the watch
> > io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager@11787b64
> > ","logger_name":"io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager","thread_name":"Thread-29","level":"DEBUG","level_value":10000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.187+02:00","@version":"1","message":"Ignoring
> > onClose for already closed/closing
> > websocket","logger_name":"io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager","thread_name":"OkHttp
> > https://172.31.64.1/...","level":"DEBUG","level_value":10000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.187+02:00","@version":"1","message":"The
> > watcher is
> > closing.","logger_name":"org.apache.flink.kubernetes.kubeclient.resources.KubernetesConfigMapWatcher","thread_name":"Thread-29","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.187+02:00","@version":"1","message":"Closing
> > websocket org.apache.flink.kubernetes.shaded.okhttp3.internal.ws
> > .RealWebSocket@4c063eb4
> > ","logger_name":"io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager","thread_name":"Thread-29","level":"DEBUG","level_value":10000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.188+02:00","@version":"1","message":"Stopping
> > DefaultLeaderRetrievalService.","logger_name":"org.apache.flink.runtime.leaderretrieval.DefaultLeaderRetrievalService","thread_name":"Thread-29","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.188+02:00","@version":"1","message":"Stopping
> > KubernetesLeaderRetrievalDriver{configMapName='job1-resourcemanager-leader'}.","logger_name":"org.apache.flink.kubernetes.highavailability.KubernetesLeaderRetrievalDriver","thread_name":"Thread-29","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.188+02:00","@version":"1","message":"Force
> > closing the watch
> > io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager@713a35c5
> > ","logger_name":"io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager","thread_name":"Thread-29","level":"DEBUG","level_value":10000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.188+02:00","@version":"1","message":"The
> > watcher is
> > closing.","logger_name":"org.apache.flink.kubernetes.kubeclient.resources.KubernetesConfigMapWatcher","thread_name":"Thread-29","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.188+02:00","@version":"1","message":"Closing
> > websocket org.apache.flink.kubernetes.shaded.okhttp3.internal.ws
> > .RealWebSocket@4839a3ec
> > ","logger_name":"io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager","thread_name":"Thread-29","level":"DEBUG","level_value":10000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.188+02:00","@version":"1","message":"WebSocket
> > close received. code: 1000, reason:
> > ","logger_name":"io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager","thread_name":"OkHttp
> > https://172.31.64.1/...","level":"DEBUG","level_value":10000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.188+02:00","@version":"1","message":"Ignoring
> > onClose for already closed/closing
> > websocket","logger_name":"io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager","thread_name":"OkHttp
> > https://172.31.64.1/...","level":"DEBUG","level_value":10000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.188+02:00","@version":"1","message":"Stopping
> > DefaultLeaderElectionService.","logger_name":"org.apache.flink.runtime.leaderelection.DefaultLeaderElectionService","thread_name":"Thread-29","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.188+02:00","@version":"1","message":"Closing
> > KubernetesLeaderElectionDriver{configMapName='job1-dispatcher-leader'}.","logger_name":"org.apache.flink.kubernetes.highavailability.KubernetesLeaderElectionDriver","thread_name":"Thread-29","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.188+02:00","@version":"1","message":"Force
> > closing the watch
> > io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager@202d9236
> > ","logger_name":"io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager","thread_name":"Thread-29","level":"DEBUG","level_value":10000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.189+02:00","@version":"1","message":"The
> > watcher is
> > closing.","logger_name":"org.apache.flink.kubernetes.kubeclient.resources.KubernetesConfigMapWatcher","thread_name":"Thread-29","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.188+02:00","@version":"1","message":"WebSocket
> > close received. code: 1000, reason:
> > ","logger_name":"io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager","thread_name":"OkHttp
> > https://172.31.64.1/...","level":"DEBUG","level_value":10000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.189+02:00","@version":"1","message":"Closing
> > websocket org.apache.flink.kubernetes.shaded.okhttp3.internal.ws
> > .RealWebSocket@d3b348f
> > ","logger_name":"io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager","thread_name":"Thread-29","level":"DEBUG","level_value":10000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.189+02:00","@version":"1","message":"Ignoring
> > onClose for already closed/closing
> > websocket","logger_name":"io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager","thread_name":"OkHttp
> > https://172.31.64.1/...","level":"DEBUG","level_value":10000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.188+02:00","@version":"1","message":"Loop
> > thread
> > interrupted","logger_name":"io.fabric8.kubernetes.client.extended.leaderelection.LeaderElector","thread_name":"KubernetesLeaderElector-ExecutorService-thread-1","level":"DEBUG","level_value":10000,"stack_trace":"java.lang.InterruptedException:
> > null
> > >
> > java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:998)
> > >
> > java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
> > > java.util.concurrent.CountDownLatch.await(CountDownLatch.java:231)
> > >
> > io.fabric8.kubernetes.client.extended.leaderelection.LeaderElector.loop(LeaderElector.java:200)
> > >
> > io.fabric8.kubernetes.client.extended.leaderelection.LeaderElector.renewWithTimeout(LeaderElector.java:100)
> > >
> > io.fabric8.kubernetes.client.extended.leaderelection.LeaderElector.run(LeaderElector.java:71)
> > >
> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > >
> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > > java.lang.Thread.run(Thread.java:748)
> > > "}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.189+02:00","@version":"1","message":"Stopping
> > SessionDispatcherLeaderProcess.","logger_name":"org.apache.flink.runtime.dispatcher.runner.SessionDispatcherLeaderProcess","thread_name":"Thread-29","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.189+02:00","@version":"1","message":"WebSocket
> > close received. code: 1000, reason:
> > ","logger_name":"io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager","thread_name":"OkHttp
> > https://172.31.64.1/...","level":"DEBUG","level_value":10000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.190+02:00","@version":"1","message":"Ignoring
> > onClose for already closed/closing
> > websocket","logger_name":"io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager","thread_name":"OkHttp
> > https://172.31.64.1/...","level":"DEBUG","level_value":10000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.190+02:00","@version":"1","message":"Closing
> > the
> > SlotManager.","logger_name":"org.apache.flink.runtime.resourcemanager.slotmanager.SlotManagerImpl","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.190+02:00","@version":"1","message":"Suspending
> > the
> > SlotManager.","logger_name":"org.apache.flink.runtime.resourcemanager.slotmanager.SlotManagerImpl","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.191+02:00","@version":"1","message":"Stopping
> > DefaultLeaderElectionService.","logger_name":"org.apache.flink.runtime.leaderelection.DefaultLeaderElectionService","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.191+02:00","@version":"1","message":"Closing
> > KubernetesLeaderElectionDriver{configMapName='job1-resourcemanager-leader'}.","logger_name":"org.apache.flink.kubernetes.highavailability.KubernetesLeaderElectionDriver","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.191+02:00","@version":"1","message":"Force
> > closing the watch
> > io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager@7be008e9
> > ","logger_name":"io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager","thread_name":"flink-akka.actor.default-dispatcher-68","level":"DEBUG","level_value":10000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.191+02:00","@version":"1","message":"The
> > watcher is
> > closing.","logger_name":"org.apache.flink.kubernetes.kubeclient.resources.KubernetesConfigMapWatcher","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.191+02:00","@version":"1","message":"Loop
> > thread
> > interrupted","logger_name":"io.fabric8.kubernetes.client.extended.leaderelection.LeaderElector","thread_name":"KubernetesLeaderElector-ExecutorService-thread-1","level":"DEBUG","level_value":10000,"stack_trace":"java.lang.InterruptedException:
> > null
> > >
> > java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:998)
> > >
> > java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
> > > java.util.concurrent.CountDownLatch.await(CountDownLatch.java:231)
> > >
> > io.fabric8.kubernetes.client.extended.leaderelection.LeaderElector.loop(LeaderElector.java:200)
> > >
> > io.fabric8.kubernetes.client.extended.leaderelection.LeaderElector.renewWithTimeout(LeaderElector.java:100)
> > >
> > io.fabric8.kubernetes.client.extended.leaderelection.LeaderElector.run(LeaderElector.java:71)
> > >
> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > >
> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > > java.lang.Thread.run(Thread.java:748)
> > > "}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.191+02:00","@version":"1","message":"Closing
> > websocket org.apache.flink.kubernetes.shaded.okhttp3.internal.ws
> > .RealWebSocket@57ae1976
> > ","logger_name":"io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager","thread_name":"flink-akka.actor.default-dispatcher-68","level":"DEBUG","level_value":10000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.192+02:00","@version":"1","message":"Stopping
> > DefaultLeaderRetrievalService.","logger_name":"org.apache.flink.runtime.leaderretrieval.DefaultLeaderRetrievalService","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.192+02:00","@version":"1","message":"Stopping
> > KubernetesLeaderRetrievalDriver{configMapName='job1-00000000000000000000000000000000-jobmanager-leader'}.","logger_name":"org.apache.flink.kubernetes.highavailability.KubernetesLeaderRetrievalDriver","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.192+02:00","@version":"1","message":"Force
> > closing the watch
> > io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager@3f6e134d
> > ","logger_name":"io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager","thread_name":"flink-akka.actor.default-dispatcher-68","level":"DEBUG","level_value":10000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.192+02:00","@version":"1","message":"WebSocket
> > close received. code: 1000, reason:
> > ","logger_name":"io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager","thread_name":"OkHttp
> > https://172.31.64.1/...","level":"DEBUG","level_value":10000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.192+02:00","@version":"1","message":"The
> > watcher is
> > closing.","logger_name":"org.apache.flink.kubernetes.kubeclient.resources.KubernetesConfigMapWatcher","thread_name":"flink-akka.actor.default-dispatcher-68","level":"INFO","level_value":20000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.193+02:00","@version":"1","message":"Ignoring
> > onClose for already closed/closing
> > websocket","logger_name":"io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager","thread_name":"OkHttp
> > https://172.31.64.1/...","level":"DEBUG","level_value":10000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.193+02:00","@version":"1","message":"Closing
> > websocket org.apache.flink.kubernetes.shaded.okhttp3.internal.ws
> > .RealWebSocket@2a1892aa
> > ","logger_name":"io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager","thread_name":"flink-akka.actor.default-dispatcher-68","level":"DEBUG","level_value":10000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.193+02:00","@version":"1","message":"WebSocket
> > close received. code: 1000, reason:
> > ","logger_name":"io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager","thread_name":"OkHttp
> > https://172.31.64.1/...","level":"DEBUG","level_value":10000}
> > >
> > {"@timestamp":"2021-10-19T04:53:17.193+02:00","@version":"1","message":"Ignoring
> > onClose for already closed/closing
> > websocket","logger_name":"io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager","thread_name":"OkHttp
> > https://172.31.64.1/...","level":"DEBUG","level_value":10000}
> > >
> > >
> >
>
