你好,


请问使用的Flink版本是哪一个呢,可以提供下完整的作业提交参数和运行环境吗?

看log猜测是某个UI请求而不是runtime内部出现了问题,确认下如果不访问UI中的作业拓扑图的话整个作业是否可以正常执行。

最后有一个建议把log作为邮件附件发送的话查看起来比较方便一点^_。





> On 28 Feb 2022, at 11:21 PM, xinzhuxiansheng <xinzhuxiansh...@126.com> wrote:
> 
> 作业在HA或者单jobManager 写入kafka,消费kafka 都没问题 , 不过,Flink UI 
> 出现拓扑图打不开的情况只在HA模式下有,单jobManager一切都正常, 
> 不知道是否是HA配置有问题,看堆栈信息,我用远程debug来调试代码,可缺少对代码上下文的理解,有些找不到头绪。 目前把理解的重心 放在 
> “java.util.concurrent.CompletableFuture.cancel()”,为啥有 
> “java.util.concurrent.CancellationException: null”  
> 
> 
> 以下是我的job关于HA的参数:
> -Dkubernetes.container.image.pull-policy=Always
> -Dhigh-availability=org.apache.flink.kubernetes.highavailability.KubernetesHaServicesFactory
> -Dhigh-availability.storageDir=s3://fmflinkha/
> -Dcontainerized.master.env.ENABLE_BUILT_IN_PLUGINS=flink-s3-fs-hadoop-1.14.2.jar
> -Dcontainerized.taskmanager.env.ENABLE_BUILT_IN_PLUGINS=flink-s3-fs-hadoop-1.14.2.jar
> -Dkubernetes.jobmanager.replicas=2
> 
> 
> 
> 
> | |
> xinzhuxiansheng
> |
> |
> xinzhuxiansh...@126.com
> |
> 签名由网易邮箱大师定制
> 
> 
> 在2022年02月28日 21:48,胡伟华<huweihua....@gmail.com> 写道:
> 日志里没有 Dispatcher 、Job 的启动日志,请问作业是正常工作的吗?
> 如果作业不正常,可以检查下 leader 选主的逻辑,或者尝试去掉 HA,观察作业是否正常
> 
> 
> 2022年2月28日 下午8:57,xinzhuxiansheng <xinzhuxiansh...@126.com> 写道:
> 
> 你是对的,在你的提醒下 ,我看了完整的 log,确实还有一些error log  
> 
> 
> 
> 
> Enabling required built-in plugins
> Linking flink-s3-fs-hadoop-1.14.2.jar to plugin directory
> Successfully enabled flink-s3-fs-hadoop-1.14.2.jar
> sed: couldn't open temporary file /opt/flink/conf/sedqVEQdn: Read-only file 
> system
> sed: couldn't open temporary file /opt/flink/conf/sedNyxXkO: Read-only file 
> system
> /docker-entrypoint.sh: line 73: /opt/flink/conf/flink-conf.yaml: Read-only 
> file system
> /docker-entrypoint.sh: line 89: /opt/flink/conf/flink-conf.yaml.tmp: 
> Read-only file system
> Starting kubernetes-application as a console application on host 
> fm-hwm8u1o2x8-9cc6947f9-tgxd5.
> Listening for transport dt_socket at address: 5006
> 2022-02-28 12:23:49,009 INFO  
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] - 
> --------------------------------------------------------------------------------
> 2022-02-28 12:23:49,012 INFO  
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] -  
> Preconfiguration:
> 2022-02-28 12:23:49,012 INFO  
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] -
> 
> 
> RESOURCE_PARAMS extraction logs:
> jvm_params: -Xmx1073741824 -Xms1073741824 -XX:MaxMetaspaceSize=268435456
> dynamic_configs: -D jobmanager.memory.off-heap.size=134217728b -D 
> jobmanager.memory.jvm-overhead.min=201326592b -D 
> jobmanager.memory.jvm-metaspace.size=268435456b -D 
> jobmanager.memory.heap.size=1073741824b -D 
> jobmanager.memory.jvm-overhead.max=201326592b
> logs: INFO  [] - Loading configuration property: blob.server.port, 6124
> INFO  [] - Loading configuration property: env.java.opts.jobmanager, 
> "-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5006"
> INFO  [] - Loading configuration property: kubernetes.jobmanager.replicas, 2
> INFO  [] - Loading configuration property: 
> jobmanager.execution.failover-strategy, region
> INFO  [] - Loading configuration property: high-availability.cluster-id, 
> fm-hwm8u1o2x8
> INFO  [] - Loading configuration property: jobmanager.rpc.address, localhost
> INFO  [] - Loading configuration property: -parallelism.default, 10
> INFO  [] - Loading configuration property: 
> containerized.master.env.ENABLE_BUILT_IN_PLUGINS, 
> flink-s3-fs-hadoop-1.14.2.jar
> INFO  [] - Loading configuration property: kubernetes.taskmanager.cpu, 2
> INFO  [] - Loading configuration property: kubernetes.cluster-id, 
> fm-hwm8u1o2x8
> INFO  [] - Loading configuration property: kubernetes.service-account, flink
> INFO  [] - Loading configuration property: s3.secret-key, ******
> INFO  [] - Loading configuration property: high-availability.storageDir, 
> s3://xx.xx/flink-ha
> INFO  [] - Loading configuration property: s3.endpoint, xx.xx.com
> INFO  [] - Loading configuration property: 
> $internal.application.program-args, 
> --bootstrapServers;192.168.xx.xx:9092;--topic;yzhoutp01;--groupId;yzhougid022801;--s3path;s3://xx.xx/flink-ha;--flink.conf;;--app.name;HA消费作业;--conf;'json://{"$internal.application.main":"com.yzhou.checkpoint.UDFDataConsumerSource"}'
> INFO  [] - Loading configuration property: kubernetes.container.image, 
> flink:1.14.2-scala_2.11-java8
> INFO  [] - Loading configuration property: parallelism.default, 10
> INFO  [] - Loading configuration property: kubernetes.namespace, yzhou
> INFO  [] - Loading configuration property: taskmanager.numberOfTaskSlots, 4
> INFO  [] - Loading configuration property: 
> kubernetes.rest-service.exposed.type, NodePort
> INFO  [] - Loading configuration property: high-availability.jobmanager.port, 
> 6123
> INFO  [] - Loading configuration property: $internal.application.main, 
> com.yzhou.checkpoint.UDFDataConsumerSource
> INFO  [] - Loading configuration property: taskmanager.memory.process.size, 
> 1728m
> INFO  [] - Loading configuration property: 
> kubernetes.internal.jobmanager.entrypoint.class, 
> org.apache.flink.kubernetes.entrypoint.KubernetesApplicationClusterEntrypoint
> INFO  [] - Loading configuration property: pipeline.name, HA消费作业
> INFO  [] - Loading configuration property: classloader.resolve-order, 
> parent-first
> INFO  [] - Loading configuration property: kubernetes.pod-template-file, 
> /Users/xx.xx/Tmp/app/fm-hwm8u1o2x8/pod-template.yaml
> INFO  [] - Loading configuration property: execution.target, 
> kubernetes-application
> INFO  [] - Loading configuration property: 
> restart-strategy.fixed-delay.attempts, 10
> INFO  [] - Loading configuration property: jobmanager.memory.process.size, 
> 1600m
> INFO  [] - Loading configuration property: jobmanager.rpc.port, 6123
> INFO  [] - Loading configuration property: taskmanager.rpc.port, 6122
> INFO  [] - Loading configuration property: s3.access-key, 
> AKIDjCPQBY8JfniPinIbXbfbji1AB4sYNsK0
> INFO  [] - Loading configuration property: 
> kubernetes.container.image.pull-policy, Always
> INFO  [] - Loading configuration property: internal.cluster.execution-mode, 
> NORMAL
> INFO  [] - Loading configuration property: $internal.pipeline.job-id, 
> 72351556fc8d911c4de6e64750fd805a
> INFO  [] - Loading configuration property: 
> execution.shutdown-on-attached-exit, false
> INFO  [] - Loading configuration property: high-availability, 
> org.apache.flink.kubernetes.highavailability.KubernetesHaServicesFactory
> INFO  [] - Loading configuration property: pipeline.jars, 
> local:///opt/flink/usrlib/app.jar
> INFO  [] - Loading configuration property: restart-strategy, fixed-delay
> INFO  [] - Loading configuration property: env.java.opts.taskmanager, 
> "-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005"
> INFO  [] - Loading configuration property: 
> containerized.taskmanager.env.ENABLE_BUILT_IN_PLUGINS, 
> flink-s3-fs-hadoop-1.14.2.jar
> INFO  [] - Loading configuration property: s3.path.style.access, true
> INFO  [] - The derived from fraction jvm overhead memory (160.000mb 
> (167772162 bytes)) is less than its min value 192.000mb (201326592 bytes), 
> min value will be used instead
> INFO  [] - Final Master Memory configuration:
> INFO  [] -   Total Process Memory: 1.563gb (1677721600 bytes)
> INFO  [] -     Total Flink Memory: 1.125gb (1207959552 bytes)
> INFO  [] -       JVM Heap:         1024.000mb (1073741824 bytes)
> INFO  [] -       Off-heap:         128.000mb (134217728 bytes)
> INFO  [] -     JVM Metaspace:      256.000mb (268435456 bytes)
> INFO  [] -     JVM Overhead:       192.000mb (201326592 bytes)
> 
> 2022-02-28 12:23:49,013 INFO  
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] - 
> --------------------------------------------------------------------------------
> 2022-02-28 12:23:49,013 INFO  
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] -  Starting 
> KubernetesApplicationClusterEntrypoint (Version: 1.14.2, Scala: 2.11, 
> Rev:6fd4b1c, Date:2021-12-14T23:22:57+01:00)
> 2022-02-28 12:23:49,013 INFO  
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] -  OS current 
> user: flink
> 2022-02-28 12:23:49,014 INFO  
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] -  Current 
> Hadoop/Kerberos user:
> 2022-02-28 12:23:49,014 INFO  
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] -  JVM: 
> OpenJDK 64-Bit Server VM - Oracle Corporation - 1.8/25.312-b07
> 2022-02-28 12:23:49,014 INFO  
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] -  Maximum 
> heap size: 989 MiBytes
> 2022-02-28 12:23:49,014 INFO  
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] -  JAVA_HOME: 
> /usr/local/openjdk-8
> 2022-02-28 12:23:49,015 INFO  
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] -  No Hadoop 
> Dependency available
> 2022-02-28 12:23:49,015 INFO  
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] -  JVM 
> Options:
> 2022-02-28 12:23:49,015 INFO  
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] -     
> -Xmx1073741824
> 2022-02-28 12:23:49,015 INFO  
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] -     
> -Xms1073741824
> 2022-02-28 12:23:49,015 INFO  
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] -     
> -XX:MaxMetaspaceSize=268435456
> 2022-02-28 12:23:49,015 INFO  
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] -     
> -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5006
> 2022-02-28 12:23:49,015 INFO  
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] -     
> -Dlog.file=/opt/flink/log/flink--kubernetes-application-0-fm-hwm8u1o2x8-9cc6947f9-tgxd5.log
> 2022-02-28 12:23:49,016 INFO  
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] -     
> -Dlog4j.configuration=file:/opt/flink/conf/log4j-console.properties
> 2022-02-28 12:23:49,016 INFO  
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] -     
> -Dlog4j.configurationFile=file:/opt/flink/conf/log4j-console.properties
> 2022-02-28 12:23:49,016 INFO  
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] -     
> -Dlogback.configurationFile=file:/opt/flink/conf/logback-console.xml
> 2022-02-28 12:23:49,016 INFO  
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] -  Program 
> Arguments:
> 2022-02-28 12:23:49,017 INFO  
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] -     -D
> 2022-02-28 12:23:49,017 INFO  
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] -     
> jobmanager.memory.off-heap.size=134217728b
> 2022-02-28 12:23:49,018 INFO  
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] -     -D
> 2022-02-28 12:23:49,018 INFO  
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] -     
> jobmanager.memory.jvm-overhead.min=201326592b
> 2022-02-28 12:23:49,018 INFO  
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] -     -D
> 2022-02-28 12:23:49,018 INFO  
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] -     
> jobmanager.memory.jvm-metaspace.size=268435456b
> 2022-02-28 12:23:49,018 INFO  
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] -     -D
> 2022-02-28 12:23:49,018 INFO  
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] -     
> jobmanager.memory.heap.size=1073741824b
> 2022-02-28 12:23:49,019 INFO  
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] -     -D
> 2022-02-28 12:23:49,019 INFO  
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] -     
> jobmanager.memory.jvm-overhead.max=201326592b
> 2022-02-28 12:23:49,019 INFO  
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] -  Classpath: 
> /opt/flink/lib/flink-csv-1.14.2.jar:/opt/flink/lib/flink-json-1.14.2.jar:/opt/flink/lib/flink-shaded-zookeeper-3.4.14.jar:/opt/flink/lib/flink-table_2.11-1.14.2.jar:/opt/flink/lib/log4j-1.2-api-2.16.0.jar:/opt/flink/lib/log4j-api-2.16.0.jar:/opt/flink/lib/log4j-core-2.16.0.jar:/opt/flink/lib/log4j-slf4j-impl-2.16.0.jar:/opt/flink/lib/flink-dist_2.11-1.14.2.jar:::
> 2022-02-28 12:23:49,019 INFO  
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] - 
> --------------------------------------------------------------------------------
> 2022-02-28 12:23:49,020 INFO  
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] - Registered 
> UNIX signal handlers for [TERM, HUP, INT]
> 2022-02-28 12:23:49,037 INFO  
> org.apache.flink.configuration.GlobalConfiguration           [] - Loading 
> configuration property: blob.server.port, 6124
> 2022-02-28 12:23:49,038 INFO  
> org.apache.flink.configuration.GlobalConfiguration           [] - Loading 
> configuration property: env.java.opts.jobmanager, 
> "-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5006"
> 2022-02-28 12:23:49,038 INFO  
> org.apache.flink.configuration.GlobalConfiguration           [] - Loading 
> configuration property: kubernetes.jobmanager.replicas, 2
> 2022-02-28 12:23:49,038 INFO  
> org.apache.flink.configuration.GlobalConfiguration           [] - Loading 
> configuration property: jobmanager.execution.failover-strategy, region
> 2022-02-28 12:23:49,096 INFO  
> org.apache.flink.configuration.GlobalConfiguration           [] - Loading 
> configuration property: high-availability.cluster-id, fm-hwm8u1o2x8
> 2022-02-28 12:23:49,097 INFO  
> org.apache.flink.configuration.GlobalConfiguration           [] - Loading 
> configuration property: jobmanager.rpc.address, localhost
> 2022-02-28 12:23:49,097 INFO  
> org.apache.flink.configuration.GlobalConfiguration           [] - Loading 
> configuration property: -parallelism.default, 10
> 2022-02-28 12:23:49,097 INFO  
> org.apache.flink.configuration.GlobalConfiguration           [] - Loading 
> configuration property: containerized.master.env.ENABLE_BUILT_IN_PLUGINS, 
> flink-s3-fs-hadoop-1.14.2.jar
> 2022-02-28 12:23:49,097 INFO  
> org.apache.flink.configuration.GlobalConfiguration           [] - Loading 
> configuration property: kubernetes.taskmanager.cpu, 2
> 2022-02-28 12:23:49,098 INFO  
> org.apache.flink.configuration.GlobalConfiguration           [] - Loading 
> configuration property: kubernetes.cluster-id, fm-hwm8u1o2x8
> 2022-02-28 12:23:49,098 INFO  
> org.apache.flink.configuration.GlobalConfiguration           [] - Loading 
> configuration property: kubernetes.service-account, flink
> 2022-02-28 12:23:49,098 INFO  
> org.apache.flink.configuration.GlobalConfiguration           [] - Loading 
> configuration property: s3.secret-key, ******
> 2022-02-28 12:23:49,098 INFO  
> org.apache.flink.configuration.GlobalConfiguration           [] - Loading 
> configuration property: high-availability.storageDir, s3://xx.xx/flink-ha
> 2022-02-28 12:23:49,099 INFO  
> org.apache.flink.configuration.GlobalConfiguration           [] - Loading 
> configuration property: s3.endpoint, xx.xx.com
> 2022-02-28 12:23:49,099 INFO  
> org.apache.flink.configuration.GlobalConfiguration           [] - Loading 
> configuration property: $internal.application.program-args, 
> --bootstrapServers;192.168.xx.xx:9092;--topic;yzhoutp01;--groupId;yzhougid022801;--s3path;s3://xx.xx/flink-ha;--flink.conf;;--app.name;HA消费作业;--conf;'json://{"$internal.application.main":"com.yzhou.checkpoint.UDFDataConsumerSource"}'
> 2022-02-28 12:23:49,099 INFO  
> org.apache.flink.configuration.GlobalConfiguration           [] - Loading 
> configuration property: kubernetes.container.image, 
> flink:1.14.2-scala_2.11-java8
> 2022-02-28 12:23:49,099 INFO  
> org.apache.flink.configuration.GlobalConfiguration           [] - Loading 
> configuration property: parallelism.default, 10
> 2022-02-28 12:23:49,099 INFO  
> org.apache.flink.configuration.GlobalConfiguration           [] - Loading 
> configuration property: kubernetes.namespace, yzhou
> 2022-02-28 12:23:49,100 INFO  
> org.apache.flink.configuration.GlobalConfiguration           [] - Loading 
> configuration property: taskmanager.numberOfTaskSlots, 4
> 2022-02-28 12:23:49,100 INFO  
> org.apache.flink.configuration.GlobalConfiguration           [] - Loading 
> configuration property: kubernetes.rest-service.exposed.type, NodePort
> 2022-02-28 12:23:49,100 INFO  
> org.apache.flink.configuration.GlobalConfiguration           [] - Loading 
> configuration property: high-availability.jobmanager.port, 6123
> 2022-02-28 12:23:49,100 INFO  
> org.apache.flink.configuration.GlobalConfiguration           [] - Loading 
> configuration property: $internal.application.main, 
> com.yzhou.checkpoint.UDFDataConsumerSource
> 2022-02-28 12:23:49,100 INFO  
> org.apache.flink.configuration.GlobalConfiguration           [] - Loading 
> configuration property: taskmanager.memory.process.size, 1728m
> 2022-02-28 12:23:49,101 INFO  
> org.apache.flink.configuration.GlobalConfiguration           [] - Loading 
> configuration property: kubernetes.internal.jobmanager.entrypoint.class, 
> org.apache.flink.kubernetes.entrypoint.KubernetesApplicationClusterEntrypoint
> 2022-02-28 12:23:49,101 INFO  
> org.apache.flink.configuration.GlobalConfiguration           [] - Loading 
> configuration property: pipeline.name, HA消费作业
> 2022-02-28 12:23:49,101 INFO  
> org.apache.flink.configuration.GlobalConfiguration           [] - Loading 
> configuration property: classloader.resolve-order, parent-first
> 2022-02-28 12:23:49,101 INFO  
> org.apache.flink.configuration.GlobalConfiguration           [] - Loading 
> configuration property: kubernetes.pod-template-file, 
> /Users/xx.xx/Tmp/app/fm-hwm8u1o2x8/pod-template.yaml
> 2022-02-28 12:23:49,101 INFO  
> org.apache.flink.configuration.GlobalConfiguration           [] - Loading 
> configuration property: execution.target, kubernetes-application
> 2022-02-28 12:23:49,102 INFO  
> org.apache.flink.configuration.GlobalConfiguration           [] - Loading 
> configuration property: restart-strategy.fixed-delay.attempts, 10
> 2022-02-28 12:23:49,102 INFO  
> org.apache.flink.configuration.GlobalConfiguration           [] - Loading 
> configuration property: jobmanager.memory.process.size, 1600m
> 2022-02-28 12:23:49,102 INFO  
> org.apache.flink.configuration.GlobalConfiguration           [] - Loading 
> configuration property: jobmanager.rpc.port, 6123
> 2022-02-28 12:23:49,102 INFO  
> org.apache.flink.configuration.GlobalConfiguration           [] - Loading 
> configuration property: taskmanager.rpc.port, 6122
> 2022-02-28 12:23:49,102 INFO  
> org.apache.flink.configuration.GlobalConfiguration           [] - Loading 
> configuration property: s3.access-key, AKIDjCPQBY8JfniPinIbXbfbji1AB4sYNsK0
> 2022-02-28 12:23:49,103 INFO  
> org.apache.flink.configuration.GlobalConfiguration           [] - Loading 
> configuration property: kubernetes.container.image.pull-policy, Always
> 2022-02-28 12:23:49,103 INFO  
> org.apache.flink.configuration.GlobalConfiguration           [] - Loading 
> configuration property: internal.cluster.execution-mode, NORMAL
> 2022-02-28 12:23:49,103 INFO  
> org.apache.flink.configuration.GlobalConfiguration           [] - Loading 
> configuration property: $internal.pipeline.job-id, 
> 72351556fc8d911c4de6e64750fd805a
> 2022-02-28 12:23:49,103 INFO  
> org.apache.flink.configuration.GlobalConfiguration           [] - Loading 
> configuration property: execution.shutdown-on-attached-exit, false
> 2022-02-28 12:23:49,103 INFO  
> org.apache.flink.configuration.GlobalConfiguration           [] - Loading 
> configuration property: high-availability, 
> org.apache.flink.kubernetes.highavailability.KubernetesHaServicesFactory
> 2022-02-28 12:23:49,104 INFO  
> org.apache.flink.configuration.GlobalConfiguration           [] - Loading 
> configuration property: pipeline.jars, local:///opt/flink/usrlib/app.jar
> 2022-02-28 12:23:49,104 INFO  
> org.apache.flink.configuration.GlobalConfiguration           [] - Loading 
> configuration property: restart-strategy, fixed-delay
> 2022-02-28 12:23:49,104 INFO  
> org.apache.flink.configuration.GlobalConfiguration           [] - Loading 
> configuration property: env.java.opts.taskmanager, 
> "-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005"
> 2022-02-28 12:23:49,104 INFO  
> org.apache.flink.configuration.GlobalConfiguration           [] - Loading 
> configuration property: 
> containerized.taskmanager.env.ENABLE_BUILT_IN_PLUGINS, 
> flink-s3-fs-hadoop-1.14.2.jar
> 2022-02-28 12:23:49,104 INFO  
> org.apache.flink.configuration.GlobalConfiguration           [] - Loading 
> configuration property: s3.path.style.access, true
> 2022-02-28 12:23:49,800 INFO  
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] - Starting 
> KubernetesApplicationClusterEntrypoint.
> 2022-02-28 12:23:49,818 INFO  
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] - Install 
> default filesystem.
> 2022-02-28 12:23:49,824 INFO  org.apache.flink.core.fs.FileSystem             
>              [] - Hadoop is not in the classpath/dependencies. The extended 
> set of supported File Systems via Hadoop is not available.
> 2022-02-28 12:23:50,117 INFO  
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] - Install 
> security context.
> 2022-02-28 12:23:50,133 INFO  
> org.apache.flink.runtime.security.modules.HadoopModuleFactory [] - Cannot 
> create Hadoop Security Module because Hadoop cannot be found in the Classpath.
> 2022-02-28 12:23:50,140 INFO  
> org.apache.flink.runtime.security.modules.JaasModule         [] - Jaas file 
> will be created as /tmp/jaas-8100113544211743196.conf.
> 2022-02-28 12:23:50,149 INFO  
> org.apache.flink.runtime.security.contexts.HadoopSecurityContextFactory [] - 
> Cannot install HadoopSecurityContext because Hadoop cannot be found in the 
> Classpath.
> 2022-02-28 12:23:50,151 INFO  
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] - 
> Initializing cluster services.
> 2022-02-28 12:23:51,258 INFO  
> org.apache.flink.runtime.rpc.akka.AkkaRpcServiceUtils        [] - Trying to 
> start actor system, external address 10.168.xx.xx:6123, bind address 
> 0.0.0.0:6123.
> 2022-02-28 12:23:53,108 INFO  akka.event.slf4j.Slf4jLogger                    
>              [] - Slf4jLogger started
> 2022-02-28 12:23:53,215 INFO  akka.remote.RemoteActorRefProvider              
>              [] - Akka Cluster not in use - enabling unsafe features anyway 
> because `akka.remote.use-unsafe-remote-features-outside-cluster` has been 
> enabled.
> 2022-02-28 12:23:53,216 INFO  akka.remote.Remoting                            
>              [] - Starting remoting
> 2022-02-28 12:23:53,609 INFO  akka.remote.Remoting                            
>              [] - Remoting started; listening on addresses 
> :[akka.tcp://fl...@10.168.xx.xx:6123]
> 2022-02-28 12:23:54,063 INFO  
> org.apache.flink.runtime.rpc.akka.AkkaRpcServiceUtils        [] - Actor 
> system started at akka.tcp://fl...@10.168.xx.xx:6123
> 2022-02-28 12:23:55,630 WARN  org.apache.hadoop.metrics2.impl.MetricsConfig   
>              [] - Cannot locate configuration: tried 
> hadoop-metrics2-s3a-file-system.properties,hadoop-metrics2.properties
> 2022-02-28 12:23:55,718 INFO  
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl            [] - Scheduled 
> Metric snapshot period at 10 second(s).
> 2022-02-28 12:23:55,719 INFO  
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl            [] - 
> s3a-file-system metrics system started
> 2022-02-28 12:23:55,909 WARN  org.apache.hadoop.util.NativeCodeLoader         
>              [] - Unable to load native-hadoop library for your platform... 
> using builtin-java classes where applicable
> 2022-02-28 12:23:58,602 INFO  
> org.apache.flink.runtime.blob.FileSystemBlobStore            [] - Creating 
> highly available BLOB storage directory at 
> s3://xx.xx/flink-ha/fm-hwm8u1o2x8/blob
> 2022-02-28 12:24:01,010 INFO  org.apache.flink.runtime.blob.BlobServer        
>              [] - Created BLOB server storage directory 
> /tmp/blobStore-39b216f8-1ad6-4e24-9132-2166dfa40ce0
> 2022-02-28 12:24:01,018 INFO  org.apache.flink.runtime.blob.BlobServer        
>              [] - Started BLOB server at 0.0.0.0:6124 - max concurrent 
> requests: 50 - max backlog: 1000
> 2022-02-28 12:24:01,107 INFO  
> org.apache.flink.runtime.metrics.MetricRegistryImpl          [] - No metrics 
> reporter configured, no metrics will be exposed/reported.
> 2022-02-28 12:24:01,114 INFO  
> org.apache.flink.runtime.rpc.akka.AkkaRpcServiceUtils        [] - Trying to 
> start actor system, external address 10.168.xx.xx:0, bind address 0.0.0.0:0.
> 2022-02-28 12:24:01,223 INFO  akka.event.slf4j.Slf4jLogger                    
>              [] - Slf4jLogger started
> 2022-02-28 12:24:01,297 INFO  akka.remote.RemoteActorRefProvider              
>              [] - Akka Cluster not in use - enabling unsafe features anyway 
> because `akka.remote.use-unsafe-remote-features-outside-cluster` has been 
> enabled.
> 2022-02-28 12:24:01,297 INFO  akka.remote.Remoting                            
>              [] - Starting remoting
> 2022-02-28 12:24:01,321 INFO  akka.remote.Remoting                            
>              [] - Remoting started; listening on addresses 
> :[akka.tcp://flink-metr...@10.168.xx.xx:38165]
> 2022-02-28 12:24:01,407 INFO  
> org.apache.flink.runtime.rpc.akka.AkkaRpcServiceUtils        [] - Actor 
> system started at akka.tcp://flink-metr...@10.168.xx.xx:38165
> 2022-02-28 12:24:01,430 INFO  
> org.apache.flink.runtime.rpc.akka.AkkaRpcService             [] - Starting 
> RPC endpoint for org.apache.flink.runtime.metrics.dump.MetricQueryService at 
> akka://flink-metrics/user/rpc/MetricQueryService .
> 2022-02-28 12:24:01,639 INFO  
> org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint [] - Upload 
> directory 
> /tmp/flink-web-813c1c10-1aeb-4c95-b010-4ecdb6fb3829/flink-web-upload does not 
> exist.
> 2022-02-28 12:24:01,697 INFO  
> org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint [] - Created 
> directory 
> /tmp/flink-web-813c1c10-1aeb-4c95-b010-4ecdb6fb3829/flink-web-upload for file 
> uploads.
> 2022-02-28 12:24:01,731 INFO  
> org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint [] - Starting 
> rest endpoint.
> 2022-02-28 12:24:02,028 INFO  
> org.apache.flink.runtime.webmonitor.WebMonitorUtils          [] - Determined 
> location of main cluster component log file: 
> /opt/flink/log/flink--kubernetes-application-0-fm-hwm8u1o2x8-9cc6947f9-tgxd5.log
> 2022-02-28 12:24:02,028 INFO  
> org.apache.flink.runtime.webmonitor.WebMonitorUtils          [] - Determined 
> location of main cluster component stdout file: 
> /opt/flink/log/flink--kubernetes-application-0-fm-hwm8u1o2x8-9cc6947f9-tgxd5.out
> 2022-02-28 12:24:02,608 INFO  
> org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint [] - Rest 
> endpoint listening at 10.168.xx.xx:8081
> 2022-02-28 12:24:02,620 INFO  
> org.apache.flink.kubernetes.kubeclient.resources.KubernetesLeaderElector [] - 
> Create KubernetesLeaderElector fm-hwm8u1o2x8-restserver-leader with lock 
> identity a9d2cf8d-1a03-437d-8215-20b08ad4813f.
> 2022-02-28 12:24:02,624 INFO  
> org.apache.flink.runtime.leaderelection.DefaultLeaderElectionService [] - 
> Starting DefaultLeaderElectionService with 
> KubernetesLeaderElectionDriver{configMapName='fm-hwm8u1o2x8-restserver-leader'}.
> 2022-02-28 12:24:02,624 INFO  
> org.apache.flink.kubernetes.kubeclient.resources.KubernetesConfigMapSharedInformer
>  [] - Starting to watch for yzhou/fm-hwm8u1o2x8-restserver-leader, watching 
> id:25697182-adb4-409e-bcb5-11c172b02bcd
> 2022-02-28 12:24:02,625 INFO  
> org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint [] - Web 
> frontend listening at http://10.168.xx.xx:8081.
> 2022-02-28 12:24:02,718 INFO  
> org.apache.flink.runtime.util.config.memory.ProcessMemoryUtils [] - The 
> derived from fraction jvm overhead memory (172.800mb (181193935 bytes)) is 
> less than its min value 192.000mb (201326592 bytes), min value will be used 
> instead
> 2022-02-28 12:24:02,908 INFO  
> org.apache.flink.kubernetes.kubeclient.resources.KubernetesLeaderElector [] - 
> Create KubernetesLeaderElector fm-hwm8u1o2x8-dispatcher-leader with lock 
> identity a9d2cf8d-1a03-437d-8215-20b08ad4813f.
> 2022-02-28 12:24:02,909 INFO  
> org.apache.flink.runtime.leaderelection.DefaultLeaderElectionService [] - 
> Starting DefaultLeaderElectionService with 
> KubernetesLeaderElectionDriver{configMapName='fm-hwm8u1o2x8-dispatcher-leader'}.
> 2022-02-28 12:24:02,909 INFO  
> org.apache.flink.runtime.resourcemanager.ResourceManagerServiceImpl [] - 
> Starting resource manager service.
> 2022-02-28 12:24:02,909 INFO  
> org.apache.flink.kubernetes.kubeclient.resources.KubernetesConfigMapSharedInformer
>  [] - Starting to watch for yzhou/fm-hwm8u1o2x8-dispatcher-leader, watching 
> id:2c14bb82-cd22-4c7b-8a4f-838e37efee81
> 2022-02-28 12:24:02,910 INFO  
> org.apache.flink.kubernetes.kubeclient.resources.KubernetesLeaderElector [] - 
> Create KubernetesLeaderElector fm-hwm8u1o2x8-resourcemanager-leader with lock 
> identity a9d2cf8d-1a03-437d-8215-20b08ad4813f.
> 2022-02-28 12:24:02,910 INFO  
> org.apache.flink.runtime.leaderelection.DefaultLeaderElectionService [] - 
> Starting DefaultLeaderElectionService with 
> KubernetesLeaderElectionDriver{configMapName='fm-hwm8u1o2x8-resourcemanager-leader'}.
> 2022-02-28 12:24:02,910 INFO  
> org.apache.flink.kubernetes.kubeclient.resources.KubernetesConfigMapSharedInformer
>  [] - Starting to watch for yzhou/fm-hwm8u1o2x8-resourcemanager-leader, 
> watching id:cd6a0293-f546-4510-aa68-2481bf4f69b9
> 2022-02-28 12:24:02,913 INFO  
> org.apache.flink.runtime.leaderretrieval.DefaultLeaderRetrievalService [] - 
> Starting DefaultLeaderRetrievalService with 
> KubernetesLeaderRetrievalDriver{configMapName='fm-hwm8u1o2x8-resourcemanager-leader'}.
> 2022-02-28 12:24:02,913 INFO  
> org.apache.flink.kubernetes.kubeclient.resources.KubernetesConfigMapSharedInformer
>  [] - Starting to watch for yzhou/fm-hwm8u1o2x8-resourcemanager-leader, 
> watching id:994bd8c2-fe3a-436c-8116-83ce42656c87
> 2022-02-28 12:24:02,913 INFO  
> org.apache.flink.runtime.leaderretrieval.DefaultLeaderRetrievalService [] - 
> Starting DefaultLeaderRetrievalService with 
> KubernetesLeaderRetrievalDriver{configMapName='fm-hwm8u1o2x8-dispatcher-leader'}.
> 2022-02-28 12:24:02,913 INFO  
> org.apache.flink.kubernetes.kubeclient.resources.KubernetesConfigMapSharedInformer
>  [] - Starting to watch for yzhou/fm-hwm8u1o2x8-dispatcher-leader, watching 
> id:5071b799-e48b-4705-9e30-5f1b190b867d
> 2022-02-28 12:24:03,109 INFO  
> org.apache.flink.kubernetes.kubeclient.resources.KubernetesLeaderElector [] - 
> New leader elected b2035865-bcec-4803-9a6c-7267d1b1ba54 for 
> fm-hwm8u1o2x8-dispatcher-leader.
> 2022-02-28 12:24:03,109 INFO  
> org.apache.flink.kubernetes.kubeclient.resources.KubernetesLeaderElector [] - 
> New leader elected b2035865-bcec-4803-9a6c-7267d1b1ba54 for 
> fm-hwm8u1o2x8-restserver-leader.
> 2022-02-28 12:24:03,207 INFO  
> org.apache.flink.kubernetes.kubeclient.resources.KubernetesLeaderElector [] - 
> New leader elected b2035865-bcec-4803-9a6c-7267d1b1ba54 for 
> fm-hwm8u1o2x8-resourcemanager-leader.
> 2022-02-28 12:26:02,178 ERROR 
> org.apache.flink.runtime.rest.handler.job.JobExceptionsHandler [] - Exception 
> occurred in REST handler: Job b2c4f20005144514a7995a33b8fe03f3 not found
> 2022-02-28 12:27:34,854 WARN  
> io.fabric8.kubernetes.client.dsl.internal.WatcherWebSocketListener [] - Exec 
> Failure java.net.SocketTimeoutException sent ping but didn't receive pong 
> within 30000ms (after 5 successful ping/pongs)
> 2022-02-28 12:27:15,149 ERROR 
> org.apache.flink.runtime.rest.handler.job.checkpoints.CheckpointingStatisticsHandler
>  [] - Unhandled exception.
> java.util.concurrent.CancellationException: null
> at java.util.concurrent.CompletableFuture.cancel(CompletableFuture.java:2276) 
> ~[?:1.8.0_312]
> at 
> org.apache.flink.runtime.rest.handler.legacy.DefaultExecutionGraphCache.getExecutionGraphInternal(DefaultExecutionGraphCache.java:98)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.legacy.DefaultExecutionGraphCache.getExecutionGraphInfo(DefaultExecutionGraphCache.java:67)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.handleRequest(AbstractExecutionGraphHandler.java:81)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.AbstractRestHandler.respondToRequest(AbstractRestHandler.java:83)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.AbstractHandler.respondAsLeader(AbstractHandler.java:195)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.lambda$channelRead0$0(LeaderRetrievalHandler.java:83)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at java.util.Optional.ifPresent(Optional.java:159) [?:1.8.0_312]
> at org.apache.flink.util.OptionalConsumer.ifPresent(OptionalConsumer.java:45) 
> [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.channelRead0(LeaderRetrievalHandler.java:80)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.channelRead0(LeaderRetrievalHandler.java:49)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.router.RouterHandler.routed(RouterHandler.java:115)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.router.RouterHandler.channelRead0(RouterHandler.java:94)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.router.RouterHandler.channelRead0(RouterHandler.java:55)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.FileUploadHandler.channelRead0(FileUploadHandler.java:238)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.FileUploadHandler.channelRead0(FileUploadHandler.java:71)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:719)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:655)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:581)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_312]
> 2022-02-28 12:35:11,614 ERROR 
> org.apache.flink.runtime.rest.handler.job.metrics.JobVertexWatermarksHandler 
> [] - Unhandled exception.
> java.util.concurrent.CancellationException: null
> at java.util.concurrent.CompletableFuture.cancel(CompletableFuture.java:2276) 
> ~[?:1.8.0_312]
> at 
> org.apache.flink.runtime.rest.handler.legacy.DefaultExecutionGraphCache.getExecutionGraphInternal(DefaultExecutionGraphCache.java:98)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.legacy.DefaultExecutionGraphCache.getExecutionGraphInfo(DefaultExecutionGraphCache.java:67)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.handleRequest(AbstractExecutionGraphHandler.java:81)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.AbstractRestHandler.respondToRequest(AbstractRestHandler.java:83)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.AbstractHandler.respondAsLeader(AbstractHandler.java:195)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.lambda$channelRead0$0(LeaderRetrievalHandler.java:83)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at java.util.Optional.ifPresent(Optional.java:159) [?:1.8.0_312]
> at org.apache.flink.util.OptionalConsumer.ifPresent(OptionalConsumer.java:45) 
> [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.channelRead0(LeaderRetrievalHandler.java:80)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.channelRead0(LeaderRetrievalHandler.java:49)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.router.RouterHandler.routed(RouterHandler.java:115)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.router.RouterHandler.channelRead0(RouterHandler.java:94)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.router.RouterHandler.channelRead0(RouterHandler.java:55)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.FileUploadHandler.channelRead0(FileUploadHandler.java:238)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.FileUploadHandler.channelRead0(FileUploadHandler.java:71)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:719)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:655)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:581)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_312]
> 2022-02-28 12:35:46,286 ERROR 
> org.apache.flink.runtime.rest.handler.job.JobDetailsHandler  [] - Unhandled 
> exception.
> java.util.concurrent.CancellationException: null
> at java.util.concurrent.CompletableFuture.cancel(CompletableFuture.java:2276) 
> ~[?:1.8.0_312]
> at 
> org.apache.flink.runtime.rest.handler.legacy.DefaultExecutionGraphCache.getExecutionGraphInternal(DefaultExecutionGraphCache.java:98)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.legacy.DefaultExecutionGraphCache.getExecutionGraphInfo(DefaultExecutionGraphCache.java:67)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.handleRequest(AbstractExecutionGraphHandler.java:81)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.AbstractRestHandler.respondToRequest(AbstractRestHandler.java:83)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.AbstractHandler.respondAsLeader(AbstractHandler.java:195)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.lambda$channelRead0$0(LeaderRetrievalHandler.java:83)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at java.util.Optional.ifPresent(Optional.java:159) [?:1.8.0_312]
> at org.apache.flink.util.OptionalConsumer.ifPresent(OptionalConsumer.java:45) 
> [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.channelRead0(LeaderRetrievalHandler.java:80)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.channelRead0(LeaderRetrievalHandler.java:49)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.router.RouterHandler.routed(RouterHandler.java:115)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.router.RouterHandler.channelRead0(RouterHandler.java:94)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.router.RouterHandler.channelRead0(RouterHandler.java:55)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.FileUploadHandler.channelRead0(FileUploadHandler.java:238)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.FileUploadHandler.channelRead0(FileUploadHandler.java:71)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:719)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:655)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:581)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_312]
> 2022-02-28 12:35:47,305 ERROR 
> org.apache.flink.runtime.rest.handler.job.JobDetailsHandler  [] - Unhandled 
> exception.
> java.util.concurrent.CancellationException: null
> at java.util.concurrent.CompletableFuture.cancel(CompletableFuture.java:2276) 
> ~[?:1.8.0_312]
> at 
> org.apache.flink.runtime.rest.handler.legacy.DefaultExecutionGraphCache.getExecutionGraphInternal(DefaultExecutionGraphCache.java:98)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.legacy.DefaultExecutionGraphCache.getExecutionGraphInfo(DefaultExecutionGraphCache.java:67)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.handleRequest(AbstractExecutionGraphHandler.java:81)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.AbstractRestHandler.respondToRequest(AbstractRestHandler.java:83)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.AbstractHandler.respondAsLeader(AbstractHandler.java:195)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.lambda$channelRead0$0(LeaderRetrievalHandler.java:83)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at java.util.Optional.ifPresent(Optional.java:159) [?:1.8.0_312]
> at org.apache.flink.util.OptionalConsumer.ifPresent(OptionalConsumer.java:45) 
> [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.channelRead0(LeaderRetrievalHandler.java:80)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.channelRead0(LeaderRetrievalHandler.java:49)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.router.RouterHandler.routed(RouterHandler.java:115)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.router.RouterHandler.channelRead0(RouterHandler.java:94)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.router.RouterHandler.channelRead0(RouterHandler.java:55)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.FileUploadHandler.channelRead0(FileUploadHandler.java:238)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.FileUploadHandler.channelRead0(FileUploadHandler.java:71)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:719)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:655)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:581)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_312]
> 2022-02-28 12:42:13,537 ERROR 
> org.apache.flink.runtime.rest.handler.job.JobDetailsHandler  [] - Unhandled 
> exception.
> java.util.concurrent.CancellationException: null
> at java.util.concurrent.CompletableFuture.cancel(CompletableFuture.java:2276) 
> ~[?:1.8.0_312]
> at 
> org.apache.flink.runtime.rest.handler.legacy.DefaultExecutionGraphCache.getExecutionGraphInternal(DefaultExecutionGraphCache.java:98)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.legacy.DefaultExecutionGraphCache.getExecutionGraphInfo(DefaultExecutionGraphCache.java:67)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.handleRequest(AbstractExecutionGraphHandler.java:81)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.AbstractRestHandler.respondToRequest(AbstractRestHandler.java:83)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.AbstractHandler.respondAsLeader(AbstractHandler.java:195)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.lambda$channelRead0$0(LeaderRetrievalHandler.java:83)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at java.util.Optional.ifPresent(Optional.java:159) [?:1.8.0_312]
> at org.apache.flink.util.OptionalConsumer.ifPresent(OptionalConsumer.java:45) 
> [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.channelRead0(LeaderRetrievalHandler.java:80)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.channelRead0(LeaderRetrievalHandler.java:49)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.router.RouterHandler.routed(RouterHandler.java:115)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.router.RouterHandler.channelRead0(RouterHandler.java:94)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.router.RouterHandler.channelRead0(RouterHandler.java:55)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.FileUploadHandler.channelRead0(FileUploadHandler.java:238)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.FileUploadHandler.channelRead0(FileUploadHandler.java:71)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:719)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:655)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:581)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_312]
> 2022-02-28 12:42:17,268 ERROR 
> org.apache.flink.runtime.rest.handler.job.JobDetailsHandler  [] - Unhandled 
> exception.
> java.util.concurrent.CancellationException: null
> at java.util.concurrent.CompletableFuture.cancel(CompletableFuture.java:2276) 
> ~[?:1.8.0_312]
> at 
> org.apache.flink.runtime.rest.handler.legacy.DefaultExecutionGraphCache.getExecutionGraphInternal(DefaultExecutionGraphCache.java:98)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.legacy.DefaultExecutionGraphCache.getExecutionGraphInfo(DefaultExecutionGraphCache.java:67)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.handleRequest(AbstractExecutionGraphHandler.java:81)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.AbstractRestHandler.respondToRequest(AbstractRestHandler.java:83)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.AbstractHandler.respondAsLeader(AbstractHandler.java:195)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.lambda$channelRead0$0(LeaderRetrievalHandler.java:83)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at java.util.Optional.ifPresent(Optional.java:159) [?:1.8.0_312]
> at org.apache.flink.util.OptionalConsumer.ifPresent(OptionalConsumer.java:45) 
> [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.channelRead0(LeaderRetrievalHandler.java:80)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.channelRead0(LeaderRetrievalHandler.java:49)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.router.RouterHandler.routed(RouterHandler.java:115)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.router.RouterHandler.channelRead0(RouterHandler.java:94)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.router.RouterHandler.channelRead0(RouterHandler.java:55)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.FileUploadHandler.channelRead0(FileUploadHandler.java:238)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.FileUploadHandler.channelRead0(FileUploadHandler.java:71)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:719)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:655)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:581)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_312]
> 2022-02-28 12:42:21,958 ERROR 
> org.apache.flink.runtime.rest.handler.job.JobDetailsHandler  [] - Unhandled 
> exception.
> java.util.concurrent.CancellationException: null
> at java.util.concurrent.CompletableFuture.cancel(CompletableFuture.java:2276) 
> ~[?:1.8.0_312]
> at 
> org.apache.flink.runtime.rest.handler.legacy.DefaultExecutionGraphCache.getExecutionGraphInternal(DefaultExecutionGraphCache.java:98)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.legacy.DefaultExecutionGraphCache.getExecutionGraphInfo(DefaultExecutionGraphCache.java:67)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.handleRequest(AbstractExecutionGraphHandler.java:81)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.AbstractRestHandler.respondToRequest(AbstractRestHandler.java:83)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.AbstractHandler.respondAsLeader(AbstractHandler.java:195)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.lambda$channelRead0$0(LeaderRetrievalHandler.java:83)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at java.util.Optional.ifPresent(Optional.java:159) [?:1.8.0_312]
> at org.apache.flink.util.OptionalConsumer.ifPresent(OptionalConsumer.java:45) 
> [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.channelRead0(LeaderRetrievalHandler.java:80)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.channelRead0(LeaderRetrievalHandler.java:49)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.router.RouterHandler.routed(RouterHandler.java:115)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.router.RouterHandler.channelRead0(RouterHandler.java:94)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.router.RouterHandler.channelRead0(RouterHandler.java:55)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.FileUploadHandler.channelRead0(FileUploadHandler.java:238)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.FileUploadHandler.channelRead0(FileUploadHandler.java:71)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:719)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:655)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:581)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_312]
> 2022-02-28 12:43:55,653 ERROR 
> io.fabric8.kubernetes.client.extended.leaderelection.LeaderElector [] - 
> Exception occurred while acquiring lock 'ConfigMapLock: yzhou - 
> fm-hwm8u1o2x8-dispatcher-leader (a9d2cf8d-1a03-437d-8215-20b08ad4813f)'
> io.fabric8.kubernetes.client.KubernetesClientException: Operation: [get]  for 
> kind: [ConfigMap]  with name: [fm-hwm8u1o2x8-dispatcher-leader]  in 
> namespace: [yzhou]  failed.
> at 
> io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:64)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:72)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> io.fabric8.kubernetes.client.dsl.base.BaseOperation.getMandatory(BaseOperation.java:205)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> io.fabric8.kubernetes.client.dsl.base.BaseOperation.get(BaseOperation.java:167)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> io.fabric8.kubernetes.client.dsl.base.BaseOperation.get(BaseOperation.java:90)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> io.fabric8.kubernetes.client.extended.leaderelection.resourcelock.ConfigMapLock.get(ConfigMapLock.java:55)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> io.fabric8.kubernetes.client.extended.leaderelection.LeaderElector.tryAcquireOrRenew(LeaderElector.java:134)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> io.fabric8.kubernetes.client.extended.leaderelection.LeaderElector.lambda$acquire$0(LeaderElector.java:82)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> io.fabric8.kubernetes.client.extended.leaderelection.LeaderElector.lambda$loop$3(LeaderElector.java:198)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [?:1.8.0_312]
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) 
> [?:1.8.0_312]
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
>  [?:1.8.0_312]
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
>  [?:1.8.0_312]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>  [?:1.8.0_312]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>  [?:1.8.0_312]
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_312]
> Caused by: java.net.SocketTimeoutException: timeout
> at 
> org.apache.flink.kubernetes.shaded.okio.Okio$4.newTimeoutException(Okio.java:232)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.kubernetes.shaded.okio.AsyncTimeout.exit(AsyncTimeout.java:276)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.kubernetes.shaded.okio.AsyncTimeout$2.read(AsyncTimeout.java:243)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.kubernetes.shaded.okio.RealBufferedSource.indexOf(RealBufferedSource.java:358)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.kubernetes.shaded.okio.RealBufferedSource.readUtf8LineStrict(RealBufferedSource.java:230)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.kubernetes.shaded.okhttp3.internal.http1.Http1ExchangeCodec.readHeaderLine(Http1ExchangeCodec.java:242)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.kubernetes.shaded.okhttp3.internal.http1.Http1ExchangeCodec.readResponseHeaders(Http1ExchangeCodec.java:213)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.kubernetes.shaded.okhttp3.internal.connection.Exchange.readResponseHeaders(Exchange.java:115)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.kubernetes.shaded.okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.java:94)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.kubernetes.shaded.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.kubernetes.shaded.okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:43)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.kubernetes.shaded.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.kubernetes.shaded.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.kubernetes.shaded.okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:94)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.kubernetes.shaded.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.kubernetes.shaded.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.kubernetes.shaded.okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.kubernetes.shaded.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.kubernetes.shaded.okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:88)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.kubernetes.shaded.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.kubernetes.shaded.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> io.fabric8.kubernetes.client.utils.BackwardsCompatibilityInterceptor.intercept(BackwardsCompatibilityInterceptor.java:133)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.kubernetes.shaded.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.kubernetes.shaded.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> io.fabric8.kubernetes.client.utils.TokenRefreshInterceptor.intercept(TokenRefreshInterceptor.java:42)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.kubernetes.shaded.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.kubernetes.shaded.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> io.fabric8.kubernetes.client.utils.ImpersonatorInterceptor.intercept(ImpersonatorInterceptor.java:68)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.kubernetes.shaded.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.kubernetes.shaded.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> io.fabric8.kubernetes.client.utils.HttpClientUtils.lambda$createApplicableInterceptors$6(HttpClientUtils.java:290)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.kubernetes.shaded.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.kubernetes.shaded.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.kubernetes.shaded.okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:229)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.kubernetes.shaded.okhttp3.RealCall.execute(RealCall.java:81) 
> ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> io.fabric8.kubernetes.client.dsl.base.OperationSupport.retryWithExponentialBackoff(OperationSupport.java:585)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:558)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:521)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleGet(OperationSupport.java:488)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleGet(OperationSupport.java:470)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> io.fabric8.kubernetes.client.dsl.base.BaseOperation.handleGet(BaseOperation.java:830)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> io.fabric8.kubernetes.client.dsl.base.BaseOperation.getMandatory(BaseOperation.java:200)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> ... 13 more
> 2022-02-28 12:47:30,763 WARN  
> io.fabric8.kubernetes.client.dsl.internal.WatcherWebSocketListener [] - Exec 
> Failure java.net.SocketTimeoutException sent ping but didn't receive pong 
> within 30000ms (after 32 successful ping/pongs)
> 2022-02-28 12:47:30,763 ERROR 
> org.apache.flink.runtime.rest.handler.job.JobDetailsHandler  [] - Unhandled 
> exception.
> java.util.concurrent.CancellationException: null
> at java.util.concurrent.CompletableFuture.cancel(CompletableFuture.java:2276) 
> ~[?:1.8.0_312]
> at 
> org.apache.flink.runtime.rest.handler.legacy.DefaultExecutionGraphCache.getExecutionGraphInternal(DefaultExecutionGraphCache.java:98)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.legacy.DefaultExecutionGraphCache.getExecutionGraphInfo(DefaultExecutionGraphCache.java:67)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.handleRequest(AbstractExecutionGraphHandler.java:81)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.AbstractRestHandler.respondToRequest(AbstractRestHandler.java:83)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.AbstractHandler.respondAsLeader(AbstractHandler.java:195)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.lambda$channelRead0$0(LeaderRetrievalHandler.java:83)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at java.util.Optional.ifPresent(Optional.java:159) [?:1.8.0_312]
> at org.apache.flink.util.OptionalConsumer.ifPresent(OptionalConsumer.java:45) 
> [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.channelRead0(LeaderRetrievalHandler.java:80)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.channelRead0(LeaderRetrievalHandler.java:49)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.router.RouterHandler.routed(RouterHandler.java:115)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.router.RouterHandler.channelRead0(RouterHandler.java:94)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.router.RouterHandler.channelRead0(RouterHandler.java:55)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.FileUploadHandler.channelRead0(FileUploadHandler.java:238)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.FileUploadHandler.channelRead0(FileUploadHandler.java:71)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:719)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:655)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:581)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_312]
> 2022-02-28 12:47:34,206 ERROR 
> io.fabric8.kubernetes.client.extended.leaderelection.LeaderElector [] - 
> Exception occurred while acquiring lock 'ConfigMapLock: yzhou - 
> fm-hwm8u1o2x8-resourcemanager-leader (a9d2cf8d-1a03-437d-8215-20b08ad4813f)'
> io.fabric8.kubernetes.client.extended.leaderelection.resourcelock.LockException:
>  Unable to update ConfigMapLock
> at 
> io.fabric8.kubernetes.client.extended.leaderelection.resourcelock.ConfigMapLock.update(ConfigMapLock.java:109)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> io.fabric8.kubernetes.client.extended.leaderelection.LeaderElector.tryAcquireOrRenew(LeaderElector.java:156)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> io.fabric8.kubernetes.client.extended.leaderelection.LeaderElector.lambda$acquire$0(LeaderElector.java:82)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> io.fabric8.kubernetes.client.extended.leaderelection.LeaderElector.lambda$loop$3(LeaderElector.java:198)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [?:1.8.0_312]
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) 
> [?:1.8.0_312]
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
>  [?:1.8.0_312]
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
>  [?:1.8.0_312]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>  [?:1.8.0_312]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>  [?:1.8.0_312]
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_312]
> Caused by: io.fabric8.kubernetes.client.KubernetesClientException: Failure 
> executing: PUT at: 
> https://10.96.0.1/api/v1/namespaces/yzhou/configmaps/fm-hwm8u1o2x8-resourcemanager-leader.
>  Message: Operation cannot be fulfilled on configmaps 
> "fm-hwm8u1o2x8-resourcemanager-leader": the object has been modified; please 
> apply your changes to the latest version and try again. Received status: 
> Status(apiVersion=v1, code=409, details=StatusDetails(causes=[], group=null, 
> kind=configmaps, name=fm-hwm8u1o2x8-resourcemanager-leader, 
> retryAfterSeconds=null, uid=null, additionalProperties={}), kind=Status, 
> message=Operation cannot be fulfilled on configmaps 
> "fm-hwm8u1o2x8-resourcemanager-leader": the object has been modified; please 
> apply your changes to the latest version and try again, 
> metadata=ListMeta(_continue=null, remainingItemCount=null, 
> resourceVersion=null, selfLink=null, additionalProperties={}), 
> reason=Conflict, status=Failure, additionalProperties={}).
> at 
> io.fabric8.kubernetes.client.dsl.base.OperationSupport.requestFailure(OperationSupport.java:673)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> io.fabric8.kubernetes.client.dsl.base.OperationSupport.assertResponseCode(OperationSupport.java:612)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:560)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:521)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleUpdate(OperationSupport.java:347)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleUpdate(OperationSupport.java:327)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> io.fabric8.kubernetes.client.dsl.base.BaseOperation.handleUpdate(BaseOperation.java:781)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.lambda$replace$1(HasMetadataOperation.java:183)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.replace(HasMetadataOperation.java:188)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.replace(HasMetadataOperation.java:130)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.replace(HasMetadataOperation.java:41)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> io.fabric8.kubernetes.client.extended.leaderelection.resourcelock.ConfigMapLock.update(ConfigMapLock.java:107)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> ... 10 more
> 
> 
> 
> 
> 
> | |
> xinzhuxiansheng
> |
> |
> xinzhuxiansh...@126.com
> |
> 签名由网易邮箱大师定制
> 
> 
> 在2022年02月28日 20:49,胡伟华<huweihua....@gmail.com> 写道:
> Hi, xinzhuxiansheng
> 
> 这个异常日志看起来不是根本原因,方便提供下全量 JobManager 日志吗?
> 
> 
> 2022年2月28日 下午7:50,xinzhuxiansheng <xinzhuxiansh...@126.com> 写道:
> 
> 我的环境:
> Flink on Native K8s Application mode,并且用S3协议来做HA部署, 部署后,作业正常工作,可Flink UI 
> 无法正常打开 作业的拓扑图,页面会提示内部错误
> 
> 
> jobmanager会提示很多 类似的异常:
> 2022-02-28 08:46:21,171 ERROR 
> org.apache.flink.runtime.rest.handler.job.JobExceptionsHandler [] - Unhandled 
> exception.
> java.util.concurrent.CancellationException: null
> at java.util.concurrent.CompletableFuture.cancel(CompletableFuture.java:2276) 
> ~[?:1.8.0_312]
> at 
> org.apache.flink.runtime.rest.handler.legacy.DefaultExecutionGraphCache.getExecutionGraphInternal(DefaultExecutionGraphCache.java:98)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.legacy.DefaultExecutionGraphCache.getExecutionGraphInfo(DefaultExecutionGraphCache.java:67)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.handleRequest(AbstractExecutionGraphHandler.java:81)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.AbstractRestHandler.respondToRequest(AbstractRestHandler.java:83)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.AbstractHandler.respondAsLeader(AbstractHandler.java:195)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.lambda$channelRead0$0(LeaderRetrievalHandler.java:83)
>  ~[flink-dist_2.11-1.14.2.jar:1.14.2]
> at java.util.Optional.ifPresent(Optional.java:159) [?:1.8.0_312]
> at org.apache.flink.util.OptionalConsumer.ifPresent(OptionalConsumer.java:45) 
> [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.channelRead0(LeaderRetrievalHandler.java:80)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.channelRead0(LeaderRetrievalHandler.java:49)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.router.RouterHandler.routed(RouterHandler.java:115)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.router.RouterHandler.channelRead0(RouterHandler.java:94)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.handler.router.RouterHandler.channelRead0(RouterHandler.java:55)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.FileUploadHandler.channelRead0(FileUploadHandler.java:238)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.runtime.rest.FileUploadHandler.channelRead0(FileUploadHandler.java:71)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> at 
> org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
>  [flink-dist_2.11-1.14.2.jar:1.14.2]
> 
> 
> 有人知道这是咋回事么 ?
> 
> 
> | |
> xinzhuxiansheng
> |
> |
> xinzhuxiansh...@126.com
> |
> 签名由网易邮箱大师定制
> 
> 

回复