hello, here's some of full GC log:
OpenJDK 64-Bit Server VM (25.232-b09) for linux-amd64 JRE (1.8.0_232-b09), built on Oct 18 2019 15:04:46 by "jenkins" with gcc 4.8.2 20140120 (Red Hat 4.8.2-15) Memory: 4k page, physical 976560k(946672k free), swap 0k(0k free) CommandLine flags: -XX:CompressedClassSpaceSize=260046848 -XX:InitialHeapSize=1073741824 -XX:MaxHeapSize=1073741824 -XX:MaxMetaspaceSize=268435456 -XX:+PrintGC -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+UseCompressedClassPointers -XX:+UseCompressedOops 2021-09-13T09:28:11.569+0200: 3.516: [Full GC (Metadata GC Threshold) 2021-09-13T09:28:11.569+0200: 3.516: [Tenured: 0K->12699K(699072K), 0.0986073 secs] 67116K->12699K(1013632K), [Metaspace: 20705K->20705K(1067008K)], 0.0987201 secs] [Times: user=0.03 sys=0.02, real=0.10 secs] 2021-09-13T09:28:15.560+0200: 7.507: [Full GC (Metadata GC Threshold) 2021-09-13T09:28:15.560+0200: 7.507: [Tenured: 12699K->24229K(699072K), 0.2937536 secs] 105133K->24229K(1013632K), [Metaspace: 33805K->33805K(1079296K)], 0.2938554 secs] [Times: user=0.13 sys=0.00, real=0.29 secs] 2021-09-13T09:28:22.744+0200: 14.691: [Full GC (Metadata GC Threshold) 2021-09-13T09:28:22.744+0200: 14.691: [Tenured: 24229K->50182K(699072K), 0.2362689 secs] 187184K->50182K(1013632K), [Metaspace: 56762K->56762K(1099776K)], 0.2363739 secs] [Times: user=0.11 sys=0.02, real=0.24 secs] 2021-09-13T09:31:50.257+0200: 222.204: [GC (Allocation Failure) 2021-09-13T09:31:50.257+0200: 222.204: [DefNew: 279616K->20089K(314560K), 0.1042210 secs] 329798K->70271K(1013632K), 0.1043736 secs] [Times: user=0.04 sys=0.03, real=0.10 secs] 2021-09-13T09:40:32.456+0200: 744.403: [GC (Allocation Failure) 2021-09-13T09:40:32.456+0200: 744.403: [DefNew: 299705K->435K(314560K), 0.0255928 secs] 349887K->56275K(1013632K), 0.0257074 secs] [Times: user=0.02 sys=0.01, real=0.03 secs] 2021-09-13T09:50:41.809+0200: 1353.756: [GC (Allocation Failure) 2021-09-13T09:50:41.809+0200: 1353.756: [DefNew: 280051K->551K(314560K), 0.0089400 secs] 335891K->56391K(1013632K), 0.0090356 secs] [Times: user=0.01 sys=0.00, real=0.01 secs] 2021-09-13T10:01:33.109+0200: 2005.056: [GC (Allocation Failure) 2021-09-13T10:01:33.109+0200: 2005.056: [DefNew: 280167K->707K(314560K), 0.0099544 secs] 336007K->56547K(1013632K), 0.0100724 secs] [Times: user=0.00 sys=0.00, real=0.01 secs] 2021-09-13T10:11:53.384+0200: 2625.331: [GC (Allocation Failure) 2021-09-13T10:11:53.384+0200: 2625.331: [DefNew: 280323K->857K(314560K), 0.0095649 secs] 336163K->56697K(1013632K), 0.0096763 secs] [Times: user=0.01 sys=0.00, real=0.01 secs] 2021-09-13T10:21:31.798+0200: 3203.745: [GC (Allocation Failure) 2021-09-13T10:21:31.798+0200: 3203.745: [DefNew: 280473K->945K(314560K), 0.0085233 secs] 336313K->56785K(1013632K), 0.0086403 secs] [Times: user=0.01 sys=0.00, real=0.01 secs] 2021-09-13T10:31:44.561+0200: 3816.508: [GC (Allocation Failure) 2021-09-13T10:31:44.561+0200: 3816.508: [DefNew: 280561K->1053K(314560K), 0.0103383 secs] 336401K->56893K(1013632K), 0.0104447 secs] [Times: user=0.01 sys=0.00, real=0.01 secs] 2021-09-13T10:41:51.289+0200: 4423.236: [GC (Allocation Failure) 2021-09-13T10:41:51.289+0200: 4423.236: [DefNew: 280669K->1009K(314560K), 0.0100803 secs] 336509K->56849K(1013632K), 0.0101961 secs] [Times: user=0.01 sys=0.00, real=0.01 secs] 2021-09-13T10:52:13.378+0200: 5045.325: [GC (Allocation Failure) 2021-09-13T10:52:13.378+0200: 5045.325: [DefNew: 280625K->1266K(314560K), 0.0091235 secs] 336465K->57106K(1013632K), 0.0092590 secs] [Times: user=0.00 sys=0.01, real=0.01 secs] 2021-09-13T11:02:20.253+0200: 5652.200: [GC (Allocation Failure) 2021-09-13T11:02:20.253+0200: 5652.200: [DefNew: 280882K->1323K(314560K), 0.0097592 secs] 336722K->57163K(1013632K), 0.0098574 secs] [Times: user=0.01 sys=0.00, real=0.01 secs] ************************************************************ and here's my flink-conf.yaml file taskmanager.numberOfTaskSlots: 2 blob.server.port: 6124 jobmanager.rpc.port: 6123 taskmanager.rpc.port: 6122 queryable-state.proxy.ports: 6125 jobmanager.memory.process.size: 1600m taskmanager.memory.process.size: 1728m parallelism.default: 2 #HA K8S kubernetes.cluster-id: myJob high-availability: org.apache.flink.kubernetes.highavailability.KubernetesHaServicesFactory high-availability.storageDir: s3://flink-data-integ/data/flink-ha/myJob kubernetes.namespace: flink-pushavoo-flink-rec high-availability.kubernetes.leader-election.lease-duration: 60 s high-availability.kubernetes.leader-election.renew-deadline: 60 s restart-strategy: fixed-delay restart-strategy.fixed-delay.attempts: 10 #Checkpoints state.backend: filesystem state.checkpoints.dir: s3://flink-data/data/checkpoints/myJob state.checkpoints.num-retained: 10 #flink-prometheus metrics.reporters: prometheus metrics.reporter.prometheus.class: org.apache.flink.metrics.prometheus.PrometheusReporter metrics.reporter.prometheus.port: 9249 #logback classloader.parent-first-patterns.additional: net.logstash.logback #S3 s3.endpoint: ******* s3.access-key: ******** s3.secret-key: ****** env.java.opts.jobmanager: -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:/opt/flink/log/jobmanager-gc.log
