yuvipanda created FLINK-29762:
---------------------------------
Summary: Can not create a standalone cluster with reactive mode
using the operator
Key: FLINK-29762
URL: https://issues.apache.org/jira/browse/FLINK-29762
Project: Flink
Issue Type: Bug
Components: Kubernetes Operator
Environment: Kubernetes Version 1.22 on EKS.
Flink Operator veresion 1.2.0
Flink Veresion 1.15 (errors in 1.14 too)
Reporter: yuvipanda
I'm trying to create a minimal running flink cluster with reactive scaling
using the kubernetes operator (running v1.2.0), with the following YAML:
{{
kind: FlinkDeployment
metadata:
name: test-flink-cluster
spec:
flinkConfiguration:
scheduler-mode: reactive
flinkVersion: v1_15
image: flink:1.15
jobManager:
replicas: 1
resource:
cpu: 0.2
memory: 1024m
mode: standalone
serviceAccount: flink
taskManager:
replicas: 1
resource:
cpu: 0.2
memory: 1024m}}
However, this causes the jobmanager to crash with the following:
{{sed: couldn't open temporary file /opt/flink/conf/sedLX7Jx8: Read-only file
system}}
{{sed: couldn't open temporary file /opt/flink/conf/sed1vva8t: Read-only file
system}}
{{/docker-entrypoint.sh: line 73: /opt/flink/conf/flink-conf.yaml: Read-only
file system}}
{{/docker-entrypoint.sh: line 89: /opt/flink/conf/flink-conf.yaml.tmp:
Read-only file system}}
{{Starting Job Manager}}
{{Starting standalonesession as a console application on host
test-flink-cluster-58cd584fdd-xwbtf.}}
{{2022-10-25 18:32:00,422 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
--------------------------------------------------------------------------------}}
{{2022-10-25 18:32:00,510 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
Preconfiguration: }}
{{2022-10-25 18:32:00,512 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - }}
{{RESOURCE_PARAMS extraction logs:}}
{{jvm_params: -Xmx469762048 -Xms469762048 -XX:MaxMetaspaceSize=268435456}}
{{dynamic_configs: -D jobmanager.memory.off-heap.size=134217728b -D
jobmanager.memory.jvm-overhead.min=201326592b -D
jobmanager.memory.jvm-metaspace.size=268435456b -D
jobmanager.memory.heap.size=469762048b -D
jobmanager.memory.jvm-overhead.max=201326592b}}
{{logs: WARNING: sun.reflect.Reflection.getCallerClass is not supported. This
will impact performance.}}
{{INFO [] - Loading configuration property: blob.server.port, 6124}}
{{INFO [] - Loading configuration property: kubernetes.jobmanager.annotations,
flinkdeployment.flink.apache.org/generation:1}}
{{INFO [] - Loading configuration property: kubernetes.jobmanager.replicas, 1}}
{{INFO [] - Loading configuration property: scheduler-mode, reactive}}
{{INFO [] - Loading configuration property:
"kubernetes.operator.metrics.reporter.prom.port", "9999"}}
{{INFO [] - Loading configuration property: jobmanager.rpc.address,
test-flink-cluster.default}}
{{INFO [] - Loading configuration property: kubernetes.taskmanager.cpu, 0.2}}
{{INFO [] - Loading configuration property: "prometheus.io/port", "9999"}}
{{INFO [] - Loading configuration property: kubernetes.service-account, flink}}
{{INFO [] - Loading configuration property: kubernetes.cluster-id,
test-flink-cluster}}
{{INFO [] - Loading configuration property: kubernetes.container.image,
flink:1.15}}
{{INFO [] - Loading configuration property: parallelism.default, 2}}
{{INFO [] - Loading configuration property: kubernetes.namespace, default}}
{{INFO [] - Loading configuration property: taskmanager.numberOfTaskSlots, 2}}
{{INFO [] - Loading configuration property:
kubernetes.rest-service.exposed.type, ClusterIP}}
{{INFO [] - Loading configuration property: "prometheus.io/scrape", "true"}}
{{INFO [] - Loading configuration property: taskmanager.memory.process.size,
1024m}}
{{INFO [] - Loading configuration property:
"kubernetes.operator.metrics.reporter.prom.class",
"org.apache.flink.metrics.prometheus.PrometheusReporter"}}
{{INFO [] - Loading configuration property: web.cancel.enable, false}}
{{INFO [] - Loading configuration property: execution.target, remote}}
{{INFO [] - Loading configuration property: jobmanager.memory.process.size,
1024m}}
{{INFO [] - Loading configuration property: taskmanager.rpc.port, 6122}}
{{INFO [] - Loading configuration property: kubernetes.internal.cluster-mode,
SESSION}}
{{INFO [] - Loading configuration property: kubernetes.jobmanager.cpu, 0.2}}
{{INFO [] - Loading configuration property: $internal.flink.version, v1_15}}
{{INFO [] - The derived from fraction jvm overhead memory (102.400mb
(107374184 bytes)) is less than its min value 192.000mb (201326592 bytes), min
value will be used instead}}
{{INFO [] - Final Master Memory configuration:}}
{{INFO [] - Total Process Memory: 1024.000mb (1073741824 bytes)}}
{{INFO [] - Total Flink Memory: 576.000mb (603979776 bytes)}}
{{INFO [] - JVM Heap: 448.000mb (469762048 bytes)}}
{{INFO [] - Off-heap: 128.000mb (134217728 bytes)}}
{{INFO [] - JVM Metaspace: 256.000mb (268435456 bytes)}}
{{INFO [] - JVM Overhead: 192.000mb (201326592 bytes)}}
{{2022-10-25 18:32:00,514 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
--------------------------------------------------------------------------------}}
{{2022-10-25 18:32:00,516 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Starting
StandaloneSessionClusterEntrypoint (Version: 1.15.1, Scala: 2.12, Rev:f494be6,
Date:2022-06-20T14:40:28+02:00)}}
{{2022-10-25 18:32:00,516 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - OS current
user: flink}}
{{2022-10-25 18:32:00,517 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Current
Hadoop/Kerberos user: <no hadoop dependency found>}}
{{2022-10-25 18:32:00,517 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - JVM: OpenJDK
64-Bit Server VM - Oracle Corporation - 11/11.0.16+8}}
{{2022-10-25 18:32:00,519 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Arch: amd64}}
{{2022-10-25 18:32:00,519 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Maximum heap
size: 433 MiBytes}}
{{2022-10-25 18:32:00,520 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - JAVA_HOME:
/usr/local/openjdk-11}}
{{2022-10-25 18:32:00,520 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - No Hadoop
Dependency available}}
{{2022-10-25 18:32:00,522 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - JVM
Options:}}
{{2022-10-25 18:32:00,523 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-Xmx469762048}}
{{2022-10-25 18:32:00,523 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-Xms469762048}}
{{2022-10-25 18:32:00,523 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-XX:MaxMetaspaceSize=268435456}}
{{2022-10-25 18:32:00,524 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-Dlog.file=/opt/flink/log/flink--standalonesession-0-test-flink-cluster-58cd584fdd-xwbtf.log}}
{{2022-10-25 18:32:00,524 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-Dlog4j.configuration=file:/opt/flink/conf/log4j-console.properties}}
{{2022-10-25 18:32:00,524 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-Dlog4j.configurationFile=file:/opt/flink/conf/log4j-console.properties}}
{{2022-10-25 18:32:00,524 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-Dlogback.configurationFile=file:/opt/flink/conf/logback-console.xml}}
{{2022-10-25 18:32:00,524 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Program
Arguments:}}
{{2022-10-25 18:32:00,526 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
--configDir}}
{{2022-10-25 18:32:00,528 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
/opt/flink/conf}}
{{2022-10-25 18:32:00,528 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
--executionMode}}
{{2022-10-25 18:32:00,528 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - cluster}}
{{2022-10-25 18:32:00,529 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - -D}}
{{2022-10-25 18:32:00,529 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
jobmanager.memory.off-heap.size=134217728b}}
{{2022-10-25 18:32:00,529 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - -D}}
{{2022-10-25 18:32:00,533 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
jobmanager.memory.jvm-overhead.min=201326592b}}
{{2022-10-25 18:32:00,533 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - -D}}
{{2022-10-25 18:32:00,534 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
jobmanager.memory.jvm-metaspace.size=268435456b}}
{{2022-10-25 18:32:00,534 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - -D}}
{{2022-10-25 18:32:00,534 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
jobmanager.memory.heap.size=469762048b}}
{{2022-10-25 18:32:00,534 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - -D}}
{{2022-10-25 18:32:00,534 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
jobmanager.memory.jvm-overhead.max=201326592b}}
{{2022-10-25 18:32:00,534 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Classpath:
/opt/flink/lib/flink-cep-1.15.1.jar:/opt/flink/lib/flink-connector-files-1.15.1.jar:/opt/flink/lib/flink-csv-1.15.1.jar:/opt/flink/lib/flink-json-1.15.1.jar:/opt/flink/lib/flink-scala_2.12-1.15.1.jar:/opt/flink/lib/flink-shaded-zookeeper-3.5.9.jar:/opt/flink/lib/flink-table-api-java-uber-1.15.1.jar:/opt/flink/lib/flink-table-planner-loader-1.15.1.jar:/opt/flink/lib/flink-table-runtime-1.15.1.jar:/opt/flink/lib/log4j-1.2-api-2.17.1.jar:/opt/flink/lib/log4j-api-2.17.1.jar:/opt/flink/lib/log4j-core-2.17.1.jar:/opt/flink/lib/log4j-slf4j-impl-2.17.1.jar:/opt/flink/lib/flink-dist-1.15.1.jar:::}}
{{2022-10-25 18:32:00,535 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
--------------------------------------------------------------------------------}}
{{2022-10-25 18:32:00,611 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Registered
UNIX signal handlers for [TERM, HUP, INT]}}
{{2022-10-25 18:32:00,635 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: blob.server.port, 6124}}
{{2022-10-25 18:32:00,715 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: kubernetes.jobmanager.annotations,
flinkdeployment.flink.apache.org/generation:1}}
{{2022-10-25 18:32:00,716 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: kubernetes.jobmanager.replicas, 1}}
{{2022-10-25 18:32:00,716 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: scheduler-mode, reactive}}
{{2022-10-25 18:32:00,716 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: "kubernetes.operator.metrics.reporter.prom.port",
"9999"}}
{{2022-10-25 18:32:00,717 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: jobmanager.rpc.address, test-flink-cluster.default}}
{{2022-10-25 18:32:00,717 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: kubernetes.taskmanager.cpu, 0.2}}
{{2022-10-25 18:32:00,717 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: "prometheus.io/port", "9999"}}
{{2022-10-25 18:32:00,717 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: kubernetes.service-account, flink}}
{{2022-10-25 18:32:00,717 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: kubernetes.cluster-id, test-flink-cluster}}
{{2022-10-25 18:32:00,718 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: kubernetes.container.image, flink:1.15}}
{{2022-10-25 18:32:00,718 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: parallelism.default, 2}}
{{2022-10-25 18:32:00,718 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: kubernetes.namespace, default}}
{{2022-10-25 18:32:00,718 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: taskmanager.numberOfTaskSlots, 2}}
{{2022-10-25 18:32:00,718 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: kubernetes.rest-service.exposed.type, ClusterIP}}
{{2022-10-25 18:32:00,718 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: "prometheus.io/scrape", "true"}}
{{2022-10-25 18:32:00,719 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: taskmanager.memory.process.size, 1024m}}
{{2022-10-25 18:32:00,719 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: "kubernetes.operator.metrics.reporter.prom.class",
"org.apache.flink.metrics.prometheus.PrometheusReporter"}}
{{2022-10-25 18:32:00,719 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: web.cancel.enable, false}}
{{2022-10-25 18:32:00,719 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: execution.target, remote}}
{{2022-10-25 18:32:00,720 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: jobmanager.memory.process.size, 1024m}}
{{2022-10-25 18:32:00,720 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: taskmanager.rpc.port, 6122}}
{{2022-10-25 18:32:00,720 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: kubernetes.internal.cluster-mode, SESSION}}
{{2022-10-25 18:32:00,720 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: kubernetes.jobmanager.cpu, 0.2}}
{{2022-10-25 18:32:00,720 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: $internal.flink.version, v1_15}}
{{2022-10-25 18:32:00,924 ERROR
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Reactive mode
is configured for an unsupported cluster type. At the moment, reactive mode is
only supported by standalone application clusters (bin/standalone-job.sh).}}
{{Exception in thread "main"
org.apache.flink.configuration.IllegalConfigurationException: Reactive mode is
configured for an unsupported cluster type. At the moment, reactive mode is
only supported by standalone application clusters (bin/standalone-job.sh).}}
{{ at
org.apache.flink.runtime.entrypoint.ClusterEntrypoint.<init>(ClusterEntrypoint.java:177)}}
{{ at
org.apache.flink.runtime.entrypoint.SessionClusterEntrypoint.<init>(SessionClusterEntrypoint.java:39)}}
{{ at
org.apache.flink.runtime.entrypoint.StandaloneSessionClusterEntrypoint.<init>(StandaloneSessionClusterEntrypoint.java:32)}}
{{ at
org.apache.flink.runtime.entrypoint.StandaloneSessionClusterEntrypoint.main(StandaloneSessionClusterEntrypoint.java:56)}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
