The network memory min (64 mb) and max (1 gb) mismatch

[Clay Teeter]

Hi flink fans,

I'm hoping for an easy solution.  I'm trying to upgrade my 9.3 cluster to
flink 10.1, but i'm running into memory configuration errors.

Such as:
*Caused by: org.apache.flink.configuration.IllegalConfigurationException:
The network memory min (64 mb) and max (1 gb) mismatch, the network memory
has to be resolved and set to a fixed value before task executor starts*

*Caused by: org.apache.flink.configuration.IllegalConfigurationException:
The required configuration option Key: 'taskmanager.cpu.cores' , default:
null (fallback keys: []) is not set*

I was able to fix a cascade of errors by explicitly setting these values:

taskmanager.memory.managed.size: {{
.Values.analytics.flink.taskManagerManagedSize }}
taskmanager.memory.task.heap.size: {{
.Values.analytics.flink.taskManagerHeapSize }}
taskmanager.memory.jvm-metaspace.size: 500m
taskmanager.cpu.cores: 4

So, the documentation implies that flink will default many of these values,
however my 101. cluster doesn't seem to be doing this.  9.3, worked great!


Do I really have to set all the memory (even network) values?  If not, what
am I missing?

If i do have to set all the memory parameters, how do I resolve "The
network memory min (64 mb) and max (1 gb) mismatch"?


My cluster runs standalone jobs on kube

flnk-config.yaml:
    state.backend: rocksdb
    state.backend.incremental: true
    state.checkpoints.num-retained: 1
    taskmanager.memory.managed.size: {{
.Values.analytics.flink.taskManagerManagedSize }}
    taskmanager.memory.task.heap.size: {{
.Values.analytics.flink.taskManagerHeapSize }}
    taskmanager.memory.jvm-metaspace.size: 500m
    taskmanager.cpu.cores: 4
    taskmanager.numberOfTaskSlots: {{
.Values.analytics.task.numberOfTaskSlots }}
    parallelism.default: {{ .Values.analytics.flink.parallelism }}


JobManger:
        command: ["/opt/flink/bin/standalone-job.sh"]
        args: ["start-foreground", "-j={{ .Values.analytics.flinkRunnable
}}",  ...

TakManager
        command: ["/opt/flink/bin/taskmanager.sh"]
        args: [
          "start-foreground",
          "-Djobmanager.rpc.address=localhost",
          "-Dmetrics.reporter.prom.port=9430"]