Hi folks,
I have a question related configuration for new memory introduced in flink
1.10. Has anyone encountered similar problem?
I'm trying to make use of *taskmanager.memory.process.size* configuration
key in combination with mesos session cluster, but I get an error like this:
2020-03-11 11:44:09,771 [main] ERROR
org.apache.flink.mesos.entrypoint.MesosTaskExecutorRunner - Error
while starting the TaskManager
org.apache.flink.configuration.IllegalConfigurationException: Failed
to create TaskExecutorResourceSpec
at
org.apache.flink.runtime.taskexecutor.TaskExecutorResourceUtils.resourceSpecFromConfig(TaskExecutorResourceUtils.java:72)
at
org.apache.flink.runtime.taskexecutor.TaskManagerRunner.startTaskManager(TaskManagerRunner.java:356)
at
org.apache.flink.runtime.taskexecutor.TaskManagerRunner.<init>(TaskManagerRunner.java:152)
at
org.apache.flink.runtime.taskexecutor.TaskManagerRunner.runTaskManager(TaskManagerRunner.java:308)
at
org.apache.flink.mesos.entrypoint.MesosTaskExecutorRunner.lambda$main$0(MesosTaskExecutorRunner.java:106)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at java.base/javax.security.auth.Subject.doAs(Subject.java:423)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1692)
at
org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
at
org.apache.flink.mesos.entrypoint.MesosTaskExecutorRunner.main(MesosTaskExecutorRunner.java:105)
Caused by: org.apache.flink.configuration.IllegalConfigurationException:
The required configuration option Key:
'taskmanager.memory.task.heap.size' , default: null (fallback keys:
[]) is not set
at
org.apache.flink.runtime.taskexecutor.TaskExecutorResourceUtils.checkConfigOptionIsSet(TaskExecutorResourceUtils.java:90)
at
org.apache.flink.runtime.taskexecutor.TaskExecutorResourceUtils.lambda$checkTaskExecutorResourceConfigSet$0(TaskExecutorResourceUtils.java:84)
at java.base/java.util.Arrays$ArrayList.forEach(Arrays.java:4390)
at
org.apache.flink.runtime.taskexecutor.TaskExecutorResourceUtils.checkTaskExecutorResourceConfigSet(TaskExecutorResourceUtils.java:84)
at
org.apache.flink.runtime.taskexecutor.TaskExecutorResourceUtils.resourceSpecFromConfig(TaskExecutorResourceUtils.java:70)
... 9 more
But when task manager is launched, it correctly parses process memory key:
2020-03-11 11:43:55,376 [main] INFO
org.apache.flink.mesos.entrypoint.MesosTaskExecutorRunner -
--------------------------------------------------------------------------------
2020-03-11 11:43:55,377 [main] INFO
org.apache.flink.mesos.entrypoint.MesosTaskExecutorRunner -
Starting MesosTaskExecutorRunner (Version: 1.10.0, Rev:aa4eb8f,
Date:07.02.2020 @ 19:18:19 CET)
2020-03-11 11:43:55,377 [main] INFO
org.apache.flink.mesos.entrypoint.MesosTaskExecutorRunner - OS
current user: root
2020-03-11 11:43:57,347 [main] WARN
org.apache.hadoop.util.NativeCodeLoader - Unable
to load native-hadoop library for your platform... using builtin-java
classes where applicable
2020-03-11 11:43:57,535 [main] INFO
org.apache.flink.mesos.entrypoint.MesosTaskExecutorRunner - JVM:
OpenJDK 64-Bit Server VM - AdoptOpenJDK - 11/11.0.2+9
2020-03-11 11:43:57,535 [main] INFO
org.apache.flink.mesos.entrypoint.MesosTaskExecutorRunner -
Maximum heap size: 746 MiBytes
2020-03-11 11:43:57,535 [main] INFO
org.apache.flink.mesos.entrypoint.MesosTaskExecutorRunner -
JAVA_HOME: (not set)
2020-03-11 11:43:57,539 [main] INFO
org.apache.flink.mesos.entrypoint.MesosTaskExecutorRunner -
Hadoop version: 2.6.5
2020-03-11 11:43:57,539 [main] INFO
org.apache.flink.mesos.entrypoint.MesosTaskExecutorRunner - JVM
Options:
2020-03-11 11:43:57,539 [main] INFO
org.apache.flink.mesos.entrypoint.MesosTaskExecutorRunner -
-Xmx781818251
2020-03-11 11:43:57,539 [main] INFO
org.apache.flink.mesos.entrypoint.MesosTaskExecutorRunner -
-Xms781818251
2020-03-11 11:43:57,540 [main] INFO
org.apache.flink.mesos.entrypoint.MesosTaskExecutorRunner -
-XX:MaxDirectMemorySize=317424929
2020-03-11 11:43:57,540 [main] INFO
org.apache.flink.mesos.entrypoint.MesosTaskExecutorRunner -
-XX:MaxMetaspaceSize=100663296
2020-03-11 11:43:57,540 [main] INFO
org.apache.flink.mesos.entrypoint.MesosTaskExecutorRunner -
-Dlog.file=/var/log/flink-session-cluster/taskmanager.log
2020-03-11 11:43:57,540 [main] INFO
org.apache.flink.mesos.entrypoint.MesosTaskExecutorRunner -
-Dlog4j.configuration=file:/opt/flink/conf/log4j.properties
2020-03-11 11:43:57,540 [main] INFO
org.apache.flink.mesos.entrypoint.MesosTaskExecutorRunner -
-Dlogback.configurationFile=file:/opt/flink/conf/logback.xml
2020-03-11 11:43:57,540 [main] INFO
org.apache.flink.mesos.entrypoint.MesosTaskExecutorRunner -
Program Arguments: (none)
2020-03-11 11:43:57,540 [main] INFO
org.apache.flink.mesos.entrypoint.MesosTaskExecutorRunner -
Classpath:
/opt/flink/lib/apache-log4j-extras-1.2.17.jar:/opt/flink/lib/flink-metrics-graphite-1.10.0.jar:/opt/flink/lib/flink-shaded-hadoop-2-uber-2.6.5-8.0.jar:/opt/flink/lib/flink-table-blink_2.12-1.10.0.jar:/opt/flink/lib/flink-table_2.12-1.10.0.jar:/opt/flink/lib/log4j-1.2.17.jar:/opt/flink/lib/slf4j-log4j12-1.7.15.jar:/opt/flink/lib/flink-dist_2.12-1.10.0.jar:
2020-03-11 11:43:57,541 [main] INFO
org.apache.flink.mesos.entrypoint.MesosTaskExecutorRunner -
--------------------------------------------------------------------------------
2020-03-11 11:43:57,542 [main] INFO
org.apache.flink.mesos.entrypoint.MesosTaskExecutorRunner -
Registered UNIX signal handlers for [TERM, HUP, INT]
2020-03-11 11:43:57,550 [main] INFO
org.apache.flink.configuration.GlobalConfiguration -
Loading configuration property: taskmanager.memory.process.size, 2g
2020-03-11 11:43:57,550 [main] INFO
org.apache.flink.configuration.GlobalConfiguration -
Loading configuration property: taskmanager.cpu.cores, 2
2020-03-11 11:43:57,551 [main] INFO
org.apache.flink.configuration.GlobalConfiguration -
Loading configuration property: taskmanager.numberOfTaskSlots, 4
2020-03-11 11:43:57,551 [main] INFO
org.apache.flink.configuration.GlobalConfiguration -
Loading configuration property: parallelism.default, 1
...
Judging by the docs specifying *taskmanager.memory.process.size* key should
be enough to launch the job, but it seems like this value is ignored.
I would appreciate any suggestion.
Regards and thanks in advance,
Alex.
