I start a standalone session on a single server with only one taskMgr.
The JVM metaspace will become bigger after submitting a new job.
Even if I cancel the submitted job, the JVM metaspace will not decrease.
After submitting about 15 times, the task manager was shut down because of
OOM
2022-04-24 21:39:29,856 ERROR
org.apache.flink.runtime.taskexecutor.TaskManagerRunner [] - Fatal
error oc
curred while executing the TaskManager. Shutting it down...
java.lang.OutOfMemoryError: Metaspace. The metaspace out-of-memory error
has occurred. This can mean two thing
s: either the job requires a larger size of JVM metaspace to load classes
or there is a class loading leak. In
the first case 'taskmanager.memory.jvm-metaspace.size' configuration
option should be increased. If the error
persists (usually in cluster after several job (re-)submissions) then
there is probably a class loading leak
in user code or some of its dependencies which has to be investigated and
fixed. The task executor has to be s
hutdown...
at java.lang.ClassLoader.defineClass1(Native Method) ~[?:1.8.0_322]
at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
~[?:1.8.0_322]
at
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
~[?:1.8.0_322]
at java.net.URLClassLoader.defineClass(URLClassLoader.java:473)
~[?:1.8.0_322]
at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
~[?:1.8.0_322]
at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
~[?:1.8.0_322]
at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
~[?:1.8.0_322]
at java.security.AccessController.doPrivileged(Native Method)
~[?:1.8.0_322]
at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
~[?:1.8.0_322]
Seems the JVM will not unload the class info after cancelling a job.
How can i solve this?
Thanks,
Lei
