Re: Flink Task Manager GC overhead limit exceeded

2020-04-29 Thread Xintong Song
Then I would suggest the following. - Check the task manager log to see if the '-D' properties are properly loaded. They should be located at the beginning of the log file. - You can also try to log into the pod and check the JVM launch command with "ps -ef | grep TaskManagerRunner". I suspect ther

Re: Flink Task Manager GC overhead limit exceeded

2020-04-29 Thread Xintong Song
Hi Eleanore, I'd like to explain about 1 & 2. For 3, I have no idea either. 1. I dont see the heap size from UI for task manager show correctly > Despite the 'heap' in the key, 'taskmanager.heap.size' accounts for the total memory of a Flink task manager, rather than only the heap memory. A Flin

Flink Task Manager GC overhead limit exceeded

2020-04-29 Thread Eleanore Jin
Hi All, Currently I am running a flink job cluster (v1.8.2) on kubernetes with 4 pods, each pod with 4 parallelism. The flink job reads from a source topic with 96 partitions, and does per element filter, the filtered value comes from a broadcast topic and it always use the latest message as the