Re: Zigzag shape in TM JVM used memory

2021-04-08 Thread Piotr Nowojski
Hi, I don't think there is a Flink specific answer to this question. Just do what you would normally do with a normal Java application running inside a JVM. If there is an OOM on heap space, you can either try to bump the heap space, or reduce usage of it. The only Flink specific part is probably

Re: Zigzag shape in TM JVM used memory

2021-04-07 Thread Lu Niu
Hi, Piotr Thanks for replying. I asked this because such a pattern might imply memory oversubscription. For example, I tuned down the memory of one app from heap 2.63GB to 367MB and the job still runs fine: before: https://drive.google.com/file/d/1o8k9Vv3yb5gXITi4GvmlXMteQcRfmOhr/view?usp=sharing

Re: Zigzag shape in TM JVM used memory

2021-04-06 Thread Piotr Nowojski
Hi, this should be posted on the user mailing list not the dev. Apart from that, this looks like normal/standard behaviour of JVM, and has very little to do with Flink. Garbage Collector (GC) is kicking in when memory usage is approaching some threshold: https://www.google.com/search?q=jvm+heap+m

Re: Zigzag shape in TM JVM used memory

2021-04-05 Thread Lu Niu
Hi, we need to update our email system then :) . Here are the links: https://drive.google.com/file/d/1lZ5_P8_NqsN1JeLzutGj4DxkyWJN75mR/view?usp=sharing https://drive.google.com/file/d/1J6c6rQJwtDp1moAGlvQyLQXTqcuG4HjL/view?usp=sharing https://drive.google.com/file/d/1-R2KzsABC471AEjkF5qTm5O3V47

Re: Zigzag shape in TM JVM used memory

2021-04-04 Thread Yun Gao
Hi Lu, The image seems not be able to shown due to the mail server limitation, could you upload it somewhere and paste the link here ? Logically, I think zigzag usually due to there are some small object get created and eliminated soon in the heap. Are you running a SQL job or a DataStream jo