Re: unexpected high mem. usage / potential config misinterpretation

2024-09-18 Thread Simon Frei
: Simon Frei Cc: user Subject: Re: unexpected high mem. usage / potential config misinterpretation Hi Simon, I hope someone corrects me if I'm wrong, but just based on "batch mode processing terabytes of data", I feel batch mode may be the issue. I am under the impression that ba

unexpected high mem. usage / potential config misinterpretation

2024-09-18 Thread Simon Frei
Hi, tl;dr: Flink batch streaming API job resident memory usage grows far beyond expectations, resulting in system OOM kill/JVM native memory allocation failure - would appreciate a look over our config/assumptions to potentially spot any obvious mistakes. Longer form: My colleague and I are t