unsubscribe

unsubscribe - 原始邮件 - 发件人：Simon Frei 收件人：Alexis Sarda-Espinosa , user 主题：Re: unexpected high mem. usage / potential config misinterpretation 日期：2024年09月19日 04点11分 Hi Alexis,   Thanks for chiming in. To my knowledge and experience that’s not how it works – we can consistently succeed to

Re: unexpected high mem. usage / potential config misinterpretation

Hi Alexis, Thanks for chiming in. To my knowledge and experience that’s not how it works – we can consistently succeed to run this job in batch mode on less data, but still considerably more data than would fit in memory. Also even on the full data, the initial source tasks always succeed. To m

Re: unexpected high mem. usage / potential config misinterpretation

Hi Simon, I hope someone corrects me if I'm wrong, but just based on "batch mode processing terabytes of data", I feel batch mode may be the issue. I am under the impression that batch mode forces everything emitted by the sources to RAM before any downstream operators do anything, so even if each

unexpected high mem. usage / potential config misinterpretation

Hi, tl;dr: Flink batch streaming API job resident memory usage grows far beyond expectations, resulting in system OOM kill/JVM native memory allocation failure - would appreciate a look over our config/assumptions to potentially spot any obvious mistakes. Longer form: My colleague and I are t

unscribe