Hi Chen,
with version 1.10 Flink introduced that RocksDB uses Flink's managed memory
[1]. This shall prevent RocksDB from exceeding the memory limits of a
process/container. Unfortunately, this is not yet perfect due to a problem
in RocksDB [2]. Due to this fact, RocksDB can still exceed the manag
Hi Till,
We did some investigation and found this memory usage point to
rocksdbstatebackend running on managed memory. So far we have seen this bug
in rocksdbstatebackend on managed memory. we followed suggestion [1] and
disabled managed memory management so far not seeing issue.
I felt this mi
Hi Chen,
You are right that Flink changed its memory model with Flink 1.10. Now the
memory model is better defined and stricter. You can find information about
it here [1]. For some pointers towards potential problems please take a
look at [2].
What you need to do is to figure out where the non-h
Hi there,
We were using flink 1.11.2 in production with a large setting. The job runs
fine for a couple of days and ends up with a restart loop caused by YARN
container memory kill. This is not observed while running against 1.9.1
with the same setting.
Here is JVM environment passed to 1.11 as we
