Hi,
1. Checkpoint size has nothing to do with managed memory size. Checkpoint files are stored in local disk while managed memory are mostly in-memory caches used by RocksDB. 2. Yes, the main difference between cp size and full cp size is that cp size only counts incremental files while full cp size counts all files. Best, Zhanghao Chen ________________________________ From: banu priya <banuke...@gmail.com> Sent: Monday, August 5, 2024 16:45 To: user@flink.apache.org <user@flink.apache.org> Subject: Managed memory and state size Hi All, As my incremental rocksdb check point size is increasing continuously, I am trying to reduce managed memory. I am expecting my states size wont grow more than managed memory (because of rocksdb compaction) so that my check points size also will be in control. I allocated very minimal memory for managed memory and tested, even then check point size is growing. Checkpoint size is more than managed memory allocated.. where can I get the state size??. I was expecting checkpoint size in flink webUI is the my state size(ie currently transfer red from Taskmanager pod to DFS). Full check point size is sum of old files ( already present in chk/shared directory) plus newly transferred file(checkpoint size). Is this understanding correct?? What is the difference between Checkpoints size and full check points size in flink webUI?? Thanks Banu
