Re: Applying backpressure to limit state memory consumption

2022-05-23 Thread Robin Cassan
Thanks Yu'an for your answer! Our issue lies in the fact that the window size is variable with the incoming traffic and would like a solution to avoid filling our window state in case of spikes. Even with RocksDB we would eventually be limited by the disk size (which, admittedly, is usually bigger

Re: Applying backpressure to limit state memory consumption

2022-05-19 Thread yu'an huang
H Robini, In my experience, the state size of memory state backend is limit by the heap memory. See this link for details: https://nightlies.apache.org/flink/flink-docs-master/docs/ops/state/state_backends/ “When deciding between HashMapStateBackend and RocksDB, it is a choice between performan