Hi Nick, Both questions are hard to answer given that it depends on your hardware, access patterns (read/update), record size/structure, parallelism, and probably a ton of other parameters.
The usual approach is to simply evaluate it in your setting. Since it's a matter of configuration, you can do some A/B testing. In general, you need RocksDB if you want to have incremental checkpoints, which is recommended if you have rather few updates and big state. On Mon, Jun 22, 2020 at 2:13 AM Nick Bendtner <buggi...@gmail.com> wrote: > Hi guys, > I have a few questions on state backends. > Is there a guideline on how big the state has to be where it makes sense > to use RocksDB rather than FsStatebackend ? Is there an analysis on > latency for a full checkpoint for FsSateBackend based on increase in state > size ? > > > Best, > Nick. > -- Arvid Heise | Senior Java Developer <https://www.ververica.com/> Follow us @VervericaData -- Join Flink Forward <https://flink-forward.org/> - The Apache Flink Conference Stream Processing | Event Driven | Real Time -- Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany -- Ververica GmbH Registered at Amtsgericht Charlottenburg: HRB 158244 B Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji (Toni) Cheng
