Re: FileSystem vs RocksDb backend

2017-06-06 Thread Fabian Hueske
Hi Fritz, the FSStateBackend is a lot faster than the RocksDBStateBackend. I don't know exact numbers but if I remember correctly somebody mentioned a 10x difference. Delta checkpointing is only supported by RocksDB since Flink 1.3. Given that the FSStateBackend is only suitable for small states,

FileSystem vs RocksDb backend

2017-06-02 Thread Fritz Budiyanto
Hi All, If my states fit in the JVM heap, should I be using filesystem backend with HDFS instead of RocksDB ? Could someone comment on the FS vs RocksDb backend performance ? I’ve also heard that RocksDb backend is required to support delta checkpointing. Is this true ? Is delta checkpointing