Re: RocksDB and local file system

2019-11-07 Thread Congxian Qiu
Hi The path will store the checkpoints, and Flink will fetch the checkpoint files to restore the state if any failure occurred. If you specify the local file system, when restoring from checkpoint, Flink may can't find the checkpoint files, and can't restore from last checkpoints. Best, Congxian

Re: RocksDB and local file system

2019-11-06 Thread vino yang
Hi Jaqie, For testing, you can use the local file system pattern (e.g. "file:///"). Technically speaking, it's OK to specify the string path provided by you. However, in the production environment, we do not recommend using the local file system. Because it does not provide high availability. Be

RocksDB and local file system

2019-11-06 Thread Jaqie Chan
Hello I am using Flink rocksDb state backend, the documentation seems to imply i can use a regular file system such as: file:///data/flink/checkpoints, but the code javadoc only mentions hdfs or s3 option here. I am wondering if it's possible to use local file system with flink rocksdb backend.