Hi James, the configuration options were consolidates a bit for Flink 1.5. What was called state.backend.fs.checkpointdir before was the same as state.checkpoints.dir, we now made this the same for the two backends. Also, the file-backend does not need local storage on the worker nodes, that's why it doesn't have an equivalent of state.backend.rocksdb.localdir. Side note: due to legacy reasons FsStateBackend (or filesystem backend) is a bit of a misnomer: both the rocksdb backend and the filesystem backend will checkpoint their state to a distributed filesystem. The difference is in how they keep the state on the workers. The rocksdb backend keeps state in a RocksDB data base, which needs disk storage while the filesystem backend keeps state as objects on the Java heap, that's also why it doesn't have the localdir config.
Regarding local recovery, please have a look at this page: https://ci.apache.org/projects/flink/flink-docs-release-1.5/ops/state/large_state_tuning.html#task-local-recovery <https://ci.apache.org/projects/flink/flink-docs-release-1.5/ops/state/large_state_tuning.html#task-local-recovery> Hope that helps. Aljoscha > On 12. Jul 2018, at 09:57, James Isaac <dataenginee...@gmail.com> wrote: > > Comparing the state backend configurations between Flink 1.4 and 1.5, I have > a question: > > In Flink 1.5, we have a state.checkpoints.dir location and a > state.backend.rocksdb.localdir. It was the same in Flink 1.4. > However, in the case of filesystem backends, we have > state.backend.fs.checkpointdir in Flink 1.4, but not in Flink 1.5. What has > changed here? > > Also what is taskmanager.state.local.root-dirs ? I don't see any description > on the docs page > (https://ci.apache.org/projects/flink/flink-docs-release-1.5/ops/config.html#taskmanager-state-local-root-dirs > > <https://ci.apache.org/projects/flink/flink-docs-release-1.5/ops/config.html#taskmanager-state-local-root-dirs>) > > Regards, > James > >
