Re: State backend configuration

2018-07-12 Thread Aljoscha Krettek
Hi James, the configuration options were consolidates a bit for Flink 1.5. What was called state.backend.fs.checkpointdir before was the same as state.checkpoints.dir, we now made this the same for the two backends. Also, the file-backend does not need local storage on the worker nodes, that's

State backend configuration

2018-07-12 Thread James Isaac
Comparing the state backend configurations between Flink 1.4 and 1.5, I have a question: In Flink 1.5, we have a *state.checkpoints.dir* location and a *state.backend.rocksdb.localdir*. It was the same in Flink 1.4. However, in the case of filesystem backends, we have *state.backend.fs.checkpointd

Re: state backend configuration

2018-03-14 Thread Aljoscha Krettek
Hi, This is not possible on versions of Flink prior to 1.5 but starting with that (as of yet unreleased) versions you'll be able to configure this via state.backend.incremental: true. Configurations that you do in code will take precedence. Best, Aljoscha > On 14. Mar 2018, at 02:33, Jayant A

state backend configuration

2018-03-14 Thread Jayant Ameta
Hi, Can I enable incremental checkpoint for rocksdb via flink-conf.yaml file? Also, I can set state.backend via the conf file as well as in my code using env.setStateBackend(). Which of these take precedence? Jayant Ameta