Hey all, We would like to start a discussion on how to enable/config Changelog Statebakcend.
As part of FLIP-158[1], Changelog state backend wraps on top of existing state backend (HashMapStateBackend, EmbeddedRocksDBStateBackend and may expect more) and delegates state changes to the underlying state backends. This thread is to discuss the problem of how Changelog StateBackend should be enabled and configured. Proposed options to enable/config state changelog is listed below: Option 1: Enable Changelog Statebackend through a Boolean Flag Option 2: Enable Changelog Statebackend through a Boolean Flag + a Special Case Option 3: Enable Changelog Statebackend through a Boolean Flag + W/O ChangelogStateBackend Exposed Option 4: Explicit Nested Configuration + “changelog.inner” prefix for inner backend Option 5: Explicit Nested Configuration + inner state backend configuration unchanged Option 6: Config Changelog and Inner Statebackend All-Together Details of each option can be found here: https://docs.google.com/document/d/13AaCf5fczYTDHZ4G1mgYL685FqbnoEhgo0cdwuJlZmw/edit?usp=sharing When considering these options, please consider these four dimensions: 1 Consistency API/config should follow a consistent model and should not have contradicted logic beneath 2 Simplicity API should be easy to use and not introduce too much burden on users 3. Explicity API/config should not contain implicit assumptions and should be intuitive to users 4. Extensibility With foreseen future, whether the current setting can be easily extended Please let us know what do you think and please keep the discussion in this mailing thread. [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-158%3A+Generalized+incremental+checkpoints Best Yuan
