Hey Yuan, thanks for the proposal I think Option 3 is the simplest to use and exposes less details than any other. It's also consistent with the current way of configuring state backends, as long as we treat change logging as a common feature applicable to any state backend, like e.g. state.backend.local-recovery.
Option 6 seems slightly less preferable as it exposes more details but I think is the most viable alternative. Regards, Roman On Mon, May 31, 2021 at 8:39 AM Yuan Mei <yuanmei.w...@gmail.com> wrote: > > Hey all, > > We would like to start a discussion on how to enable/config Changelog > Statebakcend. > > As part of FLIP-158[1], Changelog state backend wraps on top of existing > state backend (HashMapStateBackend, EmbeddedRocksDBStateBackend and may > expect more) and delegates state changes to the underlying state backends. > This thread is to discuss the problem of how Changelog StateBackend should > be enabled and configured. > > Proposed options to enable/config state changelog is listed below: > > Option 1: Enable Changelog Statebackend through a Boolean Flag > > Option 2: Enable Changelog Statebackend through a Boolean Flag + a Special > Case > > Option 3: Enable Changelog Statebackend through a Boolean Flag + W/O > ChangelogStateBackend Exposed > > Option 4: Explicit Nested Configuration + “changelog.inner” prefix for > inner backend > > Option 5: Explicit Nested Configuration + inner state backend configuration > unchanged > > Option 6: Config Changelog and Inner Statebackend All-Together > > Details of each option can be found here: > https://docs.google.com/document/d/13AaCf5fczYTDHZ4G1mgYL685FqbnoEhgo0cdwuJlZmw/edit?usp=sharing > > When considering these options, please consider these four dimensions: > 1 Consistency > API/config should follow a consistent model and should not have > contradicted logic beneath > 2 Simplicity > API should be easy to use and not introduce too much burden on users > 3. Explicity > API/config should not contain implicit assumptions and should be intuitive > to users > 4. Extensibility > With foreseen future, whether the current setting can be easily extended > > Please let us know what do you think and please keep the discussion in this > mailing thread. > > [1] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-158%3A+Generalized+incremental+checkpoints > > Best > Yuan
