+1 >From my experience, it is actually hard to come up with use cases where incremental checkpoints should explicitly not be enabled with the RocksDB state backend. If the state is so small that the full snapshots do not have any negative impact, one should consider using HashMapStateBackend anyway.
Best, Alexander Fedulov On Mon, Jun 13, 2022 at 12:26 PM Jing Ge <j...@ververica.com> wrote: > +1 > > Glad to see the kickoff of this discussion. Thanks Lihe for driving this! > > We have actually already discussed it internally a few months ago. After > considering some corner cases, all agreed on enabling the incremental > checkpoint as default. > > Best regards, > Jing > > On Mon, Jun 13, 2022 at 12:17 PM Yun Tang <myas...@live.com> wrote: > > > Strongly +1 for making incremental checkpoints as default. Many users > have > > ever been asking why this configuration is not enabled by default. > > > > BTW, from my knowledge, nothing would happen for HashMapStateBackend, > > which does not support incremental checkpoint yet, when enabling > > incremental checkpoints. > > > > > > Best > > Yun Tang > > ________________________________ > > From: Martijn Visser <martijnvis...@apache.org> > > Sent: Monday, June 13, 2022 18:05 > > To: dev@flink.apache.org <dev@flink.apache.org> > > Subject: Re: [DISCUSS ] Make state.backend.incremental as true by default > > > > Hi Lihe, > > > > What happens if we enable incremental checkpoints by default while the > used > > memory backend is HashMapStateBackend, which doesn't support incremental > > checkpoints? > > > > Best regards, > > > > Martijn > > > > Op ma 13 jun. 2022 om 11:59 schreef Lihe Ma <ma_l...@163.com>: > > > > > Hi, Everyone, > > > > > > I would like to open a discussion on setting incremental checkpoint as > > > default behavior. > > > > > > Currently, the configuration of state.backend.incremental is set as > false > > > by default. Incremental checkpoint has been adopted widely in industry > > > community for many years , and it is also well-tested from the feedback > > in > > > the community discussion. Incremental checkpointing is more > > light-weighted: > > > shorter checkpoint duration, less uploaded data and less resource > > > consumption. > > > > > > In terms of backward compatibility, enable incremental checkpointing > > would > > > not make any data loss no matter restoring from a full > > checkpoint/savepoint > > > or an incremental checkpoint. > > > > > > FLIP-193 (Snapshot ownership)[1] has been released in 1.15, incremental > > > checkpoint no longer depends on a previous restored checkpoint in > default > > > NO_CLAIM mode, which makes the checkpoint lineage much cleaner, it is a > > > good chance to change the configuration state.backend.incremental to > true > > > as default. > > > > > > Thus, based on the above discussion, I suggest to make > > > state.backend.incremental as true by default. What do you think of this > > > proposal? > > > > > > [1] > > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-193%3A+Snapshots+ownership > > > > > > Best regards, > > > Lihe Ma > > > > > >
