+1 Glad to see the kickoff of this discussion. Thanks Lihe for driving this!
We have actually already discussed it internally a few months ago. After considering some corner cases, all agreed on enabling the incremental checkpoint as default. Best regards, Jing On Mon, Jun 13, 2022 at 12:17 PM Yun Tang <myas...@live.com> wrote: > Strongly +1 for making incremental checkpoints as default. Many users have > ever been asking why this configuration is not enabled by default. > > BTW, from my knowledge, nothing would happen for HashMapStateBackend, > which does not support incremental checkpoint yet, when enabling > incremental checkpoints. > > > Best > Yun Tang > ________________________________ > From: Martijn Visser <martijnvis...@apache.org> > Sent: Monday, June 13, 2022 18:05 > To: dev@flink.apache.org <dev@flink.apache.org> > Subject: Re: [DISCUSS ] Make state.backend.incremental as true by default > > Hi Lihe, > > What happens if we enable incremental checkpoints by default while the used > memory backend is HashMapStateBackend, which doesn't support incremental > checkpoints? > > Best regards, > > Martijn > > Op ma 13 jun. 2022 om 11:59 schreef Lihe Ma <ma_l...@163.com>: > > > Hi, Everyone, > > > > I would like to open a discussion on setting incremental checkpoint as > > default behavior. > > > > Currently, the configuration of state.backend.incremental is set as false > > by default. Incremental checkpoint has been adopted widely in industry > > community for many years , and it is also well-tested from the feedback > in > > the community discussion. Incremental checkpointing is more > light-weighted: > > shorter checkpoint duration, less uploaded data and less resource > > consumption. > > > > In terms of backward compatibility, enable incremental checkpointing > would > > not make any data loss no matter restoring from a full > checkpoint/savepoint > > or an incremental checkpoint. > > > > FLIP-193 (Snapshot ownership)[1] has been released in 1.15, incremental > > checkpoint no longer depends on a previous restored checkpoint in default > > NO_CLAIM mode, which makes the checkpoint lineage much cleaner, it is a > > good chance to change the configuration state.backend.incremental to true > > as default. > > > > Thus, based on the above discussion, I suggest to make > > state.backend.incremental as true by default. What do you think of this > > proposal? > > > > [1] > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-193%3A+Snapshots+ownership > > > > Best regards, > > Lihe Ma > > >
