Re: Re: Re: [DISCUSS ] Make state.backend.incremental as true by default

2022-07-12 Thread Yun Tang
avoid introducing such new configurations unless we have to. Best Yun Tang From: Yuan Mei Sent: Monday, July 11, 2022 11:53 To: dev Cc: ro...@apache.org Subject: Re: Re: Re: [DISCUSS ] Make state.backend.incremental as true by default Hey Lihe Ma, 1. I do not

Re: Re: Re: [DISCUSS ] Make state.backend.incremental as true by default

2022-07-10 Thread Yuan Mei
Hey Lihe Ma, 1. I do not think making different state backends with different default setups (configs) is a bad idea. I have some similar ideas as Zakelly, Hangxiang and you mentioned above. Think about this: even though the incremental checkpoint for HashMap Statebackend is released in 1.16, it m

Re: Re: Re: [DISCUSS ] Make state.backend.incremental as true by default

2022-06-26 Thread Hangxiang Yu
The suggestion of Zakelly (Using noDefaultValue to make different StateBackend have its own default value) makes sense to me. I haven't gotten a better idea about it. Maybe @ro...@apache.org who is the owner of incremental checkpoint support of HashMapStateBackend could share more ideas about it.

Re: Re: [DISCUSS ] Make state.backend.incremental as true by default

2022-06-19 Thread Yuan Mei
Hey @Lihe Ma, Thanks and I appreciate your feedback! *What do I think of introducing a new state backend?* I am hesitant to follow this way of introducing a new state backend to implement `HashMapStateBackend with incremental checkpoint enabled`, because: - It causes more confusion for users: why

Re: Re: [DISCUSS ] Make state.backend.incremental as true by default

2022-06-16 Thread LuNing Wang
Strongly +1 Best, LuNing Wang Zakelly Lan 于2022年6月17日周五 00:15写道: > Thanks for bringing this up. > I'm +1 on enabling the incremental checkpoint by default on RocksDB. But I > also agree with Yuan about not enabling this on newly implemented > incremental checkpoint for hashmap statebackend. >

Re: Re: [DISCUSS ] Make state.backend.incremental as true by default

2022-06-16 Thread Zakelly Lan
Thanks for bringing this up. I'm +1 on enabling the incremental checkpoint by default on RocksDB. But I also agree with Yuan about not enabling this on newly implemented incremental checkpoint for hashmap statebackend. I am wondering can we make it behave differently for different state backends w