Re: State life-cycle for different state-backend implementations

2018-06-13 Thread Rinat
Hi Sihua, Thx for your reply > On 9 Jun 2018, at 11:42, sihua zhou wrote: > > Hi Rinat, > > I think there is one configuration {{state.checkpoints.num-retained}} to > control the maximum number of completed checkpoints to retain, the default > value is 1. So the risk you mentioned should not

State life-cycle for different state-backend implementations

2018-06-08 Thread Rinat
Hi mates, got a question about different state backends. As I've properly understood, on every checkpoint, Flink flushes it’s current state into backend. In case of FsStateBackend we’ll have a separate file for each checkpoint, and during the job lifecycle we got a risk of a huge amount of stat