Hi! Checkpoints do not really hold data processing. The only synchronous call is to create a "state snapshot". Some state backends implement that synchronously (like the FsStateBackend) some asynchronously (RocksDBStateBackend).
It depends on the state backend how much is synchronous (blocking) vs asynchronous. The checkpoint algorithm itself allows for asynchronous persistence. Stephan On Sat, Oct 29, 2016 at 6:05 AM, Renjie Liu <liurenjie2...@gmail.com> wrote: > Hi, flink dev team: > > I'm reading flink's source code and it seems that the process of saving > checkpointing data to external storage system will block data processing, > and I wonder why this has to be designed like this? Isn't is a potential > risk that may increase latency of processing? > > Is the following strategy better? > > 1. Save state data to local disk > 2. Resume data processing > 3. Saving state data to external system asynchronously > > -- > Liu, Renjie > Software Engineer, MVAD >
