Re: [Stateful Functions] Checkpoints and AtLeastOnce Guarantee

2020-11-05 Thread Igal Shilman
How do you deploy the job currently? Are you using the data stream integration / or as a Flink Jar [1] (also please note, that the directories might be created but without checkpoint interval set, they will be empty) Regarding your two questions: That is true that you can theoretically share the

Re: [Stateful Functions] Checkpoints and AtLeastOnce Guarantee

2020-11-05 Thread Jan Brusch
Hi Igal, thanks for your quick and detailed reply! For me, this is the really great defining feature of Stateful Functions: Separating StreamProcessing "Infrastructure" from Business Logic Code, possibly maintained by a different team. Regarding your points: I did add the checkpoint interval

Re: [Stateful Functions] Checkpoints and AtLeastOnce Guarantee

2020-11-05 Thread Igal Shilman
Hi Jan, The architecture outlined by you, sounds good and we've run successfully mixed architectures like this. Let me try to address your questions: 1) To enable checkpointing you need to set the relevant values in your flink-conf.yaml file. execution.checkpointing.interval: (see [1]) state.che