Re: Flink Checkpoint for Stateless Operators

Hi Raghavendar, It sounds like you don't actually have flatMap logic, in which case you should use a sink instead of a flatMap. And probably one of the existing ones, as some of them already provide exactly-once guarantee [1]. [1] https://ci.apache.org/projects/flink/flink-docs-stable/dev/connect

Re: Flink Checkpoint for Stateless Operators

Hi Roman I am just doing write operations from the flat map. Does it matter If I use a flap map or sink for this purpose? Thank you Virus-free. www.avast.com

Re: Flink Checkpoint for Stateless Operators

Flink uses checkpoint barriers that are sent through along the same channels as data. Events are included into the checkpoint if they precede the corresponding barrier (or the RPC call for sources). [1] is the algorithm description and [2] is about integration with Kafka. > In my example, I have o

Re: Flink Checkpoint for Stateless Operators

Hi Roman In general, how Flink tracks the events from source to downstream operators? We usually emit existing events from an operator or create a new instance of a class and emit it. How does Flink or Flink source know whether the events are which snapshot? > So you don't need to re-process it m

Re: Flink Checkpoint for Stateless Operators

Hi Raghavendar, In Flink, checkpoints are global, meaning that a checkpoint is successful only if all operators acknowledge it. So the offset will be stored in state and then committed to Kafka [1] only after all the tasks acknowledge that checkpoint. At that moment, the element must be either emi