Re: Check pointing for simple pipeline

2020-07-09 Thread Dawid Wysakowicz
gt; *Sent:* Tuesday, July 7, 2020 20:43 > *To:* d...@flink.apache.org ; user > > *Subject:* Check pointing for simple pipeline >   > Hi , > > I have pipeline. Source-> Map(JSON transform)-> Sink..  > > Both source and sink are Kafka.  > > What is the best checkpoint ing me

Re: Check pointing for simple pipeline

2020-07-07 Thread Yun Tang
: d...@flink.apache.org ; user Subject: Check pointing for simple pipeline Hi , I have pipeline. Source-> Map(JSON transform)-> Sink.. Both source and sink are Kafka. What is the best checkpoint ing mechanism? Is setting checkpoints incremental a good option? What should be careful of?

Check pointing for simple pipeline

2020-07-07 Thread Prasanna kumar
Hi , I have pipeline. Source-> Map(JSON transform)-> Sink.. Both source and sink are Kafka. What is the best checkpoint ing mechanism? Is setting checkpoints incremental a good option? What should be careful of? I am running it on aws emr. Will checkpoint slow the speed? Thanks, Prasanna.