Hi Yun, Thanks for your reply. I am sorry if I was not clear . What I meant was that as records are processed, the checkpoint and the corresponding stream position will move. But in case of async with unordered, will the checkpoint and the corresponding stream position move in the above scenario where Tasks T1....T100 are currently running in parallel and all tasks T1...T100 pass except for T50. Many thanks. Anurag
On Thu, Oct 10, 2019 at 11:13 AM Yun Tang <myas...@live.com> wrote: > Hi Anurag > > What do you mean "will the checkpoint pointer move at all or not"? > > If one of your thread failed to send record, and if it would cause that > sub-task to fail, it would lead to the job failover. When job failover, any > on-going checkpoint would be aborted and job would then just restore from > last latest checkpoint. > If failing to send record would not cause that sub-task to fail, nothing > would happen and job continues to run but this might be not what you want. > > Best > Yun Tang > ------------------------------ > *From:* anurag <anurag.guj...@gmail.com> > *Sent:* Friday, October 11, 2019 2:03 > *To:* user@flink.apache.org <user@flink.apache.org> > *Subject:* Async and checkpointing > > Hi All, > Thanks for your help in advance. I am using async I/O with flink 1.5.5, I > am using > > AsyncDataStream.unorderedWait method also my capacity is set to 100.My > question is since my capacity is 100, each thread will be processing one > record.Say sequence number of my records is S1,S2....S100 and say at a > particular point in time , thread T1 is processing record S1, T2 is > processing record S2 and T100 is processing record S100 and say all the > threads T1..T100 except T50 failed when sending the record to the Sink. In > this will the checkpoint pointer move at all or not. > > Apologies in advance if my question is not clear. > > Thanks, > Anurag > >
