Hi Biao, Thanks for driving this FLIP. It's meaningful to support speculative execution of sinks is important.
I have a question about this: Speculative execution of Committer will be disabled. I agree with your point and I saw the similar requirements to disable speculative execution for specified operators. However the requirement is not supported currently. I think there should be some place to describe how to support it. Best, Jing Zhang Lijie Wang <wangdachui9...@gmail.com> 于2022年12月27日周二 18:51写道： > Hi Biao, > > Thanks for driving this FLIP. > In this FLIP, it introduces "int getFinishedAttempt(int subtaskIndex)" for > OutputFormat to know which subtask attempt is the one marked as finished by > JM and commit the right data. > I have a question about this: does SinkV2 need to do the same thing? Should > we/users do it in the committer? If yes, how does the commiter know which > one is the right subtask attempt? > > Best, > Lijie > > yuxia <luoyu...@alumni.sjtu.edu.cn> 于2022年12月27日周二 10:01写道： > > > HI, Biao. > > Thanks for driving this FLIP. > > After quick look of this FLIP, I have a question about "expose the > attempt > > number which can be used to isolate data produced by speculative > executions > > with the same subtask id". > > What the sink expect to do to isolate data produced by speculative > > executions? IIUC, if the taks failover, it also generate a new attempt. > > Does it make difference in isolating data produced? > > > > Best regards, > > Yuxia > > > > ----- 原始邮件 ----- > > 发件人: "Biao Liu" <mmyy1...@gmail.com> > > 收件人: "dev" <dev@flink.apache.org> > > 发送时间: 星期四, 2022年 12 月 22日 下午 8:16:21 > > 主题: [DISCUSS] FLIP-281 Sink Supports Speculative Execution For Batch Job > > > > Hi everyone, > > > > I would like to start a discussion about making Sink support speculative > > execution for batch jobs. This proposal is a follow up of "FLIP-168: > > Speculative Execution For Batch Job"[1]. Speculative execution is very > > meaningful for batch jobs. And it would be more complete after supporting > > speculative execution of Sink. Please find more details in the FLIP > > document > > [2]. > > > > Looking forward to your feedback. > > [1] > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-168%3A+Speculative+Execution+for+Batch+Job > > [2] > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-281+Sink+Supports+Speculative+Execution+For+Batch+Job > > > > Thanks, > > Biao /'bɪ.aʊ/ > > >
