Re: [DISCUSS] FLIP-76: Unaligned checkpoints

2020-03-05 Thread Arvid Heise
gression in our previous POC testing. So the > > backpressure might not be the only factor to switch to the unaligned way > in > > practice I guess. Anyway your > > > inputs are valuable for us to make the final decision. > > > > > > Best, > > > Zhijiang >

Re: [DISCUSS] FLIP-76: Unaligned checkpoints

2020-02-28 Thread Yu Li
s POC testing. So the > backpressure might not be the only factor to switch to the unaligned way in > practice I guess. Anyway your > > inputs are valuable for us to make the final decision. > > > > Best, > > Zhijiang > > > > > > > > > > -- > > From:Yu Li &g

Re: [DISCUSS] FLIP-76: Unaligned checkpoints

2020-02-27 Thread Piotr Nowojski
gt; > Best, > Zhijiang > > > > > ------------------ > From:Yu Li > Send Time:2020 Feb. 26 (Wed.) 15:59 > To:dev ; Zhijiang > Subject:Re: [DISCUSS] FLIP-76: Unaligned checkpoints > > Hi Zhijiang, > >

Re: [DISCUSS] FLIP-76: Unaligned checkpoints

2020-02-26 Thread Zhijiang
iang -- From:Yu Li Send Time:2020 Feb. 26 (Wed.) 15:59 To:dev ; Zhijiang Subject:Re: [DISCUSS] FLIP-76: Unaligned checkpoints Hi Zhijiang, Thanks for the quick reply! For the 1st question, please allow me to confirm, that when doing asynchro

Re: [DISCUSS] FLIP-76: Unaligned checkpoints

2020-02-25 Thread Yu Li
onsider if really > necessary in future. > > Best, > Zhijiang > > > ------ > From:Yu Li > Send Time:2020 Feb. 26 (Wed.) 12:59 > To:dev > Subject:Re: [DISCUSS] FLIP-76: Unaligned checkpoints > > Hi All

Re: [DISCUSS] FLIP-76: Unaligned checkpoints

2020-02-25 Thread Zhijiang
, Zhijiang -- From:Yu Li Send Time:2020 Feb. 26 (Wed.) 12:59 To:dev Subject:Re: [DISCUSS] FLIP-76: Unaligned checkpoints Hi All, Sorry for being late to the discussion. I've gone through the latest FLIP document and have below ques

Re: [DISCUSS] FLIP-76: Unaligned checkpoints

2020-02-25 Thread Yu Li
d continuous spilling would have to effectively persist all of the data > anyway. > > > On 10 Oct 2019, at 19:51, Yun Tang wrote: > > > > Hi Arvid > > > > +1 for this future which has been hoped for a long time. End-to-end > exactly once job could benefit from

Re: [DISCUSS] FLIP-76: Unaligned checkpoints

2019-10-22 Thread Piotr Nowojski
> > > Best > Yun Tang > > From: Yun Gao > Sent: Thursday, October 10, 2019 18:39 > To: dev > Subject: Re: [DISCUSS] FLIP-76: Unaligned checkpoints > >Hi Arvid, > >Very thanks for bring up the discussion! From our side unable to > fi

Re: [DISCUSS] FLIP-76: Unaligned checkpoints

2019-10-10 Thread Yun Tang
not right. Best, Yun -- From:Arvid Heise Send Time:2019 Sep. 30 (Mon.) 20:31 To:dev Subject:[DISCUSS] FLIP-76: Unaligned checkpoints Hi Devs, I would like to start the formal discussion about FLIP-76 [1], which

Re: [DISCUSS] FLIP-76: Unaligned checkpoints

2019-10-10 Thread Yun Gao
. Best, Yun -- From:Arvid Heise Send Time:2019 Sep. 30 (Mon.) 20:31 To:dev Subject:[DISCUSS] FLIP-76: Unaligned checkpoints Hi Devs, I would like to start the formal discussion about FLIP-76 [1], which improves the

Re: [DISCUSS] FLIP-76: Unaligned checkpoints

2019-10-09 Thread Congxian Qiu
> > Many users would expect this feature and also +1 from my side. > > Best, > Zhijiang > -- > From:Piotr Nowojski > Send Time:2019年10月7日(星期一) 10:13 > To:dev > Subject:Re: [DISCUSS] FLIP-76: Unaligned ch

Re: [DISCUSS] FLIP-76: Unaligned checkpoints

2019-10-09 Thread zhijiang
Thanks for writing up this FLIP, Arvid! Many users would expect this feature and also +1 from my side. Best, Zhijiang -- From:Piotr Nowojski Send Time:2019年10月7日(星期一) 10:13 To:dev Subject:Re: [DISCUSS] FLIP-76: Unaligned

Re: [DISCUSS] FLIP-76: Unaligned checkpoints

2019-10-07 Thread Piotr Nowojski
Hi Arvid, Thanks for coming up with this FLIP. I think it addresses the issues raised in the previous mailing list discussion [2]. For the record: +1 from my side to implement this. Piotrek > On 30 Sep 2019, at 14:31, Arvid Heise wrote: > > Hi Devs, > > I would like to start the formal dis

[DISCUSS] FLIP-76: Unaligned checkpoints

2019-09-30 Thread Arvid Heise
Hi Devs, I would like to start the formal discussion about FLIP-76 [1], which improves the checkpoint latency in systems under backpressure, where a checkpoint can take hours to complete in the worst case. I recommend the thread "checkpointing under backpressure" [2] to get a good idea why users a