Re: create savepoint on bounded source in streaming mode

2022-01-26 Thread Shawn Du
DataStream API -- Sender:Guowei Ma Sent At:2022 Jan. 26 (Wed.) 21:51 Recipient:Shawn Du Cc:user Subject:Re: create savepoint on bounded source in streaming mode Hi, Shawn Thank you for your sharing. Unfortunately I do

Re: create savepoint on bounded source in streaming mode

2022-01-26 Thread Guowei Ma
ent At:2022 Jan. 26 (Wed.) 19:50 > Recipient:Shawn Du > Cc:user > Subject:Re: create savepoint on bounded source in streaming mode > > Hi,Shawn > > You want to use the correct state(n-1) for day n-1 and the full amount of > data for day n to produce the correct state(n) for da

Re: create savepoint on bounded source in streaming mode

2022-01-26 Thread Shawn Du
right! -- Sender:Guowei Ma Sent At:2022 Jan. 26 (Wed.) 19:50 Recipient:Shawn Du Cc:user Subject:Re: create savepoint on bounded source in streaming mode Hi,Shawn You want to use the correct state(n-1) for day n-1 and the

Re: create savepoint on bounded source in streaming mode

2022-01-26 Thread Guowei Ma
r:Guowei Ma > Sent At:2022 Jan. 26 (Wed.) 15:39 > > Recipient:Shawn Du > Cc:user > Subject:Re: create savepoint on bounded source in streaming mode > > Hi Shawn > Currently Flink can not trigger the sp at the end of the input. An > alternative way might be that you need to dev

Re: create savepoint on bounded source in streaming mode

2022-01-26 Thread Shawn Du
Subject:Re: create savepoint on bounded source in streaming mode Hi Shawn Currently Flink can not trigger the sp at the end of the input. An alternative way might be that you need to develop a customized source, which triggers a savepoint when it notices that all the input split has been handled

Re: create savepoint on bounded source in streaming mode

2022-01-26 Thread Shawn Du
a Sent At:2022 Jan. 26 (Wed.) 14:04 Recipient:Shawn Du Cc:user Subject:Re: create savepoint on bounded source in streaming mode Hi, Shawn I think Flink does not support this mechanism yet. Would you like to share the scenario in which you need this savepoint at the end of the bounded input? Bes

Re: create savepoint on bounded source in streaming mode

2022-01-25 Thread Dawid Wysakowicz
   Thanks. -- Sender:Guowei Ma Sent At:2022 Jan. 26 (Wed.) 14:04 Recipient:Shawn Du Cc:user Subject:Re: create savepoint on bounded source in streaming mode Hi, Shawn I think Flink does not support this mechanism yet. Would you like to share the scenari

Re: create savepoint on bounded source in streaming mode

2022-01-25 Thread Guowei Ma
any ideas? > >Thanks. > > -- > Sender:Guowei Ma > Sent At:2022 Jan. 26 (Wed.) 14:04 > Recipient:Shawn Du > Cc:user > Subject:Re: create savepoint on bounded source in streaming mode > > Hi, Shawn > I think Fl

Re: create savepoint on bounded source in streaming mode

2022-01-25 Thread Shawn Du
rebuild the state from a point. we call this as a bootstrap process. any ideas? Thanks. -- Sender:Guowei Ma Sent At:2022 Jan. 26 (Wed.) 14:04 Recipient:Shawn Du Cc:user Subject:Re: create savepoint on bounded source

Re: create savepoint on bounded source in streaming mode

2022-01-25 Thread Guowei Ma
Hi, Shawn I think Flink does not support this mechanism yet. Would you like to share the scenario in which you need this savepoint at the end of the bounded input? Best, Guowei On Wed, Jan 26, 2022 at 1:50 PM Shawn Du wrote: > Hi experts, > > assume I have several files and I want replay these

create savepoint on bounded source in streaming mode

2022-01-25 Thread Shawn Du
Hi experts, assume I have several files and I want replay these files in order in streaming mode and create a savepoint when files play at the end. it is possible? I wrote a simple test app, and job are finished when source is at the end. I have no chance to creat a savepoint. please help. Than