Hi, With:
> Can not you take into account the pending element that’s stuck somewhere in > the transit? Snapshot it as well and during recovery reprocess it? This is > exactly that’s AsyncWaitOperator is doing. I didn’t mean for you to use AsynWaitOperator, but what both me and Arvid suggested you previously: > Also as Kostas pointed out, the easiest way would be to try use > AsyncWaitOperator. If that’s not possible, you can implement your custom > logic based on its code. You can copy/duplicate & modify/adjust the AsyncWaitOperator logic inside your custom operator. You don’t have to use it if you have some special requirements, you can implement your own custom logic. Specifically I meant to mimic org.apache.flink.streaming.api.operators.async.AsyncWaitOperator#queue Field and how is it being used during snapshotting state & recovery. Piotrek > On 9 Apr 2020, at 06:10, Salva Alcántara <salcantara...@gmail.com> wrote: > > I agree with your point Piotrek, AsyncIO would handle all the pending data > for me. However, the reason why I did not want to use it is because in my > case, the callbacks are not always called in response of new data being sent > to the third party lib. Indeed, the callback will be called rather > uncommonly (since in my case it will mean that an anomaly has been > detected). This means that If I go with AsyncIO I will need to setup a max > timeout for every element, when only a few of them will actuallyinvoke the > callback (i.e., produce any data in response). This seems rather drastic > because it will probably add too much latency unnecessarily, but I agree on > that maybe there is no other way if I need exactly once guarantees. > > > > -- > Sent from: > http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
