Hi all, I have also added a 'Follow up' section at the end of the FLIP-425[1] describing this.
[1] https://cwiki.apache.org/confluence/x/S4p3EQ Best, Zakelly On Wed, Apr 9, 2025 at 12:42 PM Zakelly Lan <zakelly....@gmail.com> wrote: > Thanks for driving this! > > +1 for the FLIP given there is a solid user case behind. > > Shuai and I had a discussion and we agree that `KeyedAsyncWaitOperator` > in current FLIP shares similar functionality with the `Asynchronous > Execution Model (AEC)` introduced in FLIP-425[1]. We think it is better to > generalize the AEC for all keyed ordered cases, not only for state access. > So I'd make this happen after the approval of this FLIP. Hope this helps > all the similar operators to implement. > > [1] https://cwiki.apache.org/confluence/x/S4p3EQ > > > Best, > Zakelly > > On Tue, Apr 8, 2025 at 10:00 AM shuai xu <xushuai...@gmail.com> wrote: > >> Hi devs, >> >> I'd like to start a discussion on FLIP-519: Introduce async lookup key >> ordered mode[1]. >> >> The Flink system currently supports both record-level ordered and >> unordered output modes for asynchronous lookup joins. However, it does >> not guarantee the processing order of records sharing the same key. >> >> As highlighted in [2], there are two key requirements for enhancing >> async io operations: >> 1. Ensuring the processing order of records with the same key is a >> common requirement in DataStream. >> 2. Sequential processing of records sharing the same upsertKey when >> performing lookup join in Flink SQL is essential for maintaining >> correctness. >> >> This optimization aims to balance correctness and performance for >> stateful streaming workloads.Then the FLIP introduce a new operator >> KeyedAsyncWaitOperator to supports the optimization. Besides, a new >> option is added to control the behaviour avoid influencing existing >> jobs. >> >> please find more details in the FLIP wiki document[1]. Looking forward >> to your feedback. >> >> [1] >> https://cwiki.apache.org/confluence/display/FLINK/FLIP-519%3A++Introduce+async+lookup+key+ordered+mode >> [2] https://lists.apache.org/thread/wczzjhw8g0jcbs8lw2jhtrkw858cmx5n >> >> Best, >> Xu Shuai >> >
