Hi, Shuai.

This is a valuable addition to the current AsyncLookupJoin, and I’m 

generally in favor of it. 




I have one question. Why do we need to introduce additional parameters 

to control KEY_ORDERED and ALLOW_UNORDERED? In other words, 

what scenarios require allowing users to perform completely unordered 

async lookup joins in the presence of an upsert key?




--

    Best!
    Xuyang





在 2025-04-11 10:39:46,"shuai xu" <xushuai...@gmail.com> 写道:
>Hi all,
>
>This FLIP will primarily focus on the implementation within the table module. 
>As for support in the DataStream API, it will be addressed in a separate FLIP.
>
>> 2025年4月8日 09:57,shuai xu <xushuai...@gmail.com> 写道:
>> 
>> Hi devs,
>> 
>> I'd like to start a discussion on FLIP-519: Introduce async lookup key
>> ordered mode[1].
>> 
>> The Flink system currently supports both record-level ordered and
>> unordered output modes for asynchronous lookup joins. However, it does
>> not guarantee the processing order of records sharing the same key.
>> 
>> As highlighted in [2], there are two key requirements for enhancing
>> async io operations:
>> 1. Ensuring the processing order of records with the same key is a
>> common requirement in DataStream.
>> 2. Sequential processing of records sharing the same upsertKey when
>> performing lookup join in Flink SQL is essential for maintaining
>> correctness.
>> 
>> This optimization aims to balance correctness and performance for
>> stateful streaming workloads.Then the FLIP introduce a new operator
>> KeyedAsyncWaitOperator to supports the optimization. Besides, a new
>> option is added to control the behaviour avoid influencing existing
>> jobs.
>> 
>> please find more details in the FLIP wiki document[1]. Looking forward
>> to your feedback.
>> 
>> [1] 
>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-519%3A++Introduce+async+lookup+key+ordered+mode
>> [2] https://lists.apache.org/thread/wczzjhw8g0jcbs8lw2jhtrkw858cmx5n
>> 
>> Best,
>> Xu Shuai

Reply via email to