Hi, Shuai Thanks for driving this proposal. The FLIP looks good to me overall, +1 for it.
I have a small question about the table's contents. When 'table.exec.async-lookup.key-ordered-enabled=true' and 'table.exec.async-lookup.output-mode=ORDERD', the lookup join in key ordered should be 'yes', not 'no'. ORDERD is naturally guaranteed to be ordered, it just doesn't depend on the current implementation of this proposal, which is a bit confusing here. Best, Ron shuai xu <xushuai...@gmail.com> 于2025年4月18日周五 10:50写道： > Hi Xuyang, > > > Thanks for your response. Actually, ALLOW_UNORDERED is only enabled when > facing > > append-only streams for higher throughput. KEY_ORDERED is designed for > scenarios > > where upsert key exists. Its goal is to achieve higher performance > compared to the ORDERED > > mode while ensuring correctness is not compromised. In a word, > ALLOW_UNORDERED > > mode does works in the presence of an upsert key. > > > Aside from the literal difference in sequence, KEY_ORDER focuses on the > processing order, > > while ALLOW_UNORDERED pertains to the output order. In ALLOW_UNORDERED > mode, > > the processing order may be either ordered or unordered. > > > Best, > Xu Shuai > > > 2025年4月17日 14:53，Xuyang <xyzhong...@163.com> 写道： > > > > Hi, Shuai. > > > > This is a valuable addition to the current AsyncLookupJoin, and I’m > > > > generally in favor of it. > > > > > > > > > > I have one question. Why do we need to introduce additional parameters > > > > to control KEY_ORDERED and ALLOW_UNORDERED? In other words, > > > > what scenarios require allowing users to perform completely unordered > > > > async lookup joins in the presence of an upsert key? > > > > > > > > > > -- > > > > Best！ > > Xuyang > > > > > > > > > > > > 在 2025-04-11 10:39:46，"shuai xu" <xushuai...@gmail.com> 写道： > >> Hi all, > >> > >> This FLIP will primarily focus on the implementation within the table > module. As for support in the DataStream API, it will be addressed in a > separate FLIP. > >> > >>> 2025年4月8日 09:57，shuai xu <xushuai...@gmail.com> 写道： > >>> > >>> Hi devs, > >>> > >>> I'd like to start a discussion on FLIP-519: Introduce async lookup key > >>> ordered mode[1]. > >>> > >>> The Flink system currently supports both record-level ordered and > >>> unordered output modes for asynchronous lookup joins. However, it does > >>> not guarantee the processing order of records sharing the same key. > >>> > >>> As highlighted in [2], there are two key requirements for enhancing > >>> async io operations: > >>> 1. Ensuring the processing order of records with the same key is a > >>> common requirement in DataStream. > >>> 2. Sequential processing of records sharing the same upsertKey when > >>> performing lookup join in Flink SQL is essential for maintaining > >>> correctness. > >>> > >>> This optimization aims to balance correctness and performance for > >>> stateful streaming workloads.Then the FLIP introduce a new operator > >>> KeyedAsyncWaitOperator to supports the optimization. Besides, a new > >>> option is added to control the behaviour avoid influencing existing > >>> jobs. > >>> > >>> please find more details in the FLIP wiki document[1]. Looking forward > >>> to your feedback. > >>> > >>> [1] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-519%3A++Introduce+async+lookup+key+ordered+mode > >>> [2] https://lists.apache.org/thread/wczzjhw8g0jcbs8lw2jhtrkw858cmx5n > >>> > >>> Best, > >>> Xu Shuai > >
