Hi Jing,
> "Do you mean to support the AyncTableFunction beyond the
LookupTableSource?"
Yes, I mean to support the AyncTableFunction beyond the LookupTableSource.
The "AsyncTableFunction" is the function with ability to be executed async
(with AsyncWaitOperator).
The async lookup join is a one of usage of this. So, we don't have to bind
the AyncTableFunction with LookupTableSource.
If User-defined AsyncTableFunction is supported, user can directly use
lateral table syntax to perform async operation.
> "It would be better if you could elaborate the proposed changes wrt the
CorrelatedCodeGenerator with more details"
In the proposal, we use lateral table syntax to support the async table
function. So the planner will also treat this statement to a
CommonExecCorrelate node. So the runtime code should be generated in
CorrelatedCodeGenerator.
In CorrelatedCodeGenerator, we will know the TableFunction's Kind of
`FunctionKind.Table` or `FunctionKind.ASYNC_TABLE`
For `FunctionKind.ASYNC_TABLE` we can generate a AsyncWaitOperator to
execute the async table function.
Thanks,
Aitozi.
Jing Ge <j...@ververica.com.invalid> 于2023年5月29日周一 03:22写道：
> Hi Aitozi,
>
> Thanks for the clarification. The naming "Lookup" might suggest using it
> for table look up. But conceptually what the eval() method will do is to
> get a collection of results(Row, RowData) from the given keys. How it will
> be done depends on the implementation, i.e. you can implement your own
> Source[1][2]. The example in the FLIP should be able to be handled in this
> way.
>
> Do you mean to support the AyncTableFunction beyond the LookupTableSource?
> It would be better if you could elaborate the proposed changes wrt the
> CorrelatedCodeGenerator with more details. Thanks!
>
> Best regards,
> Jing
>
> [1]
>
> https://github.com/apache/flink/blob/678370b18e1b6c4a23e5ce08f8efd05675a0cc17/flink-table/flink-table-common/src/main/java/org/apache/flink/table/connector/source/LookupTableSource.java#L64
> [2]
>
> https://github.com/apache/flink/blob/678370b18e1b6c4a23e5ce08f8efd05675a0cc17/flink-table/flink-table-common/src/main/java/org/apache/flink/table/connector/source/AsyncTableFunctionProvider.java#L49
>
> On Sat, May 27, 2023 at 9:48 AM Aitozi <gjying1...@gmail.com> wrote:
>
> > Hi Jing,
> > Thanks for your response. As stated in the FLIP, the purpose of this
> > FLIP is meant to support
> > user-defined async table function. As described in flink document [1]
> >
> > Async table functions are special functions for table sources that
> perform
> > > a lookup.
> > >
> >
> > So end user can not directly define and use async table function now. An
> > user case is reported in [2]
> >
> > So, in conclusion, no new interface is introduced, but we extend the
> > ability to support user-defined async table function.
> >
> > [1]:
> >
> >
> https://nightlies.apache.org/flink/flink-docs-release-1.17/docs/dev/table/functions/udfs/
> > [2]: https://lists.apache.org/thread/qljwd40v5ntz6733cwcdr8s4z97b343b
> >
> > Thanks.
> > Aitozi.
> >
> >
> > Jing Ge <j...@ververica.com.invalid> 于2023年5月27日周六 06:40写道：
> >
> > > Hi Aitozi,
> > >
> > > Thanks for your proposal. I am not quite sure if I understood your
> > thoughts
> > > correctly. You described a special case implementation of the
> > > AsyncTableFunction with on public API changes. Would you please
> elaborate
> > > your purpose of writing a FLIP according to the FLIP documentation[1]?
> > > Thanks!
> > >
> > > [1]
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/Flink+Improvement+Proposals
> > >
> > > Best regards,
> > > Jing
> > >
> > > On Wed, May 24, 2023 at 1:07 PM Aitozi <gjying1...@gmail.com> wrote:
> > >
> > > > May I ask for some feedback :D
> > > >
> > > > Thanks,
> > > > Aitozi
> > > >
> > > > Aitozi <gjying1...@gmail.com> 于2023年5月23日周二 19:14写道：
> > > > >
> > > > > Just catch an user case report from Giannis Polyzos for this usage:
> > > > >
> > > > > https://lists.apache.org/thread/qljwd40v5ntz6733cwcdr8s4z97b343b
> > > > >
> > > > > Aitozi <gjying1...@gmail.com> 于2023年5月23日周二 17:45写道：
> > > > > >
> > > > > > Hi guys,
> > > > > > I want to bring up a discussion about adding support of User
> > > > > > Defined AsyncTableFunction in Flink.
> > > > > > Currently, async table function are special functions for table
> > > source
> > > > > > to perform
> > > > > > async lookup. However, it's worth to support the user defined
> async
> > > > > > table function.
> > > > > > Because, in this way, the end SQL user can leverage it to perform
> > the
> > > > > > async operation
> > > > > > which is useful to maximum the system throughput especially for
> IO
> > > > > > bottleneck case.
> > > > > >
> > > > > > You can find some more detail in [1].
> > > > > >
> > > > > > Looking forward to feedback
> > > > > >
> > > > > >
> > > > > > [1]:
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/%5BFLIP-313%5D+Add+support+of+User+Defined+AsyncTableFunction
> > > > > >
> > > > > > Thanks,
> > > > > > Aitozi.
> > > >
> > >
> >
>
