Re: [DISCUSS] Implementing SQL remote functions

2023-10-19 Thread Alan Sheinberg
Hi Feng, Krzysztof Sorry for the delayed response. FLIP-313 seems to be a good solution for the TableFunction use case. It would be nice to also have async support for ScalarFunction, AggregateFunction, and TableAggregateFunction too ideally, though this is a lot of work. ScalarFunction is real

Re: [DISCUSS] Implementing SQL remote functions

2023-09-27 Thread Krzysztof Zarzycki
Hello Alan, At my company we implemented an open source Flink HTTP connector, that you might find interesting. It can be represented as a source table as well and be used in lookups. Here is the link https://github.com/getindata/flink-http-connector Best Krzysztof On Thu, Sep 21, 2023 at 7:34 AM

Re: [DISCUSS] Implementing SQL remote functions

2023-09-20 Thread Feng Jin
Hi Alan I believe that supporting asynchronous UDF is a valuable feature. Currently, there is a similar FLIP[1] available: Can this meet your needs? [1]. https://cwiki.apache.org/confluence/display/FLINK/FLIP-313%3A+Add+support+of+User+Defined+AsyncTableFunction Best, Feng On Thu, Sep 21, 2023

Re: [DISCUSS] Implementing SQL remote functions

2023-09-20 Thread Alan Sheinberg
Hi Ron, Thanks for your response. I've answered some of your questions below. I think one solution is to support Mini-Batch Lookup Join by the framework > layer, do a RPC call by a batch input row, which can improve throughput. Would the idea be to collect a batch and then do a single RPC (or

Re: [DISCUSS] Implementing SQL remote functions

2023-09-18 Thread liu ron
Hi, Alan Thanks for driving this proposal. It sounds interesting. Regarding implementing the Remote Function, can you go into more detail about your idea, how we should support it, and how users should use it, from API design to semantic explanation?and how does the remote function help to solve y