+1 to the idea, I don't have any comments. Best, Piotrek
czw., 7 gru 2023 o 07:15 Alan Sheinberg <asheinb...@confluent.io.invalid> napisał(a): > > > > Nicely written and makes sense. The only feedback I have is around the > > naming of the generalization, e.g. "Specifically, PythonCalcSplitRuleBase > > will be generalized into RemoteCalcSplitRuleBase." This naming seems to > > imply/suggest that all Async functions are remote. I wonder if we can > find > > another name which doesn't carry that connotation; maybe > > AsyncCalcSplitRuleBase. (An AsyncCalcSplitRuleBase which handles Python > > and Async functions seems reasonable.) > > > Thanks. That's fair. I agree that "Remote" isn't always accurate. I > believe that the python calls are also done asynchronously, so that might > be a reasonable name, so long as there's no confusion between the base and > async child class. > > On Wed, Dec 6, 2023 at 3:48 PM Jim Hughes <jhug...@confluent.io.invalid> > wrote: > > > Hi Alan, > > > > Nicely written and makes sense. The only feedback I have is around the > > naming of the generalization, e.g. "Specifically, PythonCalcSplitRuleBase > > will be generalized into RemoteCalcSplitRuleBase." This naming seems to > > imply/suggest that all Async functions are remote. I wonder if we can > find > > another name which doesn't carry that connotation; maybe > > AsyncCalcSplitRuleBase. (An AsyncCalcSplitRuleBase which handles Python > > and Async functions seems reasonable.) > > > > Cheers, > > > > Jim > > > > On Wed, Dec 6, 2023 at 5:45 PM Alan Sheinberg > > <asheinb...@confluent.io.invalid> wrote: > > > > > I'd like to start a discussion of FLIP-400: AsyncScalarFunction for > > > asynchronous scalar function support [1] > > > > > > This feature proposes adding a new UDF type AsyncScalarFunction which > is > > > invoked just like a normal ScalarFunction, but is implemented with an > > > asynchronous eval method. I had brought this up including the > motivation > > > in a previous discussion thread [2]. > > > > > > The purpose is to achieve high throughput scalar function UDFs while > > > allowing that an individual call may have high latency. It allows > > scaling > > > up the parallelism of just these calls without having to increase the > > > parallelism of the whole query (which could be rather resource > > > inefficient). > > > > > > In practice, it should enable SQL integration with external services > and > > > systems, which Flink has limited support for at the moment. It should > > also > > > allow easier integration with existing libraries which use asynchronous > > > APIs. > > > > > > Looking forward to your feedback and suggestions. > > > > > > [1] > > > > > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-400%3A+AsyncScalarFunction+for+asynchronous+scalar+function+support > > > < > > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-400%3A+AsyncScalarFunction+for+asynchronous+scalar+function+support > > > > > > > > > > [2] https://lists.apache.org/thread/bn153gmcobr41x2nwgodvmltlk810hzs > > > <https://lists.apache.org/thread/bn153gmcobr41x2nwgodvmltlk810hzs> > > > > > > Thanks, > > > Alan > > > > > >
