Hi all, I had a really quick look and from my perspective the proposal looks fine. I share Jarks opinion that the instantiation could be done at a later stage. I agree with Wei it requires some changes in the internal implementation of the FunctionCatalog, to store temporary functions as catalog functions instead of FunctionDefinitions, but we have that on our agenda anyway. I would suggest investigating if we could do that as part of this flip already. Nevertheless this in theory can be also done later.
Best, Dawid On Mon, 2 Mar 2020, 14:58 Jark Wu, <imj...@gmail.com> wrote: > Thanks for the explanation, Wei! > > On Mon, 2 Mar 2020 at 20:59, Wei Zhong <weizhong0...@gmail.com> wrote: > > > Hi Jark, > > > > Thanks for your suggestion. > > > > Actually, the timing of starting a Python process depends on the UDF > type, > > because the Python process is used to provide the necessary information > to > > instantiate the FunctionDefinition object of the Python UDF. For catalog > > function, the FunctionDefinition will be instantiated when compiling the > > job, which means the Python process is required during the compilation > > instead of the registeration. For temporary system function and temporary > > catalog function, the FunctionDefinition will be instantiated during the > > UDF registeration, so the Python process need to be started at that time. > > > > But this FLIP will only support registering the temporary system function > > and temporary catalog function in SQL DDL because registering Python UDF > to > > catalog is not supported yet. We plan to support the registeration of > > Python catalog function (via Table API and SQL DDL) in a separate FLIP. > > I'll add a non-goal section to the FLIP page to illustrate this. > > > > Best, > > Wei > > > > > > > 在 2020年3月2日，15:11，Jark Wu <imj...@gmail.com> 写道： > > > > > > Hi Weizhong, > > > > > > Thanks for proposing this feature. In geneal, I'm +1 from the table's > > view. > > > > > > I have one suggestion: I think the register python function into > catalog > > > doesn't need to startup python process (the "High Level Sequence > Diagram" > > > in your FLIP). > > > Because only meta-information is persisted into catalog, we don't need > to > > > store "return type", "input types" into catalog. > > > I guess the python process is required when compiling a SQL job. > > > > > > Best, > > > Jark > > > > > > > > > > > > On Fri, 28 Feb 2020 at 19:04, Benchao Li <libenc...@gmail.com> wrote: > > > > > >> Big +1 for this feature. > > >> > > >> We built our SQL platform on Java Table API, and most common UDF are > > >> implemented in Java. However some python developers are not familiar > > with > > >> Java/Scala, and it's very inconvenient for these users to use UDF in > > SQL. > > >> > > >> Wei Zhong <weizhong0...@gmail.com> 于2020年2月28日周五 下午6:58写道： > > >> > > >>> Thank for your reply Dan! > > >>> > > >>> By the way, this FLIP is closely related to the SQL API. @Jark Wu < > > >>> imj...@gmail.com> @Timo <twal...@apache.org> could you please take a > > >>> look? > > >>> > > >>> Thanks, > > >>> Wei > > >>> > > >>>> 在 2020年2月25日，16:25，zoudan <zoud...@163.com> 写道： > > >>>> > > >>>> +1 for supporting Python UDF in Java/Scala Table API. > > >>>> This is a great feature and would be helpful for python users! > > >>>> > > >>>> Best, > > >>>> Dan Zou > > >>>> > > >>>> > > >>> > > >>> > > >> > > >> -- > > >> > > >> Benchao Li > > >> School of Electronics Engineering and Computer Science, Peking > > University > > >> Tel:+86-15650713730 > > >> Email: libenc...@gmail.com; libenc...@pku.edu.cn > > >> > > >> > > > > >
