Re: [Discussion] FLIP-79 Flink Function DDL Support

Hi Bowen, I revised the doc according to our existing agreement. In the implementation section, TODO items are split into two parts. Currently, we just want to have a basic implementation for Flink 1.10 release. Please take one more round of look. Yes. The vote only for the section of Flink 1.10

Re: [Discussion] FLIP-79 Flink Function DDL Support

Re 1) I'd prefer syntax of [LANGUAGE JVM|PYTHON|...]. It's also adopted by Postgres [1] and MySQL [2] "USING 'python .' " seems need extra parsing of the content in single quotes, which is not very ideal. Re 2) I agree. Besides, the doc proposes new field to be a string. I think it's better

Re: [Discussion] FLIP-79 Flink Function DDL Support

Hi Terry, Thanks for the quick response. We are on the same page. For the properties of function DDL, let's see whether there is such a need from other people. I will start voting on the design in 24 hours. Best Regards Peter Huang On Thu, Oct 31, 2019 at 3:18 AM Terry Wang wrote: > Hi

Re: [Discussion] FLIP-79 Flink Function DDL Support

Hi Peter, I’d like to share some thoughts from mysids: 1. what's the syntax to distinguish function language ? +1 for using `[LANGUAGE JVM|PYTHON] USING JAR` 2. How to persist function language in backend catalog ? + 1 for a separate field in CatalogFunction. But as to specific ba

Re: [Discussion] FLIP-79 Flink Function DDL Support

Hi Bowen, I can't agree more about we first have an agreement on the DDL syntax and focus on the MVP in the current phase. 1) what's the syntax to distinguish function language Currently, there are two opinions: - USING 'python .' - [LANGUAGE JVM|PYTHON] USING JAR '...' As we need to

Re: [Discussion] FLIP-79 Flink Function DDL Support

Hi Jingsong, Thanks for the input. The FLINK function DDL definitely needs to align with HQL, I updated the doc accordingly. CREATE FUNCTION [db_name.]function_name AS class_name [USING JAR|FILE|ARCHIVE 'file_uri' [, JAR|FILE|ARCHIVE 'file_uri'] ]; For you other questions below: 1) how to load

Re: [Discussion] FLIP-79 Flink Function DDL Support

Hi all, Besides all the good questions raised above, we seem all agree to have a MVP for Flink 1.10, "to support users to create and persist a java class-based udf that's already in classpath (no extra resource loading), and use it later in queries". IIUIC, to achieve that in 1.10, the following

Re: [Discussion] FLIP-79 Flink Function DDL Support

Hi Peter, Thanks for your proposal. The first thing I care about most is whether it can cover the needs of hive. Hive create function: CREATE FUNCTION [db_name.]function_name AS class_name [USING JAR|FILE|ARCHIVE 'file_uri' [, JAR|FILE|ARCHIVE 'file_uri'] ]; Hive support a list of resources, a

Re: [Discussion] FLIP-79 Flink Function DDL Support

Hi Peter, thanks for your proposal. I left some comments in the FLIP document. I agree with Terry that we can have a MVP in Flink 1.10 but should already discuss the bigger picture as a DDL string cannot be changed easily once released. In particular we should discuss how resources for funct

Re: [Discussion] FLIP-79 Flink Function DDL Support

Hi Peter, Sorry late to reply. Thanks for your efforts on this and I just looked through your design. I left some comments in the doc about alter function section and function catalog interface. IMO, the overall design is ok and we can discuss further more about some details. I also think it’

Re: [Discussion] FLIP-79 Flink Function DDL Support

Hi Xuefu, Thank you for the feedback. I think you are pointing out a similar concern with Bowen. Let me describe how the catalog function and function factory will be changed in the implementation section. Then, we can have more discussion in detail. Best Regards Peter Huang On Tue, Oct 15, 201

Re: [Discussion] FLIP-79 Flink Function DDL Support

Hi Bowen, Thanks for your kind feedback. This FLIP is to propose a function DDL syntax that is compatible with all of the other related work. I agree it is a little bit covering too many cases. From the perspective of execution, I will mainly focus on the create and persist a java class-based udf

Re: [Discussion] FLIP-79 Flink Function DDL Support

Thanks to Peter for the proposal! I left some comments in the google doc. Besides what Bowen pointed out, I'm unclear about how things work end to end from the document. For instance, SQL DDL-like function definition is mentioned. I guess just having a DDL for it doesn't explain how it's supporte

Re: [Discussion] FLIP-79 Flink Function DDL Support

Hi Zhenqiu, Thanks for taking on this effort! A couple questions: - Though this FLIP is about function DDL, can we also think about how the created functions can be mapped to CatalogFunction and see if we need to modify CatalogFunction interface? Syntax changes need to be backed by the backend. -

[Discussion] FLIP-79 Flink Function DDL Support

Dear Community, FLIP-79 Flink Function DDL Support This proposal aims to support function DDL with the consideration of SQL syntax, language compliance, and advanced external UDF lib registration. The Flink DD