Re: [DISCUSS] FLIP-525: Model ML_PREDICT, ML_EVALUATE Implementation Design

2025-05-07 Thread Hao Li
Hi Ron, I found these names in other systems: `task_type` in big query ML [1] `model_type` in databricks [2] `task` is more of an abbreviated version from `task_type`. [1] https://cloud.google.com/bigquery/docs/reference/standard-sql/bigqueryml-syntax-evaluate [2] https://www.databricks.com/blo

Re: [DISCUSS] FLIP-525: Model ML_PREDICT, ML_EVALUATE Implementation Design

2025-05-06 Thread Ron Liu
> It's mainly used for model evaluation purposes for `ML_EVALUATE`. Different loss functions will be used and different metrics will be output for `ML_EVALUATE` based on the task option of the model. Task option is not necessary if the model is not used in `ML_EVALUATE`. `ML_EVALUATE` also has an o

Re: [DISCUSS] FLIP-525: Model ML_PREDICT, ML_EVALUATE Implementation Design

2025-05-06 Thread Hao Li
Hi Yunfeng, Ron, Thanks for the feedback. > it might be better to change the configuration api_key to apikey Make sense. I updated the FLIP. > Why is it necessary to define the task option in the WITH clause of the Model DDL, and what is its purpose? It's mainly used for model evaluation purpos

Re: [DISCUSS] FLIP-525: Model ML_PREDICT, ML_EVALUATE Implementation Design

2025-05-06 Thread Ron Liu
Hi, Hao Thanks for starting this proposal, it's a great feature, +1. Since I was missing some context, I went to FLIP-437. Combining these two FLIPs, I have the following three questions: 1. Why is it necessary to define the task option in the WITH clause of the Model DDL, and what is its purpose

Re: [DISCUSS] FLIP-525: Model ML_PREDICT, ML_EVALUATE Implementation Design

2025-05-06 Thread Yunfeng Zhou
Hi Hao, Thanks for the FLIP! It provides a clearer guideline for developers to implement model functions. One minor comment: it might be better to change the configuration api_key to apikey, which corresponds to GlobalConfiguration.SENSITIVE_KEYS. Otherwise users’ secrets might be exposed in l

Re: [DISCUSS] FLIP-525: Model ML_PREDICT, ML_EVALUATE Implementation Design

2025-05-05 Thread Hao Li
Hi Yash, ML_EVALUATE itself will be an `TableAggregateFunction`. We will only provide one implementation in Flink which will be used in codegen. Only ML_PREDICT function implementation can be based on providers. Flink will also provide a default implementation for it. Thanks, Hao On Mon, May 5,

Re: [DISCUSS] FLIP-525: Model ML_PREDICT, ML_EVALUATE Implementation Design

2025-05-05 Thread Yash Anand
Hi Hao, Thanks for the proposal, these are really interesting features to extend Flink ML use case. +1 for the proposal. I just have one question, since you plan to extend SqlMlFunctionTableFunction for both ML functions builtin registrations, will ML_EVALUATE be an aggregate function or Table f

Re: [DISCUSS] FLIP-525: Model ML_PREDICT, ML_EVALUATE Implementation Design

2025-05-05 Thread Piotr Nowojski
Hi, sounds like an interesting feature! Best, Piotrek wt., 29 kwi 2025 o 03:52 Shengkai Fang napisał(a): > Hi, Hao. > > Thanks for your proposal about ML related functions. This FLIP will help > others to implement their own model provider. > > +1 for the proposal. > > Best, > Shengkai > > Hao

Re: [DISCUSS] FLIP-525: Model ML_PREDICT, ML_EVALUATE Implementation Design

2025-04-28 Thread Shengkai Fang
Hi, Hao. Thanks for your proposal about ML related functions. This FLIP will help others to implement their own model provider. +1 for the proposal. Best, Shengkai Hao Li 于2025年4月29日周二 07:22写道: > Hi All, > > I would like to start a discussion about FLIP-525 [1]: Model ML_PREDICT, > ML_EVALUAT

[DISCUSS] FLIP-525: Model ML_PREDICT, ML_EVALUATE Implementation Design

2025-04-28 Thread Hao Li
Hi All, I would like to start a discussion about FLIP-525 [1]: Model ML_PREDICT, ML_EVALUATE Implementation Design. This FLIP is co-authored with Shengkai Fang. This FLIP is a follow up of FLIP-437 [2] to propose the implementation design for ML_PREDICT and ML_EVALUATE function which were introdu