Re: [Phishing Risk] [External] [DISCUSS] FLIP-255 Introduce pre-aggregated merge to Table Store

[Jingsong Li]

Hi Nathan,

Thanks for the reply.

## Aggregate functions

I think `'fields.sum_field1.function'='sum'` is a safe one, I am more
worried about using complex json, which will affect the sql parsing.

## Supported functions

I think it is better to be consistent with Flink Table [1] and PostgreSQL [2].
- replace -> last_value
- replace_if_not_null -> last_non_null_value
- concatenate -> listagg
- or -> bool_or
- and -> bool_and

## Default function

You can update the FLIP to clarify the default value of the
corresponding agg function.

## Changelog support

Can you also specify whether functions support changelog/retraction messages?

[1] 
https://nightlies.apache.org/flink/flink-docs-release-1.15/docs/dev/table/functions/systemfunctions/#aggregate-functions
[2] https://www.postgresql.org/docs/9.5/functions-aggregate.html

Best,
Jingsong

On Tue, Aug 2, 2022 at 11:41 PM Hannan Kan <hannan...@foxmail.com> wrote:
>
> Thank you for reviewing FLIP-255.
>
> That indeed is a typo. I will correct it immediately.
>
>
> Best,
> Nathan
>
>
>
>
> ------------------&nbsp;原始邮件&nbsp;------------------
> 发件人:                                                                          
>                                               "dev"                           
>                                                          
> <liguo...@bytedance.com&gt;;
> 发送时间:&nbsp;2022年8月2日(星期二) 中午11:09
> 收件人:&nbsp;"dev"<dev@flink.apache.org&gt;;
>
> 主题:&nbsp;Re: [Phishing Risk] [External] [DISCUSS] FLIP-255 Introduce 
> pre-aggregated merge to Table Store
>
>
>
> Hi Nathan,
>
>
> Seems a great proposal for table store aggregation.
> In the example, I think the 'max_field1' should be 1 instead of 2 after the 
> max aggregation in the output result.
> And there may be a minor typo in the WITH clause, 'max_field2' -&gt; 
> 'max_field1'.
>
>
>
> Best,
> Guojun
>
>
>
> From: "Hannan Kan"<hannan...@foxmail.com&gt;
> Date:  Mon, Aug 1, 2022, 11:35 PM
> Subject:  [Phishing Risk] [External] [DISCUSS] FLIP-255 Introduce 
> pre-aggregated merge to Table Store
> To: "dev"<dev@flink.apache.org&gt;
> Cc: "lzljs3620320"<lzljs3620...@apache.org&gt;
>
>
>
> Hi everyone, I would like to open a discussion on&amp;nbsp;FLIP-255 Introduce 
> pre-aggregated merge to table store&amp;nbsp;[1]. Pre-aggregation mechanism 
> has been adopted by many big data systems (such as Apache Doris,&amp;nbsp; 
> Apache Kylin , Druid etc.)&amp;nbsp;to save storage and accelerate the 
> aggregate query. FLIP-255 proposes to introduce pre-aggregated merge into 
> Flink Table Store to acquire the same benefit.&amp;nbsp; Supported aggregate 
> functions include&amp;nbsp;sum, max/min, count, replace_if_not_null, 
> replace,&amp;nbsp; concatenate, or/and. Looking forward to your feedback. 
> [1]&amp;nbsp;https://cwiki.apache.org/confluence/display/FLINK/FLIP-255+Introduce+pre-aggregated+merge+to+table+store
>  Best, Nathan Kan (Hongnan Gan)