Thanks Nathan for starting this discussion. This [1] is a very good requirement to build the materialized view on Flink Table Store.
## Aggregate Functions For 'aggregate-function' = '{sum_field1:sum,max_field2:max}'. Do you refer to any other systems? For us at Flink, a viable approach is something like the Datagen connector [2]. Something like `'fields.sum_field1.function'='sum'`. ## Default function >> Tips: Columns which do not have designated aggregate functions using newest value to overwrite old value. Do you mean `replace`? Is there anything about default functions that other systems can refer to? ## Supported functions I'm not quite sure that the names of these functions are standard enough: `replace_if_not_null/replace/concatenate`. Can you look at other systems? You can also specify whether they support retraction messages. [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-255+Introduce+pre-aggregated+merge+to+Table+Store [2] https://nightlies.apache.org/flink/flink-docs-release-1.15/docs/connectors/table/datagen/#connector-options Best, Jingsong On Tue, Aug 2, 2022 at 11:09 AM 李国君 <liguo...@bytedance.com> wrote: > Hi Nathan, > > Seems a great proposal for table store aggregation. > In the example, I think the 'max_field1' should be 1 instead of 2 after > the max aggregation in the output result. > And there may be a minor typo in the WITH clause, 'max_field2' -> > 'max_field1'. > > Best, > Guojun > > From: "Hannan Kan"<hannan...@foxmail.com> > Date: Mon, Aug 1, 2022, 11:35 PM > Subject: [Phishing Risk] [External] [DISCUSS] FLIP-255 Introduce > pre-aggregated merge to Table Store > To: "dev"<dev@flink.apache.org> > Cc: "lzljs3620320"<lzljs3620...@apache.org> > Hi everyone, I would like to open a discussion on FLIP-255 Introduce > pre-aggregated merge to table store [1]. Pre-aggregation mechanism has > been adopted by many big data systems (such as Apache Doris, > Apache Kylin , Druid etc.) to save storage and accelerate the > aggregate query. FLIP-255 proposes to introduce pre-aggregated merge into > Flink Table Store to acquire the same benefit. Supported aggregate > functions include sum, max/min, count, replace_if_not_null, > replace, concatenate, or/and. Looking forward to your feedback. > [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-255+Introduce+pre-aggregated+merge+to+table+store > Best, Nathan Kan (Hongnan Gan) >