[
https://issues.apache.org/jira/browse/FLINK-20611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Suhan Mao updated FLINK-20611:
------------------------------
Description:
In our company, many jobs run with window aggregate and suffer from large
amounts of intermediate shuffle. If we can enable 2 phase local agg on window
aggregate, the performance will be improved.
This issue aims to support 2 phase local agg in window aggregate operator both
in SQL and Table API. Simple tumble window will be supported in the first place.
We still use below configurations, but they take effect only inside each single
window.
{{table.exec.mini-batch.enabled}}
{{table.exec.mini-batch.allow-latency}}
{{table.exec.mini-batch.size}}
was:
Currently, mini batch does not support window aggregate. In our company, many
jobs run with window aggregate and suffer from large amounts of intermediate
shuffle. If we can enable mini batch on window aggregate, the performance will
be improved.
This issue aims to support mini batch in window aggregate operator both in SQL
and Table API. Simple tumble window will be supported in the first place.
We still use below configurations, but they take effect only inside each single
window.
{{table.exec.mini-batch.enabled}}
{{table.exec.mini-batch.allow-latency}}
{{table.exec.mini-batch.size}}
> Support 2 phase local aggregate for window aggregate operation
> --------------------------------------------------------------
>
> Key: FLINK-20611
> URL: https://issues.apache.org/jira/browse/FLINK-20611
> Project: Flink
> Issue Type: New Feature
> Components: Table SQL / API, Table SQL / Planner
> Reporter: Suhan Mao
> Priority: Major
>
> In our company, many jobs run with window aggregate and suffer from large
> amounts of intermediate shuffle. If we can enable 2 phase local agg on window
> aggregate, the performance will be improved.
> This issue aims to support 2 phase local agg in window aggregate operator
> both in SQL and Table API. Simple tumble window will be supported in the
> first place.
>
> We still use below configurations, but they take effect only inside each
> single window.
> {{table.exec.mini-batch.enabled}}
> {{table.exec.mini-batch.allow-latency}}
> {{table.exec.mini-batch.size}}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
