[
https://issues.apache.org/jira/browse/FLINK-21145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17272789#comment-17272789
]
Jark Wu commented on FLINK-21145:
---------------------------------
[~HideOnBush], the pushed down optimization is not supported yet.
> Flink Temporal Join Hive optimization
> -------------------------------------
>
> Key: FLINK-21145
> URL: https://issues.apache.org/jira/browse/FLINK-21145
> Project: Flink
> Issue Type: Wish
> Components: Connectors / Hive
> Affects Versions: 1.12.0
> Reporter: HideOnBush
> Priority: Major
>
> When flink temporal join hive dimension table, the latest partition data will
> be loaded into task memory in full, which will lead to high memory overhead.
> In fact, sometimes the latest full data is not required. You can add options
> like options in future versions. Is the dimension table data filtered?
> For example, select * from dim /*'streaming-source.partition.include'
> ='latest' condition='fild1=ab'*/ filter the latest partition data as long as
> fild1=ab
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
