HideOnBush created FLINK-21145:
----------------------------------
Summary: Flink Temporal Join Hive optimization
Key: FLINK-21145
URL: https://issues.apache.org/jira/browse/FLINK-21145
Project: Flink
Issue Type: Wish
Components: Connectors / Hive
Affects Versions: 1.12.0
Reporter: HideOnBush
When flink temporal join hive dimension table, the latest partition data will
be loaded into task memory in full, which will lead to high memory overhead. In
fact, sometimes the latest full data is not required. You can add options like
options in future versions. Is the dimension table data filtered?
For example, select * from dim /*'streaming-source.partition.include' ='latest'
condition='fild1=ab'*/ filter the latest partition data as long as fild1=ab
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
