Martijn Visser created FLINK-40678:
--------------------------------------
Summary: Lookup join digest ignores table OPTIONS hints
Key: FLINK-40678
URL: https://issues.apache.org/jira/browse/FLINK-40678
Project: Flink
Issue Type: Bug
Components: Table SQL / Planner
Reporter: Martijn Visser
Two lookup joins on the same table with different OPTIONS hints are merged into
one, so the second
one silently reads the first one's connector options.
{code:sql}
SELECT s.a, d.name
FROM MyTable AS s
JOIN LookupTable /*+ OPTIONS('data-id'='probeA') */ FOR SYSTEM_TIME AS OF
s.proctime AS d
ON s.a = d.id
UNION ALL
SELECT s.a, d.name
FROM MyTable AS s
JOIN LookupTable /*+ OPTIONS('data-id'='probeB') */ FOR SYSTEM_TIME AS OF
s.proctime AS d
ON s.a = d.id
{code}
plans to a single lookup join:
{code}
Union(all=[true], union=[a, name])
:- Calc(select=[a, name])(reuse_id=[1])
: +- LookupJoin(table=[default_catalog.default_database.LookupTable,
project=[id, name], metadata=[]], joinType=[InnerJoin], lookup=[id=a],
select=[a, id, name])
: +- Calc(select=[a])
: +- DataStreamScan(table=[[default_catalog, default_database,
MyTable]], fields=[a, b, c, proctime, rowtime])
+- Reused(reference_id=[1])
{code}
{{CommonPhysicalTableSourceScan.explainTerms}} adds a {{hints}} item, so two
scans with different
OPTIONS stay separate. A lookup join never receives the scan's hints:
{{CommonLookupJoinRule}}
passes {{tableScan.getTable}} but not {{tableScan.getHints}}, so they cannot
reach
{{CommonPhysicalLookupJoin.explainTerms}} and the two nodes get the same digest.
Same class as FLINK-36808, which put the pushed-down ability specs into the
digest but not the
hints. Fixing this means threading the scan's hints through the rule into the
physical node.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
