mrhard9090 opened a new pull request, #25899:
URL: https://github.com/apache/datafusion/pull/25899
- Closes #25837.
## Rationale for this change
A `LATERAL` subquery fails with `Schema error: No field named l.id` when its
correlated filter sits inside a derived table whose alias differs from the
inner table name:
```sql
SELECT o.k, s.v
FROM o, LATERAL (
SELECT l2.v FROM (SELECT * FROM l WHERE l.id = o.k) AS l2
) AS s;
```
The same query works when the alias matches the table name, and
DuckDB/PostgreSQL return `(1, 10), (2, 20)`.
## What changes are included in this PR?
`PullUpCorrelatedExpr` extracts the join filter with the inner column
qualified by its original table name (`l.id`), but the rewritten subquery
exposes it under the derived-table alias (`l2.id`). `requalify_filter` in
`decorrelate_lateral_join.rs` only rewrote columns found verbatim in the inner
schema, so the stale qualifier survived into the join condition.
The fix adds a fallback: when a filter column is not found in the inner
schema, it is matched by unqualified field name as long as the match is
unambiguous in the inner schema **and** its qualifier occurs inside the
rewritten subquery (table name or derived-table alias). Qualifiers that appear
nowhere inside (outer references such as `o.k`) are left untouched, so nested
laterals that correlate against sibling tables keep working.
## What is the testing strategy for this PR?
- New `sqllogictest` case in
`datafusion/sqllogictest/test_files/lateral_join.slt` with the issue's repro
(verified red before the fix with the exact reported error, green after).
- Full `lateral_join.slt` passes, including the nested lateral cases that
caught an over-broad first version of this fix.
- `cargo test -p datafusion-optimizer --lib decorrelate`: 66 passed.
- `cargo fmt --all -- --check` clean; `cargo clippy -p datafusion-optimizer
--all-targets --all-features --no-deps -- -D warnings` clean (workspace-wide
clippy has a pre-existing `datafusion-common` failure on Rust 1.98.1, unrelated
to this change).
## Are there any user-facing changes?
Bug fix only, no API changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
