JoeryH opened a new pull request, #26317: URL: https://github.com/apache/datafusion/pull/26317
## Which issue does this PR close? - Closes #26313. ## Rationale for this change Planning a SELECT costs O(items × input plan size), because `SqlToRel::sql_select_to_rex` calls `LogicalPlan::using_columns()` once per SELECT item. Each call walks the whole input plan, subqueries and expressions included, and the result depends only on the input plan. Nested wide SELECTs (views over views) therefore plan in roughly O(width² × depth²). The issue has the numbers. At 4000 columns × 12 levels, `statement_to_plan` goes from 8.0 s to 4.4 s with this change alone, and to 0.6 s once the `DFSchema` scan from #26314 is also gone. ## What changes are included in this PR? - `prepare_select_exprs` computes `plan.using_columns()` once and passes it to `sql_select_to_rex`. `plan_wildcard_options` does the same for a wildcard's `REPLACE` items. - A `sql_planner` benchmark case, `logical_nested_select_700_columns_8_levels`. On an M1 Max its query goes from 178 ms to 122 ms in `statement_to_plan`, measured with a standalone harness because the full benchmark needs the clickbench data. One behaviour difference: if `using_columns()` itself fails, the error now surfaces once for the SELECT instead of once per item through `DataFusionErrorBuilder`. `using_columns` only errors if the tree walk does, so I don't expect this to be observable, but I can keep the old shape with a lazily filled `Option` if you'd prefer. ## What is the testing strategy for this PR? The planned output is unchanged, so this relies on the existing `datafusion-sql` tests and sqllogictests, which pass locally. Those cover USING / NATURAL joins and wildcard expansion over them, the cases `using_columns` exists for. The new benchmark case covers the performance side. ## Are there any user-facing changes? No. Planning is faster and plans are identical. --- Prepared with AI assistance (Claude Code). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
