gstvg opened a new pull request, #23660: URL: https://github.com/apache/datafusion/pull/23660
## Which issue does this PR close? Adapted backport of #22853 (non-breaking) ## Rationale for this change LambdaExpr previously compressed the column-index projection by enumerating every referenced Column/LambdaVariable index and packing them into a dense range. That collapse is correct for outer captures (and is a no-op for single-parameter lambdas, which is why array_transform was never affected), but it also moves lambda parameters around. A two-parameter lambda like (k, v) -> v (with k unused) would have its LambdaVariable for v re-projected from index 1 to index 0 — so at runtime the body reads the slot the higher-order function had filled with k and silently returns the wrong column. This is a latent bug today — no in-tree higher-order function exercises it — but it blocks https://github.com/apache/datafusion/pull/22689 (transform_values, which uses (k, v) -> body lambdas) and any future HOF that takes more than one parameter. ## What changes are included in this PR? Currently the lambda body and the captured record batch are projected to avoid copying uncaptured columns. This PR instead replace uncaptured columns with cheap `NullArray`'s, so no projection is necessary, and therefore no wrong projection happens ## Are these changes tested? sqllogictests in new file `multi_arg_lambda.slt` ## Are there any user-facing changes? No -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
