timsaucer opened a new pull request, #20449: URL: https://github.com/apache/datafusion/pull/20449
DRAFT: This requires a rebase after https://github.com/apache/datafusion/pull/19469 merges ## Which issue does this PR close? - Closes #. ## Rationale for this change This PR introduces an important concept in the FFI work to avoids creating wrappers upon wrappers of plans. It was discovered as part of the work to create FFI physical optimizer rules. Suppose we have a foreign plan. Then we attempt to turn this into an FFI plan. What we will end up with currently is a FFI plan where the underlying private data is a foreign plan that additionally contains a FFI plan. Instead *any* time we are creating an FFI object we should check to see if it is locally downcastable to a Foreign plan and if so to just access the already existing FFI object. This pattern is adapted across all FFI objects in this PR. With this work in place we can also properly support repartioning via FFI as well as `new_with_children` via FFI. ## What changes are included in this PR? - Adds access pattern for creating new FFI objects. When they are already a locally downcastable to a Foreign wrapper then we simply get the underlying existing FFI object instead of creating a wrapper around a wrapper. - Implement repartitioning and new_with_children via FFI on execution plans. ## Are these changes tested? Integration tests are added. ## Are there any user-facing changes? The one use facing change is that for some of the aggregates and accumulators that take in closures we require these closures to be static so that we can downcast the boxed traits. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
