alamb opened a new pull request, #16249: URL: https://github.com/apache/datafusion/pull/16249
## Which issue does this PR close? - Related to https://github.com/apache/arrow-rs/issues/6692 - Related to #3463 ## Rationale for this change I am trying to move the coalesce operation upstream into arrow-rs so that it can be reused in parquet filter pushdown and more highly optimized. See : - https://github.com/apache/arrow-rs/pull/7597 The proposed upstream coalescer API is slightly different (guaranteed batch size, and doesn't have limit) so we must adapt the DataFusion code to handle this ## What changes are included in this PR? This PR refactors the `BatchCoalescer` in DataFusion to use the proposed upstream API to show that it 1. Can be used (api is complete enough) 2. Is not any slower ## Are these changes tested? By CI tests ## Are there any user-facing changes? No -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
