brancz opened a new pull request, #27015: URL: https://github.com/apache/datafusion/pull/27015
Compact dictionary value tables larger than their key arrays before traversing retained physical children and writing IPC spill files. > [!IMPORTANT] > > **Fork CI setup:** For pull requests from a fork to `main`, enable workflows in your fork's **Actions** tab before you push. > See the [fork CI setup guide](https://datafusion.apache.org/contributor-guide/#pull-requests-from-forks). ## Which issue does this PR close? - Closes https://github.com/apache/datafusion/issues/27014 ## What changes are included in this PR? Compaction of dicts if there are more values than indices and tests. ## What is the testing strategy for this PR? Unit tests ## Are there any user-facing changes? No user facing changes, just an improvement to reduce I/O and total size written to disks when spilling. @alamb -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
