davidhewitt opened a new pull request, #14868: URL: https://github.com/apache/datafusion/pull/14868
## Which issue does this PR close? - Closes #4658 ## Rationale for this change The IPC Stream format allows for dictionary replacement, unlike the IPC File format. As per https://github.com/apache/datafusion/issues/4658#issuecomment-2679417596 the File format does not offer advantages for the spill use case. ## What changes are included in this PR? Replaced the functionality in spilled sorts to write the IPC Stream format, instead of the IPC File format. ## Are these changes tested? Covered by existing tests of spill, I adapted these where necessary. ## Are there any user-facing changes? The code change is internal only; if for some reason users were inspecting the contents of spilled files without using the datafusion APIs to read them, they will find the format has changed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
