2010YOUY01 commented on issue #15321: URL: https://github.com/apache/datafusion/issues/15321#issuecomment-2785618038
I took a look, but the reason for those test failures isn’t obvious to me. However, I think the implementation requires two changes: 1. Always use `SpillManager`'s read utility to read spill, instead of using a lower-level utility function in SMJ (I forgot to update the spill read path in https://github.com/apache/datafusion/pull/15405 🤦♂, will update it soon ) 2. We have to use `IPCStreamWriter` instead of `IPCFileWriter`, otherwise it will cause a regression for not supporting spilling dictionary type, see https://github.com/apache/datafusion/pull/14868. (unless we can also support spilling dictionaries in `IPCFileWriter`) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
