2010YOUY01 commented on PR #1511: URL: https://github.com/apache/datafusion-comet/pull/1511#issuecomment-2760428379
> While reading this I also wonder if we would be able to hook into [DF's new SpillManager](https://github.com/apache/datafusion/pull/15355). That's a task for another PR, but just leaving the thought here. Its implementation is inspired by `ShuffleWriterExec`'s spilling utilities, and I think there are still some functionality gaps (For example: `datafusion`'s `SpillManager` does not support general-purpose compression like `lz4` yet) We plan to include those features in the future to make it reusable. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
