alamb commented on PR #14975: URL: https://github.com/apache/datafusion/pull/14975#issuecomment-2739766003
> I plan to address point 1 and the above-mentioned limitation in another PR, and after that rework this PR to add disk limit feature. So closed this PR for now. THank you -- I think consolidating the spilling related utilities and making the interface clear will make it much easier as we go forward to add new features. Maybe we can take inspiration from comet's spill / shuffle writer code: https://github.com/apache/datafusion-comet/blob/382ac938f2d10666eefc08ec5c1c82025ddf3726/native/core/src/execution/shuffle/shuffle_writer.rs -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
