2010YOUY01 opened a new issue, #15553: URL: https://github.com/apache/datafusion/issues/15553
### Is your feature request related to a problem or challenge? During external queries, temporary computation results can be spilled to disk, to let the queries to complete under limited memory. A new configuration to limit the maximum total spilled file disk usage is being added in https://github.com/apache/datafusion/pull/15520 ### Describe the solution you'd like Adding a configuration to `datafusion-cli` like ``` # By default, disk usage is not limited datafusion-cli -c 'select 1, 2 from foo'; # Limit disk usage to 10GB datafusion-cli --disk-limit 10G -c 'select 1, 2 from foo'; ``` Here is a reference implementation for a very similar feature (adding configuration for memory pool) https://github.com/apache/datafusion/issues/7419 ### Describe alternatives you've considered This solution is a little bit hack, in the long term it should better be configurable through SQL interface like ``` set datafusion.runtime.disk_limit = 1GB; set datafusion.runtime.memory_limit = 100MB; select * from tbl order by c1; ``` Tracking issue is in https://github.com/apache/datafusion/issues/15552 ### Additional context _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
