naman-modi commented on issue #23658: URL: https://github.com/apache/datafusion/issues/23658#issuecomment-5131140083
Interesting read @nathanb9. The core loop i am thinking of, is close in spirit: random data + plans under a tight budget, spill on/off, and asserting the query either succeeds or fails with an expected OOM/abort (no hang). Also, our checks are more stringent than theirs. A few things though; it's multi-threaded with a shared budget, while my initial PR (#23979) is single-query per-pool. So running several plans concurrently under one shared FairSpillPool would be a good follow up work. thanks for sharing! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
