andygrove opened a new issue, #1716: URL: https://github.com/apache/datafusion-comet/issues/1716
### Describe the bug I am running with the following config: - SF 1000 (1TB) dataset - 32 executors - Each executor has 16 cores and 32 GB memory + 32 GB off-heap memory - Data is partitioned by date, so query uses DPP, causing Comet to fall back to Spark early after the scans - Comet does run a columnar shuffle, but no native shuffle - Running in k8s Executor memory grows over time, and executors start to get killed due to OOM after running the query ~37 times. This seems to indicate some kind of memory leak. ### Steps to reproduce _No response_ ### Expected behavior _No response_ ### Additional context _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
