EmilyMatt opened a new issue, #1495: URL: https://github.com/apache/datafusion-comet/issues/1495
### Describe the bug The ShuffleRepartitioner should be the memory consumer registered with the pool, with each partitionbuffer being a reservation split off from it(using new_empty()), so they are all considered part of the same operator. As it currently stands, each one of them takes a full weight as a spillable consumer, which leads to bad memory management in memory pools that take the number of spillable operators into account(such as fair pools). ### Steps to reproduce _No response_ ### Expected behavior _No response_ ### Additional context _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
