andygrove commented on issue #2452: URL: https://github.com/apache/datafusion-comet/issues/2452#issuecomment-3366667598
> > One idea is to configure the pool individually > > There is more than one memory pool? Are these blocks of memory from the memory pool that are reserved for an operator? There are multiple "memory reservations", all backed by the same pool. > > so spilling would start sooner > > That seems to be the way to do it. > > From what I understand, the issue is the spilling itself requires some memory and we have already used all the available memory when we start/try to spill? Yes, it appears so. https://private-user-images.githubusercontent.com/934084/497247145-4e37d417-3bf3-4093-b72e-24f33a765fea.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NTk1MTQxMTMsIm5iZiI6MTc1OTUxMzgxMywicGF0aCI6Ii85MzQwODQvNDk3MjQ3MTQ1LTRlMzdkNDE3LTNiZjMtNDA5My1iNzJlLTI0ZjMzYTc2NWZlYS5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjUxMDAzJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI1MTAwM1QxNzUwMTNaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT1mZmI3OGIwNWRmZTk5ZjQ2YTQwNzQyZmUyOTc2OTMzNDIyMjQ1YTIxYThjYjUxYTdmZDE3ZGY5YjQwNzcwYjY0JlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCJ9.TKAD0to73ht3-nbn-KDx_2yKnVAGUxREFrUeqDryex8 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
