ranflarion commented on PR #25763: URL: https://github.com/apache/datafusion/pull/25763#issuecomment-6390166619
> I am sorry I am confused about this PR and the issue #25762 > > There is a lot of discussion about what pool a task gets dropped on and how that should be different, but I don't understand the end to end usecase. Are you trying to free memory / spill files faster on cancel? Can you share benchmarks / end to end tests / something that show this is a problem? > > Also, can you please explain why we need a second parallel set of APIs (if the idea is to free resources faster, why not do that for all tasks)? there are end to end use cases in cases like datafusion comet (see https://github.com/apache/datafusion-comet/pull/6261) but i think this isn't necessarily the best way to handle things. will close for now and rethink. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
