pepijnve commented on PR #16398: URL: https://github.com/apache/datafusion/pull/16398#issuecomment-2987544744
What puzzles me about the performance regressions are the results from https://github.com/apache/datafusion/pull/16398#issuecomment-2981029248. At that point in time, the default behavior was what's now `datafusion_coop="per_stream"` which is the exact same strategy @zhuqi-lucas initially implemented. I kept that around to be able to easily compare that variant with the Tokio budget implementation. Performance should be essentially identical to `main`. I'll have a look at the query plans for some of the queries to make sure the optimizer rule is not injecting coop operators needlessly. The only other difference I can think of is that in this PR `CooperativeStream` (fka `YieldStream`) is a generic type. If anything, I would expect that to be faster at runtime rather than slower. Maybe my intuition is completely off here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
