pepijnve commented on PR #16196: URL: https://github.com/apache/datafusion/pull/16196#issuecomment-2924684413
Just as an FYI, I was reading through the tokio docs yesterday and found https://docs.rs/tokio/latest/tokio/task/coop/index.html#cooperative-scheduling which seems very similar to the situation we're seeing here with pipeline-breaking operators. The solution provided by https://docs.rs/tokio/latest/tokio/task/coop/fn.consume_budget.html is not directly applicable for Stream implementations since it's an async function, but it's kind of similar to the count-batches-and-yield solution. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
