ozankabak commented on PR #16196: URL: https://github.com/apache/datafusion/pull/16196#issuecomment-2940167521
> This assumption turns out to be quite easy to break as in the interleave example. I would say it is not *that* easy to break, took you some effort to contrive an example. Besides, this happens due to a particular behavior by the current implementation of `InterleaveExec`, which (I suspect) we can fix. Let's go through the following exercise together: In the most general case, let's assume that I'm wrong and there is a certain class of operators that is always prone to "eating" `Pending` values due to the nature of the actual operation they perform. If we find this to be the case, we have discovered an important piece of information we should expose via the `ExecutionPlan` APIs. We would then do that, and simply update the rule here to also insert a `YieldExec` right after such operators. This way, we would retain the ability to insert minimum necessary overhead to any given plan to have cancellability. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
