zhuqi-lucas opened a new issue, #16353: URL: https://github.com/apache/datafusion/issues/16353
### Is your feature request related to a problem or challenge? We have done the first step in https://github.com/apache/datafusion/pull/16196 for pipeline breaking cancellation support, this epic trace the remaining sub-task for the remaining improvement. cc @ozankabak @alamb @pepijnve - [ ] Adding a few tests (maybe SLT?) that show YieldStreamExec being inserted. Also add logs related to built-in YiedStream. - [ ] Improving the documentation to make it clear that any leaf (source) that already yields just has to implement ```rust fn with_cooperative_yields(self: Arc<Self>) -> Option<Arc<dyn ExecutionPlan>> { Some(self) } ``` to signal the planner that no YieldStream combinator is necessary. - [ ] Improving InsertYieldExec rule by means of an API that exposes input and output pipelining behaviors of operators effectively - [ ] Investigating whether any already-existing manual yielding (for example, like the one in RepartitionExec) can now be removed - [ ] We will think about supporting cases involving non-volcano (e.g. spill) data flow. Feel free to add more tasks, thanks! ### Describe the solution you'd like _No response_ ### Describe alternatives you've considered _No response_ ### Additional context _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
