zhuqi-lucas commented on PR #16196: URL: https://github.com/apache/datafusion/pull/16196#issuecomment-2933011699
> Agreed on the rule approach. > > Interleave will poll each of its children at most once per poll call. If none of the children returns a Ready it will return Pending itself. Each poll starts at a random child index. By varying when the yields occur in each child there’s a very high likelihood that at least one child will return Ready. As a consequence the aggregate sees an always ready stream despite the yields being injected. This is a good example, try to find a solution. > > Working on it and it seems to get cancelled indeed. I'll work on understanding why and report back. > > @ozankabak I was able to get another non-exciting plan as follows. It's a contrived example, but it does demonstrate how `Pending` does not always bubble far enough up to allow a task to actually stop. Code for this is at https://github.com/pepijnve/datafusion_cancel_test.git > > While I was at it I added a child-of-pipeline-breaker wrapping test as well which is capable of cancelling this particular plan. > > ``` > AggregateExec: mode=Single, gby=[], aggr=[total] > InterleaveExec > CoalesceBatchesExec: target_batch_size=8192 > FilterExec: value@0 > 8192 > YieldStreamExec yield=64 > InfiniteExec > CoalesceBatchesExec: target_batch_size=8192 > FilterExec: value@0 > 8111 > YieldStreamExec yield=64 > InfiniteExec > CoalesceBatchesExec: target_batch_size=8192 > FilterExec: value@0 > 8030 > YieldStreamExec yield=64 > InfiniteExec > CoalesceBatchesExec: target_batch_size=8192 > FilterExec: value@0 > 7949 > YieldStreamExec yield=64 > InfiniteExec > CoalesceBatchesExec: target_batch_size=8192 > FilterExec: value@0 > 7868 > YieldStreamExec yield=64 > InfiniteExec > CoalesceBatchesExec: target_batch_size=8192 > FilterExec: value@0 > 7787 > YieldStreamExec yield=64 > InfiniteExec > CoalesceBatchesExec: target_batch_size=8192 > FilterExec: value@0 > 7706 > YieldStreamExec yield=64 > InfiniteExec > CoalesceBatchesExec: target_batch_size=8192 > FilterExec: value@0 > 7625 > YieldStreamExec yield=64 > InfiniteExec > CoalesceBatchesExec: target_batch_size=8192 > FilterExec: value@0 > 7544 > YieldStreamExec yield=64 > InfiniteExec > CoalesceBatchesExec: target_batch_size=8192 > FilterExec: value@0 > 7463 > YieldStreamExec yield=64 > InfiniteExec > ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
