lakshmie6x opened a new pull request, #26847: URL: https://github.com/apache/datafusion/pull/26847
Every probe partition of a CollectLeft hash join reports the same shared build to `SharedBuildAccumulator`, yet the filter is only finalized after all of them have reported, so each partition waits for every other one before probing. That build is already complete at the first report. A scheduler that runs only some of a plan's partitions at a time deadlocks on this wait: the running partitions wait for reports from partitions that never get to run. The first CollectLeft report now finalizes the filter. No probe runs with a partial filter, unlike #21641, because the first report already has the whole build. Partitioned joins still wait for every build partition. The new test runs one probe partition of two; it hangs without the change. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
