rkrishn7 commented on issue #17451: URL: https://github.com/apache/datafusion/issues/17451#issuecomment-3262694518
> > Without some synchronization, the behavior is racy and it's not guaranteed that that the dynamic filter is built prior to initiating the right side's execution plan. > > I think that's fine? For a big query it will be a non issue. It could make a difference in a small query. My worry is that if you get synchronization wrong or misunderstand something you can actually slow things down, end up with deadlocks, etc. Hmm, I think we want the filter to be built prior to any scans being initiated on the right side. The underlying datasource implementation may try to make use of it immediately (e.g. pruning predicate for row group filtering). In my mind, we ideally don't the filter's presence to be a function of how long it takes to get the complete bounds information from all partitions on the left side of the join. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
