viirya commented on PR #6433: URL: https://github.com/apache/datafusion-comet/pull/6433#issuecomment-6044241474
@comphead on your #6396 question, I read through it and I think neither path is affected. For early placement, once the walk has crossed a join every filter is a boundary, so the early consumer lands above the `FilterExec` whether or not it carries a projection. The column-pruning `ProjectionExec` that #6396 keeps above it is column-only, so it is still traversed. In the fused case where the Project and Filter share a Spark node, the `FilterExec` is not wrapped as `CometFilterExec`, but it is a boundary either way and the result is the same. For the intermediate join's own reader attachment, a standalone projection already blocked it before #6396, so that did not get worse either. The PR base (`0ac4dada`) already contains #6396 (`55c70705`), and the TPC-DS q7/q19/q42 runs on this head show early pruning happening, which matches. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
