gene-bordegaray commented on code in PR #20331:
URL: https://github.com/apache/datafusion/pull/20331#discussion_r2852867045
##########
datafusion/physical-expr/src/expressions/dynamic_filters.rs:
##########
@@ -86,6 +108,11 @@ struct Inner {
/// This is redundant with the watch channel state, but allows us to
return immediately
/// from `wait_complete()` without subscribing if already complete.
is_complete: bool,
+ /// Per-partition filter expressions for partition-index routing.
+ /// When both sides of a hash join preserve their file partitioning (no
RepartitionExec(Hash)),
+ /// build-partition i corresponds to probe-partition i. This allows
storing per-partition
+ /// filters so that each partition only sees its own bounds, giving
tighter filtering.
+ partitioned_exprs: PartitionedFilters,
}
Review Comment:
I don't see the approach leak the idea of partitioning into dynamic filters,
rather its goal will be to make physical expression aware of partitioning which
I think is feasible.
Maybe I am misunderstanding what is good practice in exposing properties to
parts of the plan but it seems like a good approach to have expressions aware
of runtime features
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
