gene-bordegaray opened a new issue, #23452: URL: https://github.com/apache/datafusion/issues/23452
# Allow co-partitioned `Partitioning::Range` inputs for left-side hash joins Labels: `enhancement` Related: - [#22395](https://github.com/apache/datafusion/issues/22395) - [#21992](https://github.com/apache/datafusion/issues/21992) - [#23183](https://github.com/apache/datafusion/issues/23183) - [#23184](https://github.com/apache/datafusion/pull/23184) - [#23266](https://github.com/apache/datafusion/issues/23266) ## Is your feature request related to a problem or challenge? [#23184](https://github.com/apache/datafusion/pull/23184) lets compatible range-partitioned inputs satisfy inner partitioned hash joins without hash repartitioning. Left-side partitioned equi joins still use the conservative hash repartition path even when both inputs are already co-partitioned by the join keys. ## Example ```text HashJoinExec: mode=Partitioned, join_type=Left, on=[(range_key@0, range_key@0)] --DataSourceExec: output_partitioning=Range([range_key@0 ASC], [(10), (20), (30)], 4) --DataSourceExec: output_partitioning=Range([range_key@0 ASC], [(10), (20), (30)], 4) ``` The compatible range inputs are already co-partitioned by partition index. The optimizer should not add hash repartitions solely to satisfy the left join distribution requirement. ## Describe the solution you'd like Extend the `InputDistributionRequirements::co_partitioned(...)` model to left-side partitioned equi hash joins where execution remains partition-local. Covered join types: - `Left` - `LeftSemi` - `LeftAnti` - `LeftMark` ## Acceptance criteria - Compatible range/range inputs avoid hash repartitioning for the covered join types. - Incompatible split points, partition counts, sort options, or key expressions still trigger hash repartitioning. - Tests cover matched and unmatched rows. - `LeftMark` tests cover null-related marker behavior. - Partitioned dynamic filters remain unchanged unless separately addressed by [#23376](https://github.com/apache/datafusion/issues/23376). ## Additional context This should follow the multi-child requirement model added in [#23184](https://github.com/apache/datafusion/pull/23184). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
