gene-bordegaray opened a new issue, #23453: URL: https://github.com/apache/datafusion/issues/23453
# Allow co-partitioned `Partitioning::Range` inputs for right-side hash joins Labels: `enhancement` Related: - [#22395](https://github.com/apache/datafusion/issues/22395) - [#21992](https://github.com/apache/datafusion/issues/21992) - [#23183](https://github.com/apache/datafusion/issues/23183) - [#23184](https://github.com/apache/datafusion/pull/23184) - [#23266](https://github.com/apache/datafusion/issues/23266) ## Is your feature request related to a problem or challenge? [#23184](https://github.com/apache/datafusion/pull/23184) lets compatible range-partitioned inputs satisfy inner partitioned hash joins without hash repartitioning. Right-side partitioned equi joins still use the conservative hash repartition path even when both inputs are already co-partitioned by the join keys. ## Example ```text HashJoinExec: mode=Partitioned, join_type=Right, on=[(range_key@0, range_key@0)] --DataSourceExec: output_partitioning=Range([range_key@0 ASC], [(10), (20), (30)], 4) --DataSourceExec: output_partitioning=Range([range_key@0 ASC], [(10), (20), (30)], 4) ``` The compatible range inputs are already co-partitioned by partition index. The optimizer should preserve that layout instead of inserting hash repartitions for the right join. ## Describe the solution you'd like Extend the `InputDistributionRequirements::co_partitioned(...)` model to right-side partitioned equi hash joins where execution remains partition-local. Covered join types: - `Right` - `RightSemi` - `RightAnti` - `RightMark` ## Acceptance criteria - Compatible range/range inputs avoid hash repartitioning for the covered join types. - Incompatible split points, partition counts, sort options, or key expressions still trigger hash repartitioning. - Tests cover matched and unmatched rows. - `RightMark` tests cover null-related marker behavior. - Partitioned dynamic filters remain unchanged unless separately addressed by [#23376](https://github.com/apache/datafusion/issues/23376). ## Additional context This should follow left-side joins because right-side joins often stress output projection and swapped-side assumptions differently. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
