gene-bordegaray opened a new issue, #23454: URL: https://github.com/apache/datafusion/issues/23454
# Allow co-partitioned `Partitioning::Range` inputs for full hash joins Labels: `enhancement` Related: - [#22395](https://github.com/apache/datafusion/issues/22395) - [#21992](https://github.com/apache/datafusion/issues/21992) - [#23183](https://github.com/apache/datafusion/issues/23183) - [#23184](https://github.com/apache/datafusion/pull/23184) - [#23266](https://github.com/apache/datafusion/issues/23266) ## Is your feature request related to a problem or challenge? [#23184](https://github.com/apache/datafusion/pull/23184) lets compatible range-partitioned inputs satisfy inner partitioned hash joins without hash repartitioning. `Full` partitioned equi joins still use the conservative hash repartition path even when both inputs are already co-partitioned by the join keys. ## Example ```text HashJoinExec: mode=Partitioned, join_type=Full, on=[(range_key@0, range_key@0)] --DataSourceExec: output_partitioning=Range([range_key@0 ASC], [(10), (20), (30)], 4) --DataSourceExec: output_partitioning=Range([range_key@0 ASC], [(10), (20), (30)], 4) ``` The compatible range inputs are already co-partitioned by partition index. Matched rows, left-only rows, and right-only rows can still be produced partition-locally. ## Describe the solution you'd like Extend the `InputDistributionRequirements::co_partitioned(...)` model to `Full` partitioned equi hash joins where execution remains partition-local. ## Acceptance criteria - Compatible range/range inputs avoid hash repartitioning for `Full` joins. - Incompatible split points, partition counts, sort options, or key expressions still trigger hash repartitioning. - Tests cover matched rows, left-only unmatched rows, and right-only unmatched rows. - Partitioned dynamic filters remain unchanged unless separately addressed by [#23376](https://github.com/apache/datafusion/issues/23376). ## Additional context This is separate from left-side and right-side joins because full joins must preserve unmatched rows from both inputs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
