xavlee opened a new pull request, #26852: URL: https://github.com/apache/datafusion/pull/26852
## Which issue does this PR relate to? Part of #24438. Follows up on [the invalidation API review in #24698](https://github.com/apache/datafusion/pull/24698#discussion_r4400910694). Depends on #24698. This PR and #24497 both build on the grouping metadata introduced there. ## Rationale for this change Execution operators can clone their input's equivalence properties and call `clear_orderings()` when rearranging rows. Clearing grouping assertions at the same boundary prevents consumers from relying on contiguous groups that the output no longer guarantees. ## What changes are included in this PR? - Make `EquivalenceProperties::clear_orderings()` clear explicit grouping tuples and their normalized cache along with ordering information. - Document the invalidation behavior and the narrower `clear_groupings()` operation, which preserves ordering information. - Let partition coalescing and sort-preserving merge use the combined clear. - Cover partially clustered aggregation that preserves ordering on one key while clearing an unrelated grouping assertion. ## Stack 1. [#24737 — test: cover unsorted contiguous groups in one partition](https://github.com/apache/datafusion/pull/24737) (merged) 2. [#24697 — refactor: use group clustering terminology in aggregate execution](https://github.com/apache/datafusion/pull/24697) (merged) 3. [#24698 — feat: add grouping equivalence properties](https://github.com/apache/datafusion/pull/24698) 4. [#24497 — feat: stream aggregates over grouped input](https://github.com/apache/datafusion/pull/24497) 5. fix: clear groupings when clearing orderings ← **this PR** ## What is the testing strategy for this PR? - `clear_orderings_clears_groupings` checks that ordering-derived grouping and explicit grouping are invalidated, including the normalized grouping cache. - `clear_groupings_preserves_orderings` checks that explicit grouping can be cleared while ordering and ordering-derived grouping remain available. - `coalesce_clears_explicit_grouping` covers one and multiple input partitions. - `clustered_aggregate_clears_explicit_grouping` covers partial and single aggregation, preserving the ordering on `a` while clearing the explicit grouping on `c`. Removing the new API clear makes its regression fail. Removing the aggregate's explicit grouping clear makes both clustered aggregate cases fail. Local validation passed the extended workspace suite, all-target/all-feature Clippy, and the Rust documentation build with warnings denied. The HTML user-guide check requires local Graphviz and `cargo-depgraph` installations and was not run. ## Are there any user-facing changes? Yes. `clear_orderings()` also clears explicit grouping assertions. Operators using this API discard those assertions conservatively, including single-input `CoalescePartitionsExec`. ## Review this layer [View only this follow-up's changes over #24698](https://github.com/xavlee/datafusion/compare/2c5ab1f58eb81c6a85acb7e0c577dd6d3454cc23...2618076ec565da619a2de674a5aa92e039ae52db). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
