mrhhsg opened a new pull request, #70670: URL: https://github.com/apache/doris/pull/70670
### Background The legacy BE local-shuffle planner propagates shuffled-consumer and bucket-distribution requirements independently. Aggregation operators choose local-exchange partition expressions from FE distribution metadata or their grouping expressions. ### What problem does this PR solve? Issue Number: None Related PR: None PR Type: Bugfix Problem Summary: A bucket-preserving aggregation can receive a bucket-distribution requirement without a shuffled-consumer requirement. If tablets are bucketed by `k1` but the aggregation groups by `(k1, k2)`, the legacy planner may hash by both grouping keys and route rows outside a BE's owned buckets. Use the FE distribution key whenever bucket distribution must be preserved, while keeping the existing grouping-key fallback. ### Details - **Frequency:** Multi-stage DISTINCT aggregations using legacy BE local shuffle when BEs own only a subset of the table's buckets. - **Root cause:** Ordinary and distinct-streaming aggregation selected FE distribution expressions only for a shuffled-consumer requirement, overlooking the independently propagated bucket-distribution requirement. - **Why existing tests missed it:** They did not combine a bucket-preserving child, a false shuffled-consumer flag, differing grouping and bucket keys, and partial bucket ownership. - **Fix:** Select FE distribution expressions when either shuffled-consumer or bucket-distribution is required; retain the original grouping-key fallback when FE distribution expressions are unavailable or neither requirement applies. - **Other paths and branches:** FE-planned and disabled local shuffle retain their existing behavior. This PR does not change release branches. ### Testing - Before the fix, reproduced `Rows mismatched / BUCKET_HASH_SHUFFLE` locally with three BEs, partial bucket ownership, and aggregation phases 3 and 4; control queries with local shuffle disabled or FE planning enabled returned the expected result. - Built BE and FE locally with ASAN, including FE checkstyle. - Ran focused BE aggregation, local-exchange, and pipeline unit tests: 19 passed. - Generated and verified the SQL regression expected output, then repeated the suite five times; also verified it with two partial-bucket ownership layouts across three BEs. All runs passed. - Ran three adjacent local-shuffle suites and the four-phase aggregation suite; all passed. - Ran BE formatting, build hygiene, clang-tidy, and diff checks; all passed. Full CI, cloud mode, and mixed-version execution were not run. ### Release note Affected multi-stage DISTINCT aggregations no longer fail from incorrect bucket routing under legacy BE local shuffle. ### Check List (For Author) - Behavior changed: Yes (see Details) - Does this need documentation: No - Pre-submission status: Partially verified: local ASAN build, focused unit tests, and three-BE regressions passed; full CI and cloud execution were not run. ### AI Disclosure - Tool: Codex - Generated: Code, tests, and this description - Human review: Pending -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
