HappenLee opened a new pull request, #69690: URL: https://github.com/apache/doris/pull/69690
### What problem does this PR solve? Related PR: https://github.com/StarRocks/starrocks/pull/81481 Problem Summary: `map(k, v)` serializes its keys and allocates deduplication buffers even though each output row has only one pair. String `array_distinct` grows its character buffer and copies each retained value separately. - Skip `deduplicate_keys()` for a single key/value pair. Multiple-pair construction keeps its existing last-value-wins deduplication. - Collect retained positions and call `insert_indices_from()` once for string batches without NULL elements. Use the existing SIMD NULL-map scan to select the scalar path when any NULL exists; this avoids copying hidden string payloads beneath NULLs. Nullable inputs with an all-zero NULL map still use batch copying. - Keep scalar copying when the nested element count exceeds the UInt32 selector limit, and use `insert(...).second` to avoid separate hash lookups. First-occurrence ordering and SQL NULL semantics are unchanged. Tests cover NULL keys/values, constant keys, multiple-pair deduplication, string ordering, hidden NULL payloads, a NULL at the end of the input, empty inputs, long strings and selector growth. Paired Release microbenchmarks used the complete original and updated `execute_impl` entry points in one process with identical inputs. Results below are median CPU milliseconds per 4,096 rows/arrays on a pinned Xeon Platinum 8457C CPU, seven repetitions (selected earlier MAP cases had nine). These are function-level measurements, not end-to-end SQL speedups. | Case | Before (ms) | After (ms) | Before / after | | --- | ---: | ---: | ---: | | Single-pair MAP, BIGINT key/value | 0.0804 | 0.0629 | 1.28x | | Single-pair MAP, 16-byte string key, BIGINT value | 0.0986 | 0.0753 | 1.31x | | Single-pair MAP, 128-byte string key, BIGINT value | 0.1665 | 0.1033 | 1.61x | | ARRAY, 32 distinct 16-byte strings | 2.063 | 1.664 | 1.24x | | ARRAY, 32 distinct 128-byte strings | 22.993 | 10.550 | 2.18x | | ARRAY, 128 distinct 128-byte strings | 116.683 | 57.871 | 2.02x | | ARRAY, 128 x 128-byte strings, 25% retained | 31.968 | 20.596 | 1.55x | | ARRAY, 128 x 128-byte strings, one distinct value | 10.792 | 11.104 | 0.97x | | ARRAY, 32 NULL elements with empty payloads | 0.1173 | 0.1219 | 0.96x | | ARRAY, 8 NULL elements with 4KiB hidden payloads | 0.0400 | 0.0413 | 0.97x | Highly duplicated and all-NULL inputs showed little benefit; small differences in these controls were within the observed sample variability. NULL batches keep scalar copying. Temporary frozen-baseline benchmark code and build-entry changes are excluded from this PR. ### Release note Improve execution efficiency of single-pair MAP construction and string ARRAY_DISTINCT batches without NULL elements, preserving SQL results. ### Check List (For Author) - Test - [ ] Regression test - [ ] Unit Test — 8 ARRAY tests added and 3 MAP tests added; both test translation units type-checked with clang-tidy. The complete BE unit suite was not executed locally; these cases require CI execution. - [x] Manual test — Release build via `build.sh --benchmark -j48`; paired function microbenchmarks, including 30 ARRAY implementation/input combinations with seven repetitions. Before timing, validate every output row and result byte size. MAP benchmarks also validate the complete output against the original constructor. clang-format 16.0.6, build hygiene, diff checks and changed-code clang-tidy passed. - Behavior changed: - [x] No. - [ ] Yes. - Does this need documentation? - [x] No. - [ ] Yes. ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
