zghong commented on PR #67507: URL: https://github.com/apache/doris/pull/67507#issuecomment-6505194598
### 1 Numbering cross-reference We numbers the changes as "optimization points 1..4". The mapping: | Optimization | Ablation report | |---|---| | Optimization 1: comparator by reference | `byref` | | Optimization 2: comparator as template | `byref_tmpl` | | Optimization 3: compact DUP row tracking | `opt1` vs `byref_tmpl` | | Optimization 4: zero-key skip sort | - | ### 2 Variant definitions | Variant | Content | Isolates | |---|---|---| | `master` | `origin/master` = `1d132b6a46b` | baseline | | `byref` | master + comparator by reference (`patches/byref.patch`, one line) | optimization 1 | | `byref_tmpl` | byref + comparator as a template parameter (`patches/byref_tmpl.patch`) | optimization 2 | | `opt1` | `1ed3cf43364` (before the refactor: optimizations 1+2+3+4) | optimizations 3, 4 | | `head` | `822ac43567d` (= `opt1` + refactor) | refactor | ### 3 Absolute timings (min, ms) | Variant | DUP single batch | UNIQUE single batch | AGG single batch | |---|---:|---:|---:| | `master` | 63.302 | 63.506 | 63.283 | | `byref` | 13.094 | 14.546 | 14.562 | | `byref_tmpl` | 10.890 | 12.393 | 12.384 | | `opt1` | 7.839 | 12.264 | 12.390 | | `head` | 7.891 | 12.277 | 12.369 | ### 4 Key scenarios (min, ms) | Scenario | `master` | `byref_tmpl` | `opt1` | `head` | |---|---:|---:|---:|---:| | `dup_single_batch` | 61.751 | 10.961 | 7.876 | 7.977 | | `dup_multi_batch_1k` | 53.284 | 10.826 | 8.067 | 8.227 | | `dup_multi_batch_lsn` | 53.242 | 10.914 | 8.316 | 8.442 | | `dup_three_key_columns` | 66.949 | 12.572 | 9.083 | 9.241 | | `dup_no_dup_keys` | 7.235 | 2.648 | 0.880 | 0.884 | | `dup_zero_key_columns` | 2.165 | 2.161 | 0.184 | 0.185 | | `agg_all_dup_keys` | 8.977 | 4.288 | 4.210 | 4.135 | | `unique_cluster_keys` | 64.968 | 13.293 | 13.226 | 13.122 | ### 5 Methodology & limitations | Item | Setting | |---|---| | Measured region | `MemTable::create` → `insert` (1 or N batches) → `to_block` → destructor | | Outside the region | schema / descriptor table / input `Block` construction (built once, before the loop) | | Build | `BUILD_TYPE=Release`, `./build.sh --benchmark`, clang-21 | | Isolation | one worktree, sequential checkout + incremental builds; benchmark code copied from the reference tree | | Run | single-threaded, pinned with `taskset`, local FE/BE cluster stopped during the run | | Sampling | 5 interleaved rounds × 5 repetitions = 25 samples per cell (round 1: A..F, round 2: A..F, …) | | Machine | 96-core shared host, load average 19–35 during the runs, CPU frequency scaling enabled | **Do not extrapolate to**: partial update (fixed/flexible + skip bitmap), UNIQUE merge-on-read, the flush path, variable-length columns (STRING / ARRAY), concurrent multi-threaded loads, or end-to-end load throughput. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
