txwyy123 commented on PR #26149: URL: https://github.com/apache/datafusion/pull/26149#issuecomment-6178060678
@jayzhan211 I investigated `column_count/64` thoroughly. Updated A/B benchmark (3 runs each, same machine, back-to-back): | Case | main (ns) | PR (ns) | Δ% | |------|-----------|---------|-----| | `column_count/1` | 16.6 | 17.9 | +7.8% (+1.3ns) | | `column_count/4` | 57.8 | 53.8 | **−6.9%** | | `column_count/16` | 269 | 223 | **−17.1%** | | **`column_count/64`** | **2,181** | **2,112** | **−3.2%** | | `row_count/*` | 57.5 | 53.4 | **−7.1%** | | `array_layout/prim` | 57.8 | 53.4 | **−7.6%** | | `array_layout/list` | 130 | 119 | **−8.5%** | | `array_layout/struct` | 174 | 159 | **−8.6%** | | `shared_slices/4` | 1,807 | 1,636 | **−9.5%** | | `shared_slices/16` | 8,710 | 7,595 | **−12.8%** | | `shared_slices/64` | 34,777 | 33,190 | **−4.6%** | `column_count/64` is now **−3.2%** (faster than main). The +7% from my earlier benchmark was system noise — the effect is not reproducible across multiple fresh runs. The `#[inline(always)]` + `.position()` changes you suggested were sufficient. **13 of 14 cases improved**, `column_count/1` is +1.3ns absolute (structural cost of NonZero wrapping + refcount init on 2 buffers — unavoidable for any refcounted tracker). I also investigated a lazy `HashSet→HashMap` upgrade approach (stay in 8-byte/slot HashSet while count-only, upgrade to 16-byte HashMap on first decrement). It eliminated the theoretical overflow regression but introduced +8% regression on *all* inline cases due to the `Option<enum>` discriminant breaking niche optimization + larger struct footprint. Since the current numbers are already clean, the complexity isn't worth it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
