comphead opened a new pull request, #7674:
URL: https://github.com/apache/datafusion-comet/pull/7674

   ## Which issue does this PR close?
   
   Part of #7660.
   
   ## Rationale for this change
   
   The grouped decimal SUM and AVG accumulators check the running sum's 
precision on every row with `Decimal128Type::is_valid_decimal_precision`. 
arrow-rs does not mark that method `#[inline]` ([`types.rs` L1540 in 
59.3.2](https://github.com/apache/arrow-rs/blob/59.3.2/arrow-array/src/types.rs#L1540)),
 and release builds of `libcomet` call it out of line, with its precision guard 
and two table lookups, on every row. In a TPC-H Q1 profile it took 11.6% of CPU 
samples across its callers (#7660).
   
   ## What changes are included in this PR?
   
   - `Decimal128Bounds` in `utils.rs` looks up the bounds for one precision 
once. Its `contains` is two comparisons.
   - `SumDecimalGroupsAccumulator` and `AvgDecimalGroupsAccumulator` keep these 
bounds instead of the precision. They compare against them in update, merge, 
and SUM's evaluate. Every path accepts exactly the values it accepted before.
   - `benches/decimal_groups.rs` drives both accumulators through 
`update_batch` over 4 (TPC-H Q1's shape), 1024, and 64K groups, with nulls, a 
capped precision, and values near the limit.
   
   The ungrouped AVG accumulator is left alone, since #7116 tracks giving it a 
wider running sum.
   
   ## How are these changes tested?
   
   The existing tests cover overflow, recovery, and ANSI errors for both 
accumulators. A new test checks `Decimal128Bounds` against 
`is_valid_decimal_precision` for precisions 0, 1, 18, 25, 38, and 39, at and 
around each precision's bound.
   
   **Release builds.** These are calls to arrow's out-of-line `<Decimal128Type 
as DecimalType>::is_valid_decimal_precision` in the release `libcomet.dylib` 
(macOS aarch64, `cargo build --release`), counted with `objdump 
--disassemble-symbols`:
   
   | Function | `main` | This PR |
   |---|---|---|
   | `SumDecimalGroupsAccumulator::update_single` (every row) | 1 | 0 |
   | `SumDecimalGroupsAccumulator::merge_batch` | 1 | 0 |
   | `SumDecimalGroupsAccumulator::evaluate` | 2 | 0 |
   | `AvgDecimalGroupsAccumulator::update_batch` (every row, both loops) | 2 | 
0 |
   | `AvgDecimalGroupsAccumulator::merge_batch` | 1 | 0 |
   
   The `main` column comes from the release `libcomet.dylib` of `780a860`. Its 
`sum_decimal.rs`, `avg_decimal.rs`, and `utils.rs` match this PR's base. The 
function stays in the library for its other callers, so the zero counts mean 
the calls are gone, not that the symbol is.
   
   **Bench builds** inline arrow's call already, so criterion shows only the 
part that comes from loading the bounds once. `cargo bench --bench 
decimal_groups -- --baseline base`, two samples, on a shared Apple M-series 
laptop (load average about 4):
   
   | Shape | SUM | AVG |
   |---|---|---|
   | (25,2), 4 groups | −9.4%, −9.7% | +1.1%, +0.6% |
   | (25,2), 1024 groups | −12.3%, −11.6% | −3.3%, −2.8% |
   | (25,2), 64K groups | +2.5%, −3.6% | −2.6%, −1.8% |
   | (25,2), 1024 groups, 10% nulls | −8.3%, −8.9% | −4.6%, −4.3% |
   | (38,6), 1024 groups | −12.4%, −11.8% | −2.6%, −1.5% |
   | (38,6), 1024 groups, near limit | −10.4%, −11.2% | +0.5%, −0.4% |
   
   DataFusion's `sum` in the same bench, which this PR does not touch, moved 
−1.8% and +0.5%. The AVG 4-group shape stays within 1.1%. That is criterion's 
noise threshold, and release builds also remove a call from that loop.
   
   For scale, #7660 forced `main`'s call out of line in the bench, as release 
builds compile it. That made grouped SUM 22% to 30% slower and grouped AVG 10% 
to 37% slower than with the call inlined. This PR removes the call. I have not 
measured a release build end to end.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to