comphead opened a new pull request, #7674: URL: https://github.com/apache/datafusion-comet/pull/7674
## Which issue does this PR close? Part of #7660. ## Rationale for this change The grouped decimal SUM and AVG accumulators check the running sum's precision on every row with `Decimal128Type::is_valid_decimal_precision`. arrow-rs does not mark that method `#[inline]` ([`types.rs` L1540 in 59.3.2](https://github.com/apache/arrow-rs/blob/59.3.2/arrow-array/src/types.rs#L1540)), and release builds of `libcomet` call it out of line, with its precision guard and two table lookups, on every row. In a TPC-H Q1 profile it took 11.6% of CPU samples across its callers (#7660). ## What changes are included in this PR? - `Decimal128Bounds` in `utils.rs` looks up the bounds for one precision once. Its `contains` is two comparisons. - `SumDecimalGroupsAccumulator` and `AvgDecimalGroupsAccumulator` keep these bounds instead of the precision. They compare against them in update, merge, and SUM's evaluate. Every path accepts exactly the values it accepted before. - `benches/decimal_groups.rs` drives both accumulators through `update_batch` over 4 (TPC-H Q1's shape), 1024, and 64K groups, with nulls, a capped precision, and values near the limit. The ungrouped AVG accumulator is left alone, since #7116 tracks giving it a wider running sum. ## How are these changes tested? The existing tests cover overflow, recovery, and ANSI errors for both accumulators. A new test checks `Decimal128Bounds` against `is_valid_decimal_precision` for precisions 0, 1, 18, 25, 38, and 39, at and around each precision's bound. **Release builds.** These are calls to arrow's out-of-line `<Decimal128Type as DecimalType>::is_valid_decimal_precision` in the release `libcomet.dylib` (macOS aarch64, `cargo build --release`), counted with `objdump --disassemble-symbols`: | Function | `main` | This PR | |---|---|---| | `SumDecimalGroupsAccumulator::update_single` (every row) | 1 | 0 | | `SumDecimalGroupsAccumulator::merge_batch` | 1 | 0 | | `SumDecimalGroupsAccumulator::evaluate` | 2 | 0 | | `AvgDecimalGroupsAccumulator::update_batch` (every row, both loops) | 2 | 0 | | `AvgDecimalGroupsAccumulator::merge_batch` | 1 | 0 | The `main` column comes from the release `libcomet.dylib` of `780a860`. Its `sum_decimal.rs`, `avg_decimal.rs`, and `utils.rs` match this PR's base. The function stays in the library for its other callers, so the zero counts mean the calls are gone, not that the symbol is. **Bench builds** inline arrow's call already, so criterion shows only the part that comes from loading the bounds once. `cargo bench --bench decimal_groups -- --baseline base`, two samples, on a shared Apple M-series laptop (load average about 4): | Shape | SUM | AVG | |---|---|---| | (25,2), 4 groups | −9.4%, −9.7% | +1.1%, +0.6% | | (25,2), 1024 groups | −12.3%, −11.6% | −3.3%, −2.8% | | (25,2), 64K groups | +2.5%, −3.6% | −2.6%, −1.8% | | (25,2), 1024 groups, 10% nulls | −8.3%, −8.9% | −4.6%, −4.3% | | (38,6), 1024 groups | −12.4%, −11.8% | −2.6%, −1.5% | | (38,6), 1024 groups, near limit | −10.4%, −11.2% | +0.5%, −0.4% | DataFusion's `sum` in the same bench, which this PR does not touch, moved −1.8% and +0.5%. The AVG 4-group shape stays within 1.1%. That is criterion's noise threshold, and release builds also remove a call from that loop. For scale, #7660 forced `main`'s call out of line in the bench, as release builds compile it. That made grouped SUM 22% to 30% slower and grouped AVG 10% to 37% slower than with the call inlined. This PR removes the call. I have not measured a release build end to end. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
