wudidapaopao opened a new issue, #25847: URL: https://github.com/apache/datafusion/issues/25847
### Is your feature request related to a problem or challenge? DataFusion represents row-count aggregates such as `COUNT(*)` as `COUNT(1)`. During execution, the scalar `1` is expanded into a full `Int64Array` for every input batch, although the accumulator only needs the number of rows. Similarly, `COUNT(non_nullable_column)` unnecessarily keeps the column in the scan. This was identified while profiling an `AVG(expr)` decomposition in https://github.com/apache/datafusion/pull/25536#discussion_r4065767785: a separate `COUNT` could make the decomposed plan slower because of argument-array materialization. ### Describe the solution you'd like Simplify non-`DISTINCT` `COUNT` calls whose arguments are safe to elide and provably non-null (for example, non-null literals and direct non-nullable columns) to a nullary `COUNT()`, while preserving the original output name. Teach aggregate execution to pass the input row count explicitly so nullary `COUNT` works without materializing an argument array in grouped and ungrouped aggregation. Nullable arguments, `DISTINCT`, and arbitrary expressions that may error or be volatile should not be rewritten. ### Describe alternatives you've considered - Rewrite to `COUNT(1)`: enables column pruning but still materializes an array per batch. - Cache arrays for literal aggregate arguments: helps full batches, but is less effective for filtered or variable-sized batches and can complicate memory accounting when slices are retained. ### Additional context Related prior discussions: - https://github.com/apache/datafusion/issues/9943 - https://github.com/apache/datafusion/issues/11686 - https://github.com/apache/datafusion/pull/14824 In a targeted local benchmark over one million in-memory rows, avoiding argument materialization improved single-threaded `SELECT COUNT(*) FROM t WHERE condition` by up to approximately 27% at high filter selectivity. Typical ClickBench queries showed no measurable regression or improvement. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
