QinXi-ai commented on issue #14574: URL: https://github.com/apache/datafusion/issues/14574#issuecomment-5614236211
I'd like to work on a scoped implementation for fully ordered group keys, starting with multi-column keys and keeping the existing paths for partially ordered and unordered input. I checked main at `423df6f4`: `OrderedAggregateTableBuffer` still calls `new_group_values`, and fully ordered multi-column inputs use `GroupValuesColumn<true>`. I also read #9662 and its review: the mixed performance results, especially the single-column regression, are a reason to establish a repeatable benchmark before choosing where to enable a new implementation. My first step is a benchmark and correctness prototype covering cardinality, key types, run lengths, and groups spanning input batches. The implementation would preserve GROUP BY equality semantics and the existing emission/index contracts. I will compare both the grouping microbenchmark and end-to-end ordered aggregation against current main, and leave single-column specializations alone unless measurements justify changing them. I am treating #23986 as related work: it accelerates consecutive keys while retaining the general hash-based path; this issue can use a proven full ordering to avoid that path. I'll keep that boundary explicit and credit the earlier implementation in #9662. Suggestions on representative ordered workloads or overlapping work are welcome. This work is AI-assisted. So far I have inspected source and previous discussion; I have not yet run performance or correctness experiments and am not claiming a speedup. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
