parthchandra opened a new issue, #26891: URL: https://github.com/apache/datafusion/issues/26891
### Describe the bug Filed as a follow up to review comments in https://github.com/apache/datafusion-comet/pull/7191#discussion_r4407318204. In datafusion-functions-aggregate/src/min_max/min_max_struct.rs, set_value adds get_array_memory_size() bytes to total_data_bytes for each stored struct. But emit_to(EmitTo::First(n)) subtracts s.len() for each emitted value, which is 1 for a one-row StructArray. So size() keeps counting memory for groups that were already emitted. It only resets on EmitTo::All. Ordered aggregation emits First(n) after every batch, so the reported size grows with the number of groups seen. Reproduction: a Partial AggregateExec feeding a Final one over sorted input, with 4,096 groups of 4 KiB struct values, 64-row batches, a 2 MiB GreedyMemoryPool and the disk manager disabled. It fails with ResourcesExhausted in OrderedFinalAggregateStream. Subtracting get_array_memory_size() of the emitted values instead makes it complete. The code looks copied from min_max_bytes, where len() is a byte count. ### To Reproduce _No response_ ### Expected behavior _No response_ ### Additional context _No response_ ### Willingness to contribute None -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
