jackkleeman opened a new issue, #20445: URL: https://github.com/apache/datafusion/issues/20445
### Describe the bug Partial aggregation with `PartiallySorted` ordering panics when memory pressure triggers the `EmitEarly` OOM path: ``` assertion failed: *current_sort >= n at aggregates/order/partial.rs:174 ``` #19287 changed `Partial` aggregation to always use `OutOfMemoryMode::EmitEarly`: ```rust (AggregateMode::Partial, _) => OutOfMemoryMode::EmitEarly, ``` The `EmitEarly` handler calls `emit(EmitTo::First(n))` with `n` based on batch size, ignoring the sort boundary tracked by `GroupOrderingPartial`. Previously, `emit_early_if_necessary()` had a guard: ```rust && matches!(self.group_ordering, GroupOrdering::None) ``` This guard was lost in the refactor. ## Trigger conditions 1. `Partial` aggregate with `PartiallySorted` ordering (e.g. `GROUP BY a, b` where input is sorted on `a`) 2. Memory pressure triggers `EmitEarly` before a sort key boundary is reached 3. Many groups accumulate with the same sort key value (so `current_sort` stays at 0 while `n` grows) ### To Reproduce A unit test is included in the linked PR ### Expected behavior OOM, but not a assertion panic ### Additional context _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
