gboucher90 commented on code in PR #20858:
URL: https://github.com/apache/datafusion/pull/20858#discussion_r2914617665
##########
datafusion/physical-plan/src/aggregates/row_hash.rs:
##########
@@ -1267,6 +1267,18 @@ impl GroupedHashAggregateStream {
// on the grouping columns.
self.group_ordering =
GroupOrdering::Full(GroupOrderingFull::new());
+ // Recreate group_values to use streaming mode
(GroupValuesColumn<true>
+ // with scalarized_intern) which preserves input row order, as
required
+ // by GroupOrderingFull. This is only needed for multi-column
group by,
+ // since single-column uses GroupValuesPrimitive which is always
safe.
+ let group_schema = self
+ .spill_state
+ .merging_group_by
+ .group_schema(&self.spill_state.spill_schema)?;
+ if group_schema.fields().len() > 1 {
+ self.group_values = new_group_values(group_schema,
&self.group_ordering)?;
Review Comment:
Note:
> only recreates group_values for multi-column GROUP BY (fields().len() >
1), since single-column uses GroupValuesPrimitive which doesn't have the
vectorized_intern bug. This avoids the memory overhead that was causing
aggregate_source_not_yielding_with_spill to fail with its tight 2600-byte
budget.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
