gianm commented on code in PR #19061:
URL: https://github.com/apache/druid/pull/19061#discussion_r2897710048
##########
multi-stage-query/src/main/java/org/apache/druid/msq/exec/ControllerImpl.java:
##########
@@ -1824,9 +1832,20 @@ private static Function<Set<DataSegment>,
Set<DataSegment>> addCompactionStateTo
);
DimensionsSpec dimensionsSpec = dataSchema.getDimensionsSpec();
- CompactionTransformSpec transformSpec =
TransformSpec.NONE.equals(dataSchema.getTransformSpec())
- ? null
- :
CompactionTransformSpec.of(dataSchema.getTransformSpec());
+
+ // if the clustered by requires virtual columns, preserve them here so
that we can rebuild during compaction
+ CompactionTransformSpec transformSpec;
+ if (clusterBy == null || clusterBy.getVirtualColumnMap().isEmpty()) {
+ transformSpec = TransformSpec.NONE.equals(dataSchema.getTransformSpec())
+ ? null
+ :
CompactionTransformSpec.of(dataSchema.getTransformSpec());
+ } else {
+ transformSpec = new CompactionTransformSpec(
+ dataSchema.getTransformSpec().getFilter(),
+ VirtualColumns.create(clusterBy.getVirtualColumnMap().values())
Review Comment:
By real columns I mean actual physically stored columns. I didn't catch the
change that recently added `virtualColumns` to `CompactionTransformSpec`. I
suppose I assumed they worked the same way as `transforms` on the regular
`TransformSpec`, in that they actually create columns. It would be good to have
javadocs on `CompactionTransformSpec` that explain that the `virtualColumns`
are just for use by the `filter`.
But I guess in this patch you're adding a new use for them? How does the new
use work?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
