BiteTheDDDDt commented on code in PR #65853:
URL: https://github.com/apache/doris/pull/65853#discussion_r3637390265
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/glue/translator/RuntimeFilterPartitionPruneClassifier.java:
##########
@@ -173,19 +173,36 @@ private static boolean isPartitionColumnSlot(SlotRef
slotRef, List<Column> parti
}
private static boolean sameColumn(Column targetColumn, Column
partitionColumn) {
+ targetColumn = getBaseColumn(targetColumn);
+ if (targetColumn == null) {
+ return false;
+ }
if (targetColumn == partitionColumn) {
return true;
}
+ // Metadata reload and deep copy do not preserve Column object
identity.
+ // Prefer stable unique IDs when present; legacy schemas without unique
+ // IDs must fall back to structural equality.
int targetUniqueId = targetColumn.getUniqueId();
int partitionUniqueId = partitionColumn.getUniqueId();
if (targetUniqueId != Column.COLUMN_UNIQUE_ID_INIT_VALUE
- && partitionUniqueId != Column.COLUMN_UNIQUE_ID_INIT_VALUE
- && targetUniqueId == partitionUniqueId) {
- return true;
+ && partitionUniqueId != Column.COLUMN_UNIQUE_ID_INIT_VALUE) {
+ return targetUniqueId == partitionUniqueId;
}
return targetColumn.equals(partitionColumn);
}
+ private static Column getBaseColumn(Column targetColumn) {
+ if (!targetColumn.isMaterializedViewColumn()) {
+ return targetColumn;
+ }
+ Expr defineExpr = targetColumn.getDefineExpr();
+ if (defineExpr instanceof SlotRef && ((SlotRef)
defineExpr).getColumn() != null) {
+ return ((SlotRef) defineExpr).getColumn();
Review Comment:
Fixed in `ad3bc5b0471`. I added a shared
`RuntimeFilterPartitionColumnResolver` used by both the classifier and
`OlapScanNode`; it rejects `targetColumn.isAggregated()` before unwrapping a
materialized column's `defineExpr`, so `SUM(k)` can no longer inherit raw `k`
partition boundaries. The regression directly scans the aggregate MV index:
source values `6,7` in `k=[0,10)` produce `sum_k=13`, the query returns `1 /
13`, and the profile asserts `total=0, pruned=0`. The focused FE suite passes
15/15 and `rf_partition_pruning` passes both generated-output and normal
comparison runs.
##########
fe/fe-core/src/main/java/org/apache/doris/planner/OlapScanNode.java:
##########
@@ -1408,22 +1413,64 @@ private void setPartitionBoundaries(TOlapScanNode
olapScanNode) {
if (item == null) {
continue;
}
- if (item instanceof RangePartitionItem) {
- addRangeBoundaries(boundaries, partitionId,
(RangePartitionItem) item,
- partColumns, partColToSlotId);
- } else if (item instanceof ListPartitionItem) {
- addListBoundaries(boundaries, partitionId, (ListPartitionItem)
item,
- partColumns, partColToSlotId);
+ for (Map.Entry<Integer, Integer> entry :
pruningSlotToPartitionColumnIndex.entrySet()) {
+ int slotId = entry.getKey();
+ int partitionColumnIndex = entry.getValue();
+ if (item instanceof RangePartitionItem) {
+ Preconditions.checkState(partitionColumnIndex == 0);
+ addRangeBoundary(boundaries, partitionId,
(RangePartitionItem) item,
+ partColumns.size(), slotId);
+ } else if (item instanceof ListPartitionItem) {
+ addListBoundary(boundaries, partitionId,
(ListPartitionItem) item,
+ partitionColumnIndex, slotId);
+ }
}
}
if (!boundaries.isEmpty()) {
olapScanNode.setPartitionBoundaries(boundaries);
}
}
- private void addRangeBoundaries(List<TPartitionBoundary> boundaries, long
partitionId,
- RangePartitionItem rangeItem, List<Column> partColumns,
- Map<String, Integer> partColToSlotId) {
+ private int findPartitionColumnIndex(Column targetColumn, List<Column>
partitionColumns) {
+ targetColumn = getBaseColumn(targetColumn);
+ if (targetColumn == null) {
+ return -1;
+ }
+ for (int i = 0; i < partitionColumns.size(); i++) {
+ Column partitionColumn = partitionColumns.get(i);
+ if (targetColumn == partitionColumn) {
+ return i;
+ }
+ // Metadata reload and deep copy do not preserve Column object
identity.
+ // Prefer stable unique IDs when present; legacy schemas without
unique
+ // IDs must fall back to structural equality.
+ int targetUniqueId = targetColumn.getUniqueId();
Review Comment:
Fixed in `ad3bc5b0471`. The shared resolver now compares unique IDs only
inside the base-index domain (or after a sync-MV `SlotRef` has resolved to its
bound base column). For an ordinary selected rollup, it maps only the rollup's
preserved base column name and never compares rollup-local IDs with base-index
IDs. The regression creates the exact light-schema-change collision `(id uid0,
k uid1, v uid2)` / rollup `(id uid0, v uid1)`, directly scans the rollup,
returns `1 / 107`, and asserts `total=0, pruned=0`. I also added focused unit
cases for rejecting the collision and accepting the actual partition column.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
