yujun777 commented on code in PR #68648:
URL: https://github.com/apache/doris/pull/68648#discussion_r4143093521
##########
fe/fe-core/src/main/java/org/apache/doris/catalog/MTMV.java:
##########
@@ -1608,9 +1608,11 @@ public Map<String, Map<MTMVRelatedTableIf, Set<String>>>
calculatePartitionMappi
Map<PartitionKeyDesc, Map<MTMVRelatedTableIf, Set<String>>>
pctPartitionDescs = MTMVPartitionUtil
.generateRelatedPartitionDescs(mvPartitionInfo, mvProperties,
getPartitionColumns(),
effectiveFilter, pinnedSnapshots);
+ Map<MTMVRelatedTableIf, String> defaultListPartitions =
defaultListPartitionsOf();
for (Entry<String, PartitionItem> entry : mvPartitionItems.entrySet())
{
- res.put(entry.getKey(),
-
pctPartitionDescs.getOrDefault(entry.getValue().toPartitionKeyDesc(),
Maps.newHashMap()));
+ res.put(entry.getKey(), withDefaultListPartitions(
Review Comment:
Fixed in 5ba21b735d2, by preserving the whole inverse set.
`MTMVRewriteUtil.getPctToMv` answers each base partition with the set of MV
partitions that read it now, and the rewrite's partition selection adds all of
them, so a query that reads a default partition is held against every MV
partition its rows can be in rather than against whichever one the map happened
to keep. The UT covers the fan-out (`t1_default` read by `mv_a` and `mv_b`,
answered with both).
That is the other half of the fan-out the previous commit introduced, and it
is what the mapping's own contract says: the mapping is what the read and the
record agree on, so anything inverting it has to keep all of it.
##########
regression-test/suites/mtmv_p0/test_mtmv_base_partition_read_scope.groovy:
##########
@@ -0,0 +1,224 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+suite("test_mtmv_base_partition_read_scope") {
+ String dbName = context.config.getDbNameByFile(context.file)
+ // A refresh reads the base partitions the MV partition is recorded with,
and no others. The window of
+ // partition_sync_limit below keeps the last two days, so the day before
them is recorded nowhere: it
+ // must not be read either, or its rows would sit in the MV partition --
whose key range does cover
+ // them -- while the snapshot says the MV does not hold that partition. A
base partition dropped after
+ // such a read would then be invisible to the sync check, and the
transparent rewrite would serve the
+ // rows of a partition the base table no longer has.
+ //
+ // The MV partition is a year and the base table's are days, so the range
it is read through is wider
+ // than what it is recorded with. Days are taken relative to today, and
the two sides are asserted
+ // separately rather than as a total, so that where the window's edge
falls does not decide the case.
+ def today = java.time.LocalDate.now()
Review Comment:
Fixed in 5ba21b735d2. The fixture's days now come from `select curdate()`,
the FE session's own date, which is the calendar `partition_sync_limit` cuts
its window from, instead of the runner's local date.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
