github-actions[bot] commented on code in PR #68648:
URL: https://github.com/apache/doris/pull/68648#discussion_r4371902154
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/RefreshMTMVCommand.java:
##########
@@ -194,12 +208,55 @@ protected LogicalPlan createRefreshCommand(MTMV mtmv,
StatementContext statement
statementContext.setExcludedTriggerTables(mtmv.getExcludedTriggerTables());
return createIvmIncrRefreshManager().buildInsertCommand(mtmv);
case COMPLETE:
+
statementContext.setExcludedTriggerTables(mtmv.getExcludedTriggerTables());
+ Map<TableIf, String> tableWithPartKey =
getIncrementalTableMap(mtmv);
+ // Pinned before the context is built, the way a refresh task
pins its inputs before its
+ // own: the mapping, the read predicates and the scan binding
then all resolve one
+ // snapshot per input, instead of the mapping re-materializing
the latest one per MV
+ // partition and the scan possibly binding to a different one
than the mapping read.
+ Map<MvccTableInfo, MvccSnapshot> pinnedSnapshots =
+ pinBaseTableSnapshots(mtmv, statementContext);
+ // The refresh aligns before it reads anything -- the
partitions the base partitions no
+ // longer describe are dropped, the ones they now produce are
created, and only then is the
+ // set to rebuild taken from the MV -- so the preview takes
the same set. A dropped partition
+ // left in it would be read through a scope the refresh never
reads, and being undescribed
+ // there it would also cost the retained partitions their
scopes; see
+ // MTMVPartitionUtil#mappedBasePartitionsByMvPartition. The
partitions the alignment creates
+ // are rebuilt in the same round, so the preview covers their
read scopes as well, even
+ // though it cannot target them in the sink; see
MTMVPartitionUtil#pendingPartitions.
+ CompleteRefreshPreview preview =
+ completeRefreshPreview(mtmv, pinnedSnapshots,
tableWithPartKey);
+ MTMVRefreshContext refreshContext =
+ MTMVRefreshContext.buildContext(mtmv,
Maps.newHashMap(), pinnedSnapshots);
+ // The base partitions a retained MV partition is read from,
from the mapping the refresh
+ // records it with, and those of the partitions the alignment
creates, from the alignment
+ // itself. Paired per partition rather than unioned for the
batch; see
+ // UpdateMvByPartitionCommand#constructTableWithPredicates.
+ Map<String, Map<BaseTableInfo, Set<String>>> readScopes =
+
MTMVPartitionUtil.mappedBasePartitionsByMvPartition(tableWithPartKey,
refreshContext,
+ preview.getSinkPartitionNames());
+ readScopes.putAll(preview.readScopesByPendingName());
if (mtmv.isIvm()) {
-
statementContext.setIvmRewriteContext(Optional.of(IvmRewriteContext.fullExplain(mtmv)));
+ // The explained refresh covers every MV partition in one
plan, which is the refresh-all
+ // first batch a refresh would run: its PCT inputs are
reset to the base partitions the
+ // mapping names -- a table it names as empty is read as
no rows -- and a non-PCT input is
+ // read in RESET mode. Both are derived the way MTMVTask
derives them, from the same
+ // scopes, so the preview's reads are the reads execution
makes; see IvmFullRefreshMTMV.
+
statementContext.setIvmRewriteContext(Optional.of(IvmRewriteContext.fullExplain(mtmv,
Review Comment:
[P2] Preview COMPLETE resets without requiring an existing IVM stream
After a populated IVM MV's internal stream is dropped with `DROP STREAM ...
FORCE`, `EXPLAIN REFRESH MATERIALIZED VIEW mv COMPLETE` now fails. This
`fullExplain(..., RESET)` enables `IvmFullRefreshMTMV`, whose `rewriteScan`
calls `IvmUtil.getIvmStream` and throws `IVM stream not found` (or rejects a
stale/disabled stream). Real COMPLETE calls `MTMVTask.reconcileIvmStreams`
before building these scans and recreates the stream, so the refresh being
previewed succeeds; the existing `test_ivm_partitions_fallback_stream_unusable`
fixture already exercises that recovery state. The previous preview did not
require a stream lookup. Build the equivalent scoped RESET base-table read
without requiring a usable persisted stream, keep EXPLAIN read-only, and cover
EXPLAIN before stream recovery.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
