yujun777 opened a new pull request, #68193: URL: https://github.com/apache/doris/pull/68193
### What problem does this PR solve? Trace issue: https://github.com/apache/doris/issues/65418 **This PR adds no behaviour of its own. It adds the state that the following PRs need, and the channel that persists it, so that they can be reviewed as logic alone.** An IVM materialized view has to invalidate the MV partitions that a base-table change really affected. A partition drop / truncate / replace / recover changes the base table through metadata and emits no row binlog, so the affected MV partitions must be rebuilt; today the only answer the MV has is "rebuild all of them", which throws away partitions that are still correct. Deciding per partition needs a per-partition answer to two questions: - which generation of data does this MV partition currently hold? - which generation must it hold? That pair is `MTMVPartitionState { refreshEpoch, latestEpoch }`, one entry per MV partition, keyed by partition name. `latestEpoch` is the requirement, `refreshEpoch` is the reality, and a partition whose requirement is ahead of its reality is dirty: it holds rows read before a change that left no binlog, so it can no longer be maintained incrementally and has to be rebuilt. The requirement has to survive a restart, because an invalidation that only lives in memory is lost the moment the FE restarts, and a partition that is then refreshed incrementally keeps the stale rows forever with no error anywhere. So this PR adds 1. the state itself (`MTMVPartitionState`, plus a copy helper for taking a detached snapshot), and 2. the channel that carries it into the journal and back: a field on the MV, its own field on the alter record, a dedicated alter op with its replay branch, and the replay handling of the task result. Nothing in the FE decides anything from the state yet, and nothing but a replay ever writes it, so every MV behaves exactly as before. That is deliberate: it makes this step independently mergeable and independently testable, which is what the PR that starts using the state needs underneath it. ### Scope | | | | --- | --- | | Adds | `MTMVPartitionState`, the `partitionStates` field, the alter record field, the new alter op and its replay branch | | Does not touch | any criterion, routing or invalidation decision; `IvmInfo`; the refresh path | | Field is shared, behaviour is not | the field sits on the MV, so both kinds of MV carry it; only an IVM MV ever populates it, and the journal of a non-IVM MV stays byte-for-byte what it was | | Compatibility | an image written before this PR has no such field and loads as an empty map; an ADD_TASK journal written before it applies nothing on replay instead of clearing what is there | The state is on the MV rather than inside `IvmInfo`, and the alter op is its own rather than riding on `ALTER_IVM_INFO`, whose branch only swaps the `IvmInfo` object. Both are structural: the same state is meant to serve a non-IVM MV later, and its journal payload must not be reconstructed as a side effect of replaying some other op. ### Key changes - Add `MTMVPartitionState`, a persisted `refreshEpoch` / `latestEpoch` pair keyed by MV partition name, and `MTMVPartitionState.copyOf` for taking a detached snapshot of a state map. A partition gets a new id on every refresh, so the name is the only identity it can have. - Add `MTMV.partitionStates` with its getter and its replay setter; `gsonPostProcess` initializes it, so an image written before the field existed and a non-IVM MV both load as an empty map. - Carry the state in the ADD_TASK payload under the same condition as `ivmInfo`, which keeps the journal of a non-IVM MV byte-for-byte unchanged, and apply it on replay only when the field is present, so an old journal applies nothing rather than clearing the state. - Add `MTMVAlterOpType.ALTER_PARTITION_STATES` and its `Alter.processAlterMTMV` branch. ### Release note None ### Check List (For Author) - Test - [ ] Regression test - [x] Unit Test <!-- image round trip, an image and an ADD_TASK journal written before the field existed, both replay directions, the payload of a non-IVM MV --> - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - Behavior changed: - [x] No. <!-- the state is never populated and never read outside a replay --> - Does this need documentation? - [x] No. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
