yujun777 opened a new pull request, #70630: URL: https://github.com/apache/doris/pull/70630
### Background Incremental materialized views maintain outer joins by writing joined-row changes and repairs for rows whose nullable side is filled with NULL. Unique-key views apply these writes through merge-on-write keys and sequence numbers. ### What problem does this PR solve? Issue Number: None Related PR: #62606 (introduced the bug) PR Type: Bugfix Problem Summary: Incremental refresh can lose a padded NULL row when the nullable-side join subtree emits cancelling positive and negative events. Order NULL-row repairs by their operation sign so reinsertion wins when both repairs target the same row. For a view over `l LEFT JOIN (a JOIN b)`, updating the outer join key in `a` and deleting its matching `b` row in the same refresh window produced: | Result | Rows | |---|---| | Before the fix, one observed incremental refresh | `(1,10,NULL)` | | With the fix, incremental and complete refresh | `(1,10,NULL), (2,20,NULL)` | ### Details - **Frequency:** Intermittent when changes to both inputs of a nullable-side join produce cancelling rows within one refresh window. - **Root cause:** `paddingSequence()` ignored the repair sign. Both the NULL-row deletion and reinsertion used the same sequence, so merge-on-write could retain the deletion according to arrival order. - **Why existing tests missed it:** `IvmJoinDeltaHandlerTest` checked the repair sign and sequence column name without checking their ordering. `test_ivm_refresh_dry_run` masks the sequence value, while final-row checks can pass when reinsertion happens to arrive last. - **Fix:** Reuse the existing signed sequence encoder in both the general repair and equality-event paths. Preserve the refresh prefix and delta index, and distinguish repairs using the existing operation bit without adding scans or joins. - **Other paths and branches:** LEFT, RIGHT and FULL joins share these repair producers. BIGINT and LARGEINT encodings are covered; the aggregate final-sequence path and append-only detail model are unchanged. The handler introduced by #62606 is absent from branch-3.1, branch-4.0, branch-4.1 and branch-4.2. - **Behavior change:** DRY RUN now exposes an even sequence for negative NULL-row repairs and a larger odd sequence for positive repairs; its result schema is unchanged. ### Testing - Three targeted unit tests fail on the unpatched code at the repair-sequence assertions. - The IVM regression directory passed locally: 117 suites, zero failures and zero skips, including the new `test_ivm_outer_join_null_repair_sequence` case for equality events, residual predicates, RIGHT JOIN and FULL JOIN. - IVM FE unit tests and create/show/drop materialized-view tests were run locally; the final combined verification is in progress. ### Release note None ### Check List (For Author) - Behavior changed: Yes (see Details) - Does this need documentation: No - Pre-submission status: Partially verified: final combined FE unit-test run is in progress -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
