jaredmdobson opened a new pull request, #70765:
URL: https://github.com/apache/doris/pull/70765
## Design summary
- **Precommit pins the load.** In cloud mode, a MoW 2PC precommit goes
through the BE. The BE writes the tablet/backend mappings, the pending rowset
IDs and a MoW marker into the same `TxnInfoPB` write that precommits.
MetaService must acknowledge the write; an older MetaService fails the request
with "upgrade MetaService".
- **The commit is fenced.** The final commit takes the table's delete-bitmap
lock under an initiator that is unique to that attempt. It then computes the
bitmaps on the BEs and publishes in one FoundationDB transaction. That
transaction checks the lock owner and partition versions, and rechecks that the
pinned rowset set is unchanged. A losing attempt can unlock safely and never
publishes.
- **Fenced results stay out of the read cache.** A fenced attempt's bitmap
is never promoted to the BE's read-side SUCCEED cache. Readers fetch the
winning bitmap from MetaService.
- **Fast path, with a safe fallback.** A fenced attempt reuses the writer's
own delete bitmap as its baseline and computes only the increment against
rowsets published since. The writer entry's lifetime is capped at 30 minutes. A
cache miss, a pinned-rowset mismatch, or a writer bitmap that marks rows of the
pending rowset itself on a sequence-column table instead recovers the pending
rowset from MetaService and recomputes from scratch. Compaction can make those
self-marks stale.
- **Capability gates.** FE advertises
`TStreamLoadPutResult.supports_cloud_mow_2pc`. The BE rejects a MoW 2PC load
with "upgrade FE" before writing anything if the flag is missing.
Partial-update and changelog loads are rejected before writing.
- **Fully filtered loads.** A load whose rows were all filtered can
precommit and commit with no tablet mappings, but only when MetaService finds
no pending rowsets for it. A fenced commit request for such a load is rejected.
- **Upgrade order:** MetaService → Recycler → BE → FE.
### Background
A two-phase stream load writes temporary rowsets, precommits, and is
published by a separate commit request. On cloud merge-on-write tables,
publishing needs delete bitmaps calculated against the latest visible partition
versions, under the table's delete-bitmap lock. Today both cloud stream-load
entry points reject MoW 2PC.
### What problem does this PR solve?
Issue Number: None
Related PR: #36816 (added the guards that reject cloud MoW 2PC)
PR Type: Feature
Problem Summary:
Cloud users cannot run two-phase commit stream loads on merge-on-write
tables, so exactly-once sinks into cloud MoW tables cannot use 2PC. Removing
the guards alone would publish rowsets without the delete bitmaps they need.
This PR persists the load's tablet mappings at precommit and runs the existing
MoW lock and bitmap flow at commit, fenced per attempt. Most of the diff is
tests. Existing 1PC and non-MoW 2PC loads behave as before.
### Details
- **Change map:** about 1,280 changed lines of production code, 1,760 of
tests and 30 of protobuf/thrift.
- MetaService precommit/commit, fence, pinned rowset set and empty-load
path: `meta_service_txn.cpp`, `meta_service.cpp`, `meta_service_helper.h`.
- BE precommit, fenced bitmap calculation, writer-baseline reuse and
fallback: `cloud_engine_calc_delete_bitmap_task.cpp`, `cloud_meta_mgr.cpp`,
`cloud_tablet.cpp`, `cloud_txn_delete_bitmap_cache.cpp`.
- Admission gates: `stream_load.cpp`, `http_stream.cpp`,
`cloud_stream_load_executor.cpp`.
- FE commit flow: `CloudGlobalTransactionMgr.java`.
- Recycler retention of rowsets of PRECOMMITTED MoW 2PC transactions:
`recycler.cpp`.
- **Scope:** Cloud mode, `stream_load` and `http_stream`, merge-on-write
unique-key tables, full-row loads.
- **Design:** See the summary above.
- **Usage:** Unchanged. `two_phase_commit: true` on a cloud MoW table now
succeeds.
- **Default behavior:** Unchanged for 1PC and non-MoW 2PC. Lazy commit is
not used for 2PC commits.
- **Compatibility:**
- Upgrade MetaService, then Recycler, then BE, then FE. Each mixed-version
combination fails closed with an "upgrade MetaService" or "upgrade FE" error.
- Serialized `TxnInfoPB` is limited by FoundationDB's 100,000-byte value
limit. A precommit whose mappings exceed it is rejected.
- Rows stay PRECOMMITTED for the FE precommit window. During that window
the recycler retains the pending rowsets.
### Testing
UT Job on a Linux ARM64 build of the PR head:
| Step | Result |
|---|---|
| FE `CloudGlobalTransactionMgrTest`, `StreamLoadHandlerTest`,
`HttpStreamLoadTablePropertiesTest` | 81 run, 0 failed |
| FE Checkstyle | passed |
| Cloud `meta_service_test`, all enabled tests except `Fdb/*`, which needs a
real FoundationDB | 309 run, 0 failed |
| Cloud `recycler_test` | 212 run, 0 failed |
| `doris_cloud` link | passed |
| BE `CloudMow2PCTest`, `StreamLoadTest`, `CloudMetaMgrTest` | 37 run, 0
failed |
| clang-format 16 on changed C++ | passed |
Regression suites (`load_p0/stream_load/test_cloud_stream_load_2pc_mow`,
`load_p0/http_stream/test_cloud_http_stream_2pc_mow`,
`fault_injection_p0/cloud/test_cloud_mow_2pc_cache_miss`) were not run in a
cloud regression environment for this PR.
### Notes for Reviewers
- **Recycler race (pre-existing, 1PC too):** the recycler can queue a
PREPARED owner's rowset for deletion under immediate recycling or near-zero
retention. The defaults close it: `retention_seconds=259200` and
`force_immediate_recycle=false`. `enable_mark_delete_rowset_before_recycle` and
`enable_abort_txn_and_job_for_delete_rowset_before_recycle` both default to
false, and setting either to true also closes it. This PR adds the counter
`recycler_tmp_rowsets_retained_owner_lookup_failed`.
- **1PC stale self-mark (pre-existing):** the 1PC incremental publish has
the same window, with a stale writer self-mark after compaction on
sequence-column tables, only shorter. It is unchanged here.
- **Counters:** `cloud_mow_2pc_baseline_recompute_count` counts fallbacks
caused by writer self-marks. `cloud_mow_2pc_cache_recovery_count` counts cache
misses and pinned-rowset mismatches.
- **Where to start reading:** `MetaServiceImpl::precommit_txn` and
`commit_txn_immediately`, then
`CloudTabletCalcDeleteBitmapTask::_handle_rowset`.
### Release note
Cloud stream load and http_stream support two-phase commit on merge-on-write
tables. Upgrade MetaService, Recycler, BE and FE in that order.
### Check List (For Author)
- Behavior changed: Yes (see Details)
- Does this need documentation: Yes. The cloud stream-load 2PC limitation
and the upgrade order need updating; a doris-website PR will follow.
- Pre-submission status: Partially verified. The unit tests, checkstyle,
link and format passed (see Testing). The cloud regression suites were not run.
### AI Disclosure
- Tool: Codex (implementation and tests), Claude Code (specs, reviews and
verification).
- Generated: Code, tests and this description.
- Human review: Pending. The author reviews before this PR leaves draft.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]