hubgeter opened a new pull request, #69176:
URL: https://github.com/apache/doris/pull/69176
### What problem does this PR solve?
Issue Number: N/A
Problem Summary:
For an Iceberg table, a row-level DELETE always scans the matching rows and
writes delete files (position deletes on v2, deletion vectors on v3), so a data
file whose rows are all gone stays in the snapshot and every later scan still
reads it.
This PR adds the whole-file "metadata delete" fast path that Spark /
StarRocks / Trino use for whole-partition and whole-file predicates: the DELETE
is committed as one metadata operation that drops the matching data files, so
the statement reads no rows, writes no delete files, and later scans no longer
read those files.
- The engine only tries the fast path when the statement predicate can be
extracted exactly (single-table DELETE, leaf is the target table scan, every
conjunct representable). Anything else keeps the current row-level path.
`PARTITION(...)` scoped deletes, UPDATE and MERGE are unchanged.
- The Iceberg connector lowers the predicate with the strictly
all-or-nothing rewrite matrix and only commits when every candidate file is
matched in full (whole-partition shortcut, strict partition projection or file
metrics); otherwise it falls back to the row-level delete, so a partially
matched file is never dropped.
- Like the engines above, the committed snapshot applies the predicate to
the current table state, and the reported row count is iceberg's
`deleted-records` minus the rows the removed delete files had already hidden.
### Release note
None
### Check List (For Author)
- Test
- [ ] Regression test
- [x] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason
- Behavior changed:
- [ ] No.
- [x] Yes.
- Does this need documentation?
- [x] No.
- [ ] Yes.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
