hubgeter opened a new pull request, #69327:
URL: https://github.com/apache/doris/pull/69327
### What problem does this PR solve?
Issue Number: N/A
Related PR: N/A
Problem Summary:
Iceberg required fields could be written as NULL for ORC and Parquet, and
failed writes could leave data files behind. This PR adds a shared BE Block
validator for required nested fields and MAP keys, runs validation before
writers/files are created, and cleans up failed files without publishing a
snapshot. FE rollback now also removes files already reported by completed BE
fragments.
### Release note
Fix Iceberg ORC/Parquet writes to reject required NULL values and clean up
failed files without publishing a snapshot.
### Check List (For Author)
- Test
- [ ] Regression test
- [x] Unit Test
- BE validator, TableWriter, and cleanup tests pass.
- FE rollback tests pass.
- [x] Manual test
- Verified ORC/Parquet failures leave 0 snapshots, 0 physical
objects, and 0 rows; optional parent NULL writes successfully.
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason
- Behavior changed:
- [ ] No.
- [x] Yes.
- Does this need documentation?
- [x] No.
- [ ] Yes.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
