OIiveirra opened a new pull request, #68964:
URL: https://github.com/apache/doris/pull/68964
### What problem does this PR solve?
Issue Number: N/A
Related PR: #68726
Problem Summary:
Stream Load reads the base schema version without holding the table read
lock. Concurrent schema change removes the old base-index metadata before
switching the base index ID under the table write lock, so this read can
dereference missing metadata and fail the load. A deterministic FE unit test
reproduces the null dereference on the pre-optimization baseline.
Protect the result metadata reads in both the regular Stream Load and HTTP
`load_sql` paths with the request-bounded table read lock and release it in
`finally`. For a normal HTTP load, planning has already begun a transaction. If
the new lock times out, abort that transaction because the BE receives a failed
plan without its transaction ID. Group commit has no such FE transaction and
does not enter this cleanup. Preserve the timeout and record an abort failure
if cleanup fails.
This fix is separated from the scheduling and file TVF COUNT optimization in
#68726.
### Release note
Prevent Stream Load schema-version reads from racing with schema change, and
clean up HTTP load transactions when the metadata lock times out.
### Check List (For Author)
- Test
- [ ] Regression test
- [x] Unit Test
- [ ] Manual test
- [ ] No need to test or manual test. Explain why:
Validation performed:
- Standard focused FE UT reactor passed: `StreamLoadHandlerTest` (10) and
`HttpStreamLoadTablePropertiesTest` (5), 15 tests total with zero
failures/errors. Maven Checkstyle and `git diff --check` passed.
- Coverage includes concurrent base-index replacement with real table
locks, HTTP metadata read-lock ownership and release, forced timeout while a
real write lock is held, transaction abort, group-commit exclusion, and
abort-error preservation.
- The baseline differential test reproduced the original null dereference;
this is not a full Cloud CI reproduction.
- Behavior changed:
- [ ] No.
- [x] Yes. Metadata reads wait for concurrent schema updates within the
RPC timeout; normal HTTP transactions are aborted if the metadata read lock
times out.
- Does this need documentation?
- [x] No.
- [ ] Yes.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
