0AyanamiRei opened a new pull request, #66634:
URL: https://github.com/apache/doris/pull/66634
### What problem does this PR solve?
Issue Number: N/A
Related PR: #64878
Problem Summary:
This is Part 1 of the Routine Load target-table ALTER work. It contains only
the persistence and recovery prerequisite; it does not add target-table ALTER
syntax, fields, or behavior.
RoutineLoadJob currently persists origStmt from CREATE and reparses it
during image recovery. Because origStmt never changes, load clauses and
job/data-source properties changed by ALTER are not represented by that
recovery source.
This PR:
- persists a LoadDefinition snapshot containing the current RoutineLoadDesc,
job properties, and data-source properties;
- reconstructs and validates CREATE semantics from LoadDefinition for new
images, while retaining origStmt parsing for images written by old FEs;
- carries only changed RoutineLoadDesc clauses in ALTER edit logs, then
reconstructs the complete definition from runtime state during apply/replay;
- validates failure-prone Kafka and Kinesis ALTER changes before mutating
runtime fields, preserving the invariant that LoadDefinition matches the
effective runtime CREATE semantics;
- adds image, legacy compatibility, edit-log compatibility, and failed-ALTER
atomicity unit tests.
### Release note
Routine Load jobs now recover the current effective load definition after
ALTER.
### Check List (For Author)
- Test
- [ ] Regression test
- [x] Unit Test
- [ ] Manual test
- [ ] No need to test or manual test
- Behavior changed:
- [ ] No.
- [x] Yes. New images restore the current effective Routine Load
definition; legacy images continue to recover from origStmt.
- Does this need documentation?
- [x] No.
- [ ] Yes.
Unit test command:
./run-fe-ut.sh --run
org.apache.doris.load.routineload.KafkaRoutineLoadJobTest,org.apache.doris.load.routineload.KinesisRoutineLoadJobTest,org.apache.doris.persist.AlterRoutineLoadOperationLogTest
Result: 27 tests passed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
