gavinchou commented on code in PR #69129:
URL: https://github.com/apache/doris/pull/69129#discussion_r4290512144
##########
fe/fe-core/src/main/java/org/apache/doris/job/extensions/insert/streaming/StreamingInsertJob.java:
##########
@@ -1761,7 +1778,9 @@ private void persistOffsetProviderIfNeed(
return;
}
- logUpdateOperation();
+ logUpdateOperation(jdbcOffsetProvider.isTableSchemasDirty()
+ ? offsetProviderPersist :
jdbcOffsetProvider.getPersistInfoWithoutTableSchemas());
Review Comment:
The condition that the restored schema matches the current source schema is
sufficient for this particular recovery problem to be avoided, but it is not
guaranteed by the code.
On the argument that this only matters when DDL happens during a rolling
upgrade, and that a failed task can simply be restarted from scratch:
- The DDL does not have to happen during the upgrade window. It can already
be ahead of the committed CDC offset before the upgrade begins. Recovery by an
old FE after rollback is another relevant path.
- A task error/pause contains the immediate execution failure, but resuming
the same offset does not recreate the lost historical schema baseline.
Repeating the recovery can encounter the same mismatch.
- Starting from scratch is a reinitialization/full-resnapshot decision
rather than transparent recovery from the committed offset. It may be an
acceptable operational workaround, but its recovery time and target-data
reconciliation depend on the workload and configuration. That workaround has
not been validated by this review.
The compatibility defect therefore remains valid even if its deployment
frequency is judged low. For this PR, I am accepting the documented
compatibility/reinitialization trade-off and approving the change without
requiring an additional switch. The finding remains recorded as a known
limitation; the approval is a risk-acceptance decision.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
