github-actions[bot] commented on code in PR #69129:
URL: https://github.com/apache/doris/pull/69129#discussion_r4280915515
##########
fe/fe-core/src/main/java/org/apache/doris/job/extensions/insert/streaming/StreamingInsertJob.java:
##########
@@ -1761,7 +1778,9 @@ private void persistOffsetProviderIfNeed(
return;
}
- logUpdateOperation();
+ logUpdateOperation(jdbcOffsetProvider.isTableSchemasDirty()
+ ? offsetProviderPersist :
jdbcOffsetProvider.getPersistInfoWithoutTableSchemas());
Review Comment:
[P1] Gate compact journals until all FE readers support merging
Once a schema-bearing commit is followed by an ordinary commit with no
schema update, this branch writes `opp` without `ts` using the existing journal
operation. An older FE follower still executes
`setOffsetProviderPersist(replayJob.getOffsetProviderPersist())`, so replay
discards its last committed schema. If that FE becomes master during a rolling
upgrade, or the deployment is rolled back, `replayIfNeed` restores a null
schema and the resumed task omits `tableSchemas`. A fresh MySQL reader then
discovers the current JDBC schema; when source DDL is ahead of the committed
binlog position, it can fail decoding the pre-DDL records because the required
schema baseline is gone. Keep full records until every possible reader supports
merging, or add a compatibility gate, and cover old-reader replay followed by
recovery.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
