github-actions[bot] commented on code in PR #70237:
URL: https://github.com/apache/doris/pull/70237#discussion_r4581538592


##########
fs_brokers/cdc_client/src/main/java/org/apache/doris/cdcclient/service/PipelineCoordinator.java:
##########
@@ -853,6 +866,12 @@ private boolean shouldStop(
 
     private synchronized DorisBatchStreamLoad getOrCreateBatchStreamLoad(
             WriteRecordRequest writeRecordRequest) {
+        DorisBatchStreamLoad previous = 
batchStreamLoadMap.get(writeRecordRequest.getJobId());
+        String previousTaskId = previous == null ? null : 
previous.getCurrentTaskId();
+        if (previousTaskId != null) {
+            // The previous task has not finished flushing; isolate its 
in-flight loads.
+            closeJobStreamLoad(writeRecordRequest.getJobId());

Review Comment:
   [P1] Preserve load ordering when replacing an occupied writer
   
   After PAUSE/RESUME, task A can still have a healthy Stream Load of `k=v1` 
executing: `/releaseReader` deliberately leaves the job writer alive. `close()` 
only sets flags and calls `shutdownNow()`; it does not wait for or abort that 
accepted transaction. The new writer lets B replay through `k=v2` and commit 
its offset before A's pending load commits. CDC targets do not require a 
sequence column, and Doris assigns partition versions at commit, so A's later 
commit can overwrite `v2` (or resurrect a deleted row) after the offset has 
advanced. Previously, the single cached loader serialized A's and B's loads. 
Preserve an ordering barrier or effective transaction/source-order fence before 
the replacement starts loading; suppressing A's offset commit does not protect 
the table rows.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to