github-actions[bot] commented on code in PR #69683:
URL: https://github.com/apache/doris/pull/69683#discussion_r4387198347
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/split/SplitSource.java:
##########
@@ -147,10 +121,10 @@ public synchronized List<TScanRangeLocations>
getNextBatch(int maxBatchSize) thr
throw new UserException("Timeout. Max wait time(ms): " +
maxWaitTime);
}
try {
- batch = splitAssignment.awaitBatch(backend, remainingNanos);
- if (batch != null) {
+ TScanRangeLocations split =
splitAssignment.awaitSplit(backend, remainingNanos);
+ if (split != null) {
splitAssignment.notifyQueueSpace(backend);
- pendingSplits = batch.iterator();
+ scanRanges.add(split);
Review Comment:
[P2] Recheck terminal state after an awaited split
With `split_initial_batch_size=1` (or a BE request size of 1), an
empty-queue fetch waits in `awaitSplit`. A producer can enqueue a range and
then publish a listing failure before the waiting RPC thread resumes.
`awaitSplit` still polls that range, and adding it here fills the request, so
the loop exits without another `getAssignedSplits`/`isStop` check.
`fetchSplitBatch` consequently returns OK for an already-failed assignment. A
LIMIT can consume that range and stop without another fetch; cleanup only logs
the stop exception. Previously, the awaited batch remained pending until the
loop rechecked terminal state. Apply the same error/cancellation recheck used
by the bulk-drain branch after awaiting/notifying, and cover a one-split wait
racing with producer failure.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
