github-actions[bot] commented on code in PR #69683:
URL: https://github.com/apache/doris/pull/69683#discussion_r4396124903
##########
fe/fe-core/src/main/java/org/apache/doris/qe/SessionVariable.java:
##########
@@ -2622,22 +2624,27 @@ public Map<String, Action> getForceEagerAggHintMap() {
description = "If the number of partitions exceeds the threshold,
scan ranges will be got through batch "
+ "mode.",
needForward = true)
- public int numPartitionsInBatchMode = 1024;
+ public int numPartitionsInBatchMode = 1;
Review Comment:
[P2] Reconcile early dispatch with the one-second split-fetch deadline
With the new defaults, two selected Hive partitions become two one-partition
producers. If producer A returns its sole range while producer B is still doing
HMS/directory IO, `SplitAssignment.init()` releases the planner on A's sample.
`FileQueryScanNode` creates a `SplitSource` for every BE, so a BE without a
range waits only the default `fetch_splits_max_wait_time_ms=1000`; while B is
still listing, `SplitSource.getNextBatch()` throws and the BE treats the
resulting RPC error as query failure. Even one BE can hit this after consuming
A. Previously this tiny scan either planned synchronously or used one streaming
producer whose default batch of 128 collected both ranges before publishing the
sample. Please reconcile the normal fetch wait with unfinished listing before
enabling these defaults, and cover two partitions with one listing blocked
beyond the fetch deadline after the other publishes its range.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
