github-actions[bot] commented on code in PR #63743:
URL: https://github.com/apache/doris/pull/63743#discussion_r3310304614
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/distribute/worker/job/UnassignedScanMetadataJob.java:
##########
@@ -56,6 +65,16 @@ protected Map<DistributedPlanWorker, UninstancedScanSource>
multipleMachinesPara
);
}
+ /**
+ * If no workers could be selected for the metadata scan (e.g. all
backends are
+ * unavailable), create a single empty instance on a random available
worker
Review Comment:
This example is misleading. `fillUpAssignedJobs()` only reaches
`fillUpSingleEmptyInstance()` when the selected scan ranges produced no
assigned jobs; it does not recover from all BEs being unavailable. The fallback
still calls `workerManager.randomAvailableWorker(catalogId)`, which throws a
`NereidsException` when no scan backend is available. Please describe this as
the empty-scan-range/pruned case instead of an all-backends-unavailable
fallback that prevents query failure.
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/distribute/worker/job/UnassignedShuffleJob.java:
##########
@@ -50,6 +50,22 @@ public UnassignedShuffleJob(
super(statementContext, fragment, ImmutableList.of(),
exchangeToChildJob);
}
+ /**
+ * Compute assigned jobs for a shuffle (data redistribution) fragment.
+ * The instance count is determined by the parallelism of the largest child
+ * fragment. When the expected instance count is lower than the child count
+ * (e.g. due to session variable limits or query cache constraints),
workers
+ * are shuffled to spread instances across different backends for load
balancing.
+ * When more instances are needed, worker assignment follows the child
layout.
Review Comment:
This sentence does not match the implementation. When `expectInstanceNum` is
greater than or equal to the largest child fragment size, the `else` branch
still calls `buildInstances(biggestParallelChildFragment.size(), ...)`, so this
method never creates "more instances" than the child layout. It only preserves
the child instance count and worker placement. Please reword this to avoid
implying that higher requested exchange parallelism increases the instance
count here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
