924060929 commented on code in PR #69683:
URL: https://github.com/apache/doris/pull/69683#discussion_r4399629437
##########
be/src/exec/scan/split_source_connector.h:
##########
@@ -40,8 +41,10 @@ class SplitSourceConnector {
* Get the next scan range. has_next should be to true to fetch the next
scan range.
* @param has_next whether exists the next scan range
* @param range the obtained next scan range
+ * @param scanner_stop the caller's local stop signal, which can be set
without query cancellation
*/
- virtual Status get_next(bool* has_next, TFileRangeDesc* range) = 0;
+ virtual Status get_next(bool* has_next, TFileRangeDesc* range,
+ const std::atomic_bool* scanner_stop) = 0;
Review Comment:
Fixed in fdce9a41dd7. The WAL test connector now implements the
three-argument SplitSourceConnector::get_next signature, including the
scanner-stop parameter. I searched the BE test tree for other overrides; the
WAL stub was the remaining one. Clang-format 16 and header hygiene pass. Local
BE UT cannot run because the host Xcode license blocks the build, so buildall
CI will verify compilation.
##########
fe/fe-core/src/main/java/org/apache/doris/qe/SessionVariable.java:
##########
@@ -2622,22 +2624,27 @@ public Map<String, Action> getForceEagerAggHintMap() {
description = "If the number of partitions exceeds the threshold,
scan ranges will be got through batch "
+ "mode.",
needForward = true)
- public int numPartitionsInBatchMode = 1024;
+ public int numPartitionsInBatchMode = 1;
Review Comment:
Fixed in 1c7a1ac589b. FileQueryScanNode now passes the effective query
timeout to SplitAssignment.init instead of using a fixed 30-second first-split
deadline. This allows MaxCompute read-session creation to finish within the
configured query/connector timeouts; the wait still terminates on producer
error, completion, interruption, or its caller deadline. The deadline uses
monotonic time. SplitAssignmentTest covers delayed first-split success and
timeout before any split; SplitAssignmentTest (22 tests) and SplitProducerTest
(18 tests) pass, and FE Checkstyle passes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
