github-actions[bot] commented on code in PR #69422:
URL: https://github.com/apache/doris/pull/69422#discussion_r4639053760
##########
be/src/runtime/exec_env.cpp:
##########
@@ -205,20 +208,38 @@ std::map<TNetworkAddress, FrontendInfo>
ExecEnv::get_running_frontends() {
return res;
}
+void ExecEnv::wait_for_shutdown_heartbeat() {
+ int32_t elapsed = 0;
+ while (!k_shutdown_heartbeat_sent && elapsed <
config::grace_shutdown_heartbeat_wait_seconds) {
+ sleep(1);
+ ++elapsed;
+ }
+ LOG(INFO) << "shutdown heartbeat prepared: " <<
k_shutdown_heartbeat_sent.load() << ", waited "
+ << elapsed << " seconds";
+ // The response may still be in transit. This buffer also gives followers
time
+ // to replay shutdown state. It does not prove that every FE has applied
it.
+ sleep(std::max(0, config::grace_shutdown_heartbeat_propagation_seconds));
+}
+
void ExecEnv::wait_for_all_tasks_done() {
- // For graceful shutdown, need to wait for all running queries to stop
+ // The cloud load receiver and coordinator must finish. For example,
+ // an HTTP upload or transaction commit can outlive all of its pipeline
fragments.
int32_t wait_seconds_passed = 0;
while (true) {
int num_queries = _fragment_mgr->running_query_num();
- if (num_queries < 1) {
Review Comment:
[P2] Remove failed channels before counting them during shutdown
`LoadChannelMgr::open()` inserts a new channel into `_load_channels` at
lines 116-120 before `channel->open(params)` returns. If that open fails before
acceptance—for example cloud mode rejects `txn_expiration <= 0` in
`LoadChannel::open()`—the error returns here without erasing or cancelling the
`_opened == false` channel; only the background timeout cleaner can remove it.
The new `num_active_channels()` census therefore keeps
`wait_for_all_tasks_done()` alive for up to `grace_shutdown_wait_seconds` even
though this request never became an active receiver. Remove/cancel the exact
failed channel on open failure (or count only successfully opened channels) and
cover this path with a shutdown test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]