raghav-reglobe opened a new issue, #67503: URL: https://github.com/apache/doris/issues/67503
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Version master (current head 219c6193f) and the 3.x/4.x lineage since #64799. ### What's Wrong? Since #64799 the FE keeps a BE-served Flight SQL query's coordinator alive after `GetFlightInfo` so the BE can still fetch external-table splits during `DoGet`. The coordinator is finalized only when the session runs its next query or is closed (`CloseSession`, idle kill, token expiry). While it lives it holds the query's workload-group queue slot and its `information_schema.active_queries` registration. A Flight session idles under the same `wait_timeout` as a MySQL connection (28800s by default). Most Flight clients open a session per query and never send `CloseSession` (the ADBC and JDBC drivers do not send it on connection close, as the comment in `DorisFlightSqlProducer.closeSession` notes). So each abandoned session pins one queue slot for up to eight hours. With `max_concurrency=8` on the group, eight finished queries were enough: they stayed `RUNNING` in `active_queries` for over an hour and every later query in the group failed with `query queue timeout, timeout: 60000 ms`. `SELECT 1`-class queries kept working because they never enter the queue, which makes the lockout look intermittent. ### What You Expected? An abandoned Flight session should not hold a queue slot for hours. Either the slot is released once the result has been fetched, or Flight sessions get an idle bound much shorter than `wait_timeout` — without ever cutting a query short of its own `query_timeout`. ### How to Reproduce? 1. `CREATE WORKLOAD GROUP wg_t PROPERTIES ('max_concurrency'='2','max_queue_size'='5','queue_timeout'='10000')`, grant it to a user and set it as their default workload group. 2. From a Flight SQL client, twice: handshake → `GetFlightInfo` on a scanning query (any table) → `DoGet` the result → do NOT close the session (or exit the process without `CloseSession`). 3. On the FE: `SELECT * FROM information_schema.active_queries` — both finished queries are still `RUNNING`; `SHOW PROCESSLIST` shows two `Sleep` sessions with `Host = 0.0.0.0:0`. 4. Run a third scanning query in that group from any protocol: it queues and fails after 10s with `query queue timeout`. 5. `KILL CONNECTION <id>` on either Flight session: the slot is released immediately. ### Anything Else? The MySQL protocol path releases the slot when the result is sent (`finalizeCommand`), so only Flight is affected. On the Doris side the natural fix is a Flight-specific idle bound — a new FE config applied by the existing connection timeout checker, floored at the query's exec timeout so a long `DoGet` drain is never killed early. Clients should still send `CloseSession`. ### Are you willing to submit PR? - [x] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
