raghav-reglobe opened a new pull request, #67504:
URL: https://github.com/apache/doris/pull/67504
### What problem does this PR solve?
Issue Number: close #67503
Related PR: #64799 (kept the coordinator alive across GetFlightInfo → DoGet
so the BE can fetch external-table splits; this PR bounds how long an abandoned
session can hold it)
Problem Summary:
A Flight SQL session idles under the same `wait_timeout` as a MySQL
connection (28800s by default). Since #64799 the coordinator of a BE-served
Flight query is kept alive until the session's next query or its close, and
that coordinator holds the query's workload-group queue slot and
`active_queries` registration. Most Flight clients open a session per query and
never send `CloseSession` (the ADBC/JDBC drivers don't do it on connection
close, as the comment in `DorisFlightSqlProducer.closeSession` already notes),
so each abandoned session pins one slot for up to eight hours.
Observed on a 2-FE cluster with a `max_concurrency=8` workload group: eight
finished Flight queries sat in `information_schema.active_queries` as RUNNING
for over an hour, and every later query in the group failed with `query queue
timeout, timeout: 60000 ms`. Killing the idle sessions (`SHOW PROCESSLIST` rows
with `Host = 0.0.0.0:0`) released the slots immediately. Full write-up in
#67503.
This PR adds a mutable FE config, `arrow_flight_session_idle_timeout_second`
(default 3600), and applies it in the existing connection timeout checker to
`ARROW_FLIGHT_SQL` contexts only:
```
idle bound = min(wait_timeout, max(config, exec timeout))
```
The exec-timeout floor is deliberate: a Flight session is `COM_SLEEP` while
the client drains the result via DoGet, and its idle clock runs from the
query's start, so a bound below `query_timeout` would kill a long result stream
before the query's own timeout could. `0` disables the bound. MySQL-protocol
connections are unchanged; the kill log line now reports the effective idle
timeout.
Verified on a test cluster with the bound set to 8s: an abandoned Flight
session and its `active_queries` row were gone within 14s and the kill was
logged as `connection type: ARROW_FLIGHT_SQL, ... idle timeout: 8`; an idle
MySQL connection survived the same window; `0` left the Flight session alive.
### Release note
Arrow Flight SQL sessions get their own idle timeout
(`arrow_flight_session_idle_timeout_second`, default 1h), so an abandoned
Flight session no longer holds its query's workload-group queue slot until
`wait_timeout`.
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [x] Unit Test (`FlightSqlSessionIdleTimeoutTest`: the bound tightens
but never widens `wait_timeout`, the exec-timeout floor, `0` disables, MySQL
context untouched)
- [x] Manual test (steps above)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [x] Yes. Idle Flight SQL sessions are now killed after 1h by default
instead of after `wait_timeout` (8h); set
`arrow_flight_session_idle_timeout_second=0` for the previous behavior.
- Does this need documentation?
- [ ] No.
- [x] Yes. New FE config `arrow_flight_session_idle_timeout_second` —
doris-website PR to follow once this is reviewed.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should
merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
