nryanov opened a new pull request, #7624:
URL: https://github.com/apache/kyuubi/pull/7624
### Why are the changes needed?
Add an experimental Arrow Flight SQL frontend so clients can run SQL and
metadata operations through Kyuubi while reusing the existing session/engine
lifecycle (Spark, Trino, Hive, JDBC, etc.). The producer is a transport adapter
only: it calls BackendService and does not invoke engine APIs directly.
Main capabilities:
- Enable via `kyuubi.frontend.protocols=FLIGHT_SQL` and configure with
`kyuubi.frontend.flight.sql.*` keys (default bind port `10299`).
- Statement queries, catalogs/schemas/tables/table-types/type info, Arrow
IPC streaming with bounded page fetch, cancellation/cleanup.
- Auth: NONE, Basic/LDAP, Kerberos SPNEGO bootstrap with short-lived bearer
tokens; TLS via PEM (optional keystore→PEM materialization).
- HA: dedicated discovery namespace (`kyuubi.ha.flight.sql.namespace`),
advertised Flight endpoint, node-affine tickets (no transparent mid-query
failover).
- Flight-specific metrics.
Out of scope / intentional limits:
- Not end-to-end zero-copy past the existing BackendService Thrift/TRowSet
Arrow boundary.
- Explicit unsupported Flight SQL paths (ingest, Substrait, transactions,
full prepared-statement binding) return unimplemented status.
### How was this patch tested?
- Unit/integration tests under `kyuubi-server`:
- `KyuubiFlightSqlFrontendServiceSuite` (lifecycle / protocol enablement)
- `KyuubiFlightSqlQuerySuite` (execute + Arrow stream via Java
`FlightSqlClient`; run with Slow tag enabled)
- `KyuubiFlightAuthHandlerSuite`
- `KyuubiFlightArrowUtilsSuite`
- Shared harness: `WithFlightSqlServer`
- Config/HA coverage: `KyuubiConfSuite`, `HighAvailabilityConfSuite`
### Was this patch authored or co-authored using generative AI tooling?
Yes. Assisted-by: Cursor Grok 4.5
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines:
https://kyuubi.readthedocs.io/en/master/contributing/code/index.html
2. If the PR is related to an issue in
https://github.com/apache/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title,
e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g.,
'[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### Why are the changes needed?
<!--
Please clarify why the changes are needed. For instance,
1. If you propose a new API, clarify the use case for a new API.
2. If you fix a bug, you can clarify why it is a bug, and what versions
are affected.
-->
### How was this patch tested?
<!--
If tests were added, say they were added here. Please make sure to add some
test cases that check the changes thoroughly including negative and positive
cases if possible.
If it was tested in a way different from regular unit tests, please clarify
how you tested step by step, ideally copy and paste-able, so that other
reviewers can test and check, and descendants can verify in the future.
If tests were not added, please describe why they were not added and/or why
it was difficult to add.
-->
### Was this patch authored or co-authored using generative AI tooling?
<!--
If generative AI tooling has assisted in authoring or reviewing this patch,
please include
an 'Assisted-by: ' trailer that identifies the agent and model.
For example: 'Assisted-by: Claude:claude-opus-4-7' or 'Assisted-by: Claude
Opus 4.7'.
If no, write 'No'.
Please refer to the [ASF Generative Tooling
Guidance](https://www.apache.org/legal/generative-tooling.html) for details.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
