fdolce opened a new pull request, #29036:
URL: https://github.com/apache/flink/pull/29036
## What is the purpose of the change
Adds `pyflink.dataframe.sql()` (FLINK-40197), which executes a SQL SELECT
query and returns the result as a `DataFrame`, so SQL and DataFrame operations
can be mixed freely. Referenced DataFrames are registered as temporary views
only for the duration of the call and dropped afterwards. By default the
caller's variables are scanned and DataFrames are registered under their Python
variable names (auto-bind); explicit keyword bindings choose the SQL names
directly.
UDF/UDTF bindings are intentionally out of scope and will be added in a
separate PR once the DataFrame API gains UDF support in general.
## Brief change log
- Added `pyflink/dataframe/sql.py` implementing `sql(query, *,
auto_bind=True, **bindings)`, exported as `pyflink.dataframe.sql`
- Auto-bind is best-effort: it warns and skips names that are not valid SQL
identifiers, collide with existing tables/views, or belong to a different
TableEnvironment; it never shadows permanent catalog objects
- Explicit bindings are strict (`ValueError` on temporary-view collisions,
`TypeError` for non-DataFrame values), take precedence over auto-bind, and are
the intentional way to shadow a permanent catalog table
- Only SELECT-style queries are accepted; other statements raise
`ValueError` pointing to `TableEnvironment.execute_sql()`
- Registered views are always dropped after the call, including on failure
and on partially-completed registration
- Added the Sphinx API reference page
(`docs/reference/pyflink.dataframe/sql.rst`)
## Verifying this change
This change added tests and can be verified as follows:
- Added `pyflink/dataframe/tests/test_sql.py` (27 tests) covering:
SELECT-only enforcement and error translation, auto-bind over locals/globals
(including locals-over-globals precedence), joins across auto-bound DataFrames,
explicit bindings with and without auto-bind, precedence on name collisions,
strict collision errors for explicit bindings, warn-and-skip on collisions with
existing temporary views and permanent tables, identifier validation (invalid
names, unicode names, SQL keywords via backtick quoting), cleanup after success
/ query failure / partial registration failure, rejection of DataFrames from a
foreign TableEnvironment, rejection of UDF bindings, and composition of the
result with the DataFrame API
- Run with: `cd flink-python && python -m pytest
pyflink/dataframe/tests/test_sql.py`
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: yes, the new `sql()` function is annotated
`@PublicEvolving`
- The serializers: no
- The runtime per-record code paths (performance sensitive): no (view
registration happens once per call at query compilation time)
- Anything that affects deployment or recovery: no
- The S3 file system connector: no
## Documentation
- Does this pull request introduce a new feature? yes
- If yes, how is the feature documented? docs (Sphinx API reference page)
and the `sql()` docstring with examples
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes (please specify the tool below)
Generated-by: Claude Code 2.1.250 (Claude Fable 5)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
