Milesian111 opened a new pull request, #29033:
URL: https://github.com/apache/flink/pull/29033
## What is the purpose of the change
This pull request implements FLINK-40422 by adding row-slicing
transformations to the PyFlink DataFrame API.
The new APIs reuse the existing Table API operations and remain lazy:
- `DataFrame.limit(n)` delegates to `Table.fetch(n)`.
- `DataFrame.offset(n)` delegates to `Table.offset(n)`.
- `DataFrame.head(n)` delegates to `DataFrame.limit(n)`.
`offset` and `limit` can be composed for pagination. The documentation
clarifies that row order is deterministic only when the underlying table has an
explicit ordering.
## Brief change log
- Added the public `DataFrame.limit`, `DataFrame.offset`, and
`DataFrame.head` APIs.
- Added validation requiring `n` to be a non-negative integer.
- Kept `head` as a delegating wrapper around `limit`.
- Added tests covering:
- delegation to the underlying Table API
- lazy execution and preservation of the original DataFrame
- pagination through `offset(a).limit(b)`
- equivalence between `head` and `limit`
- zero-length slices
- negative values and unsupported argument types
- deterministic batch results with explicit ordering
- Added the new methods to the DataFrame API reference documentation.
## Verifying this change
This change added tests and can be verified as follows:
- Added unit tests covering delegation to the existing Table API, lazy
transformations, preservation of the original DataFrame, `head` delegation,
pagination composition, zero-length slices, and invalid arguments.
- Added deterministic batch integration tests covering `limit`, `offset`,
`offset(a).limit(b)`, equivalence between `head` and `limit`, and `n == 0`.
Local execution of the batch integration tests was blocked by pre-existing
invalid Java planner artifacts in the local build environment. The added batch
tests are included for verification in CI.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: yes
- The serializers: no
- The runtime per-record code paths (performance sensitive): no
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
- The S3 file system connector: no
## Documentation
- Does this pull request introduce a new feature? yes
- If yes, how is the feature documented? docs and Python API docstrings
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes (please specify the tool below)
Generated-by: Codex GPT-5
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
