Cribbee opened a new pull request, #29219: URL: https://github.com/apache/flink/pull/29219
## What is the purpose of the change This pull request resolves [FLINK-6417](https://issues.apache.org/jira/browse/FLINK-6417) for `FileSource` by allowing its default file enumerators to accept glob patterns in input paths. This makes it possible to select partition directories and files without constructing every concrete path in user code. The implementation is file-system independent. It traverses from the longest non-glob path prefix and reuses Flink's existing `GlobFilePathFilter`, rather than changing a specific file system implementation or the semantics of `FileSystem#listStatus`. ## Brief change log - Add glob path handling to `NonSplittingRecursiveEnumerator`, which is also inherited by `BlockSplittingRecursiveEnumerator`. - Support `*`, `?`, and character classes in any path segment. - Return no splits when a glob has no matches, including when its non-glob search root does not exist. - Document glob input paths in the FileSource JavaDocs and DataStream filesystem connector documentation. ## Verifying this change This change added tests covering: - wildcard matches across directory and file path segments; - recursive enumeration when a glob matches a directory; - `?` and character class patterns; - glob paths with no matches and with a missing search root; - both non-splitting and block-splitting recursive enumerators. The complete `flink-connector-files` test suite passes locally: 211 tests run, 0 failures, 0 errors, 6 skipped. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): no - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: yes (behavior and JavaDocs only; no signature changes) - The serializers: no - The runtime per-record code paths (performance sensitive): no (file enumeration only) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no - The S3 file system connector: yes (generic FileSource enumeration behavior) ## Documentation - Does this pull request introduce a new feature? yes - If yes, how is the feature documented? docs and JavaDocs --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes (please specify the tool below) Generated-by: Codex (GPT-5) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
