MartijnVisser opened a new pull request, #235:
URL: https://github.com/apache/flink-connector-jdbc/pull/235
## What is the purpose of the change
`JdbcSource` handed the same `JdbcConnectionProvider` to every split reader,
while
flink-connector-base builds one per split fetcher and closes the reader of a
finished split while
the next split's reader is already reading. A provider holds one connection
and is
`@NotThreadSafe`, so the finished reader closed the running reader's result
set. FLINK-39206 has
the mechanism.
## Brief change log
- `JdbcSource` copies the connection provider per split reader
- `JdbcSourceSplitReader.close()` closes the provider it now exclusively
owns
- `JdbcConnectionProvider` documents the per-split-reader copy
## Verifying this change
This change added tests and can be verified as follows:
- Added `JdbcSourceTest#testSplitReadersDoNotShareAConnection`, red with
the copy removed
- Stress on `DerbyDynamicTableSourceITCase#testLimit`: base 2 failures in
200 runs, head 0 in 299
- `mvn verify` on `flink-connector-jdbc-core`, plus the MySQL and Db2
source ITCases
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: yes
- The serializers: no
- The runtime per-record code paths (performance sensitive): no
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
- The S3 file system connector: no
`JdbcSource` and `JdbcConnectionProvider` are `@PublicEvolving`; no
signature changed and the new
`splitReaderSupplier` is package private and `@VisibleForTesting`. A
user-supplied provider is now
copied per split reader, so one that wraps a pool gets a copy per reader;
the interface Javadoc
says so. Peak connections per subtask go from one to two during a split
hand-over, though the
total over a scan is unchanged. Note japicmp gives no signal in this repo:
`japicmp.referenceVersion`
is `3.0.0-1.16`, which `flink-connector-jdbc-core` was never published at.
## Documentation
- Does this pull request introduce a new feature? no
- If yes, how is the feature documented? not applicable
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes (please specify the tool below)
Generated-by: Claude Code (Opus 5)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
