fightBoxing opened a new pull request, #4510:
URL: https://github.com/apache/flink-cdc/pull/4510
## Motivation
The Oracle pipeline connector currently only supports `initial` / `snapshot`
/ `latest-offset` startup modes, while the MySQL pipeline connector already
supports `specific-offset`. For Oracle users who want to skip the full snapshot
phase and start incremental (LogMiner redo log) consumption from a known
position, there is no way to specify a starting SCN.
Note: the underlying Debezium property `log.mining.start.position.scn`
cannot be used as a workaround, because the shaded Debezium dependency in the
connector jar does not include that configuration, and passing it through the
`debezium.*` prefix is silently ignored.
## Modification
- `OracleDataSourceOptions`: add new option
`scan.startup.specific-offset.scn` (Long, no default value)
- `OracleDataSourceFactory`: add `specific-offset` branch in
`getStartupOptions()`, which builds `StartupOptions.specificOffset(Map{scn,
commit_scn})`. The base framework (`StreamSplitAssigner#createStreamSplit`)
already natively consumes SPECIFIC_OFFSETS offsets, so no framework change is
needed.
## Usage
```yaml
source:
type: oracle
scan.startup.mode: specific-offset
scan.startup.specific-offset.scn: 2053676
```
## Verification
Verified end-to-end on a real cluster (Flink 1.20.1 YARN session + Oracle
12c LogMiner + Iceberg sink):
- JobManager log confirms the stream split starts exactly at the specified
SCN: `Assign split StreamSplit{offset={commit_scn=0, scn=2053676}}`
- Only changes committed after the specified SCN are captured; all
pre-existing snapshot rows are correctly skipped
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
