shaohui hong created FLINK-36775:
------------------------------------
Summary: fetchSize set in OracleIncrementalSource is not working
Key: FLINK-36775
URL: https://issues.apache.org/jira/browse/FLINK-36775
Project: Flink
Issue Type: Bug
Components: Flink CDC
Affects Versions: 3.0.0
Reporter: shaohui hong
Attachments: OracleScanFetchTask.java
FetchSize is an import parameter which affects the speed of selecting data from
database using jdbc, because it tells jdbc the max size of records to get for
each query. A bigger fetchSize leads to a smaller query times, whichi can
siganifcantly reduce the time of getting all snapshot data.
OracleIncrementalSource.java uses OracleScnFetchTask.java to do all the
snapshot work by jdbc, and has a config named fetchSize. This config is used in
jdbc as described as above, but it does not actually take affect. At 332nd line
of OracleScnFetchTask.java, the function named readTableSplitDataStatement
formats the selectStatement which is used to execute query, and then set
fetchSize of this selectStatement. The value of fetchSize comes from
OracleConnectorConfig.java the parent class of which is
CommonConnectorConfig.java where a field named query.fetch.size exists. This is
the config place of fetchSize that is actually used.
In order to make fetchSize parameter set in OracleIncrementalSource take
affect, it just needs to add one line of code in OracleSourceConfigFactory.java:
props.setProperty(CommonConnectorConfig.QUERY_FETCH_SIZE.name(),
String.valueOf(fetchSize))
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
