[ https://issues.apache.org/jira/browse/FLINK-36813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ruan Hang updated FLINK-36813: ------------------------------ Fix Version/s: cdc-3.4.0 (was: cdc-3.3.0) > MySQLCDC supports synchronization of specified fields > ----------------------------------------------------- > > Key: FLINK-36813 > URL: https://issues.apache.org/jira/browse/FLINK-36813 > Project: Flink > Issue Type: Improvement > Components: Flink CDC > Affects Versions: cdc-3.2.1 > Reporter: Di Wu > Priority: Major > Labels: CDC, pull-request-available > Fix For: cdc-3.4.0 > > > *Background* > In some scenarios, MySQL synchronization only expects to synchronize > specified fields instead of all fields in the table. > 1. The user only has the permission for some fields in MySQL > 2. The user has too many fields in a single table and only wants to > synchronize some fields, for example, here > [https://github.com/apache/flink-cdc/discussions/3058] > *Current situation* > For the incremental stage, you only need to configure the column.include.list > property of debezium to support the synchronization of some fields in the > incremental stage, refer to: > [https://debezium.io/documentation/reference/1.9/connectors/mysql.html#mysql-property-column-include-list] > For the full snapshot stage, * is currently used in > {_}MySqlSnapshotSplitReadTask{_}, refer to > {code:java} > if (isScanningData) { > return buildSelectWithRowLimits( > tableId, limitSize, "*", Optional.ofNullable(condition), Optional.empty()); > {code} > > *Solution* > We can refer to debezium > [RelationalSnapshotChangeEventSource](https://github.com/debezium/debezium/blob/main/debezium-core/src/main/java/io/debezium/relational/RelationalSnapshotChangeEventSource.java#L752-L776), > The user configures column.include.list, and then captures the specific > columns in MySqlSnapshotSplitReadTask, and splices them when constructing > Scan SQL. -- This message was sent by Atlassian Jira (v8.20.10#820010)