chengcongchina opened a new pull request, #4302: URL: https://github.com/apache/flink-cdc/pull/4302
This closes [FLINK-39200](https://issues.apache.org/jira/browse/FLINK-39200). ### What is the purpose of the change Currently, the MySQL row event deserializers in Flink CDC always fully deserialize WRITE/UPDATE/DELETE binlog events for all tables, regardless of whether the tables are actually subscribed by the connector. This leads to unnecessary CPU and memory overhead, especially when only a small portion of tables are monitored. This PR implements early skipping of deserialization for unsubscribed tables in binary log client to improve performance. This optimization is applied by default. ### Brief change log - Introduce `TableIdFilter` to support filtering tables during event deserialization. - Add optimized event data deserializers (`WriteRowsEventDataDeserializer`, `UpdateRowsEventDataDeserializer`, `DeleteRowsEventDataDeserializer`) that support skipping deserialization for excluded tables. - Update `MySqlStreamingChangeEventSource` to use the `TableIdFilter` based on the connector's table subscription configuration. ### Verifying this change This change is verified by existing tests: - `MySqlPipelineITCase` - `BinlogSplitReaderTest` ### Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): no - The public API, i.e., is any changed class annotated with @Public(Evolving): no - The serializers: no - The runtime per-record code paths (performance sensitive): yes - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no - The S3 file system connector: no ### Documentation - Does this pull request introduce a new feature? no - If yes, how is the feature documented? not applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
