lvyanquan commented on code in PR #4418: URL: https://github.com/apache/flink-cdc/pull/4418#discussion_r3570442612
########## flink-cdc-connect/flink-cdc-source-connectors/flink-connector-mysql-cdc/src/main/java/org/apache/flink/cdc/connectors/mysql/source/enumerator/MySqlSourceEnumerator.java: ########## Review Comment: I think there is another unsafe case around stale `BinlogSplitMetaRequestEvent`. This PR handles stale `BinlogSplitMetaAssembledEvent` by using the assignment generation, but `BinlogSplitMetaRequestEvent` is still not tied to any reader attempt. A request sent by a failed reader attempt may arrive after the current reader has already assembled the metadata and the checkpoint completing the release has finished. After the release, `releaseSnapshotMetadata()` clears `assignedSplits`, `splitFinishedOffsets`, and `tableSchemas`, and `MySqlSourceEnumerator#notifyCheckpointComplete()` also sets `binlogSplitMeta = null`. If a stale `BinlogSplitMetaRequestEvent` arrives after that, `sendBinlogMeta()` will try to rebuild `binlogSplitMeta` from `splitAssigner.getFinishedSplitInfos()`. Since the assigner metadata has already been cleared, this can return an empty list and throw `FlinkRuntimeException`. This means a stale metadata request from a failed reader attempt can fail the enumerator/job after the metadata has been released. Could we explicitly handle this case by ignoring/logging metadata requests after the snapshot metadata has been released, instead of trying to rebuild the metadata and failing the job? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
