[
https://issues.apache.org/jira/browse/FLINK-36573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ruan Hang updated FLINK-36573:
------------------------------
Fix Version/s: cdc-3.4.0
(was: cdc-3.3.0)
(was: cdc-3.2.1)
> Add table information of binlog offsets when checkpointing
> ----------------------------------------------------------
>
> Key: FLINK-36573
> URL: https://issues.apache.org/jira/browse/FLINK-36573
> Project: Flink
> Issue Type: Improvement
> Components: Flink CDC
> Affects Versions: cdc-3.2.0
> Reporter: Runkang He
> Priority: Minor
> Labels: pull-request-available
> Fix For: cdc-3.4.0
>
>
> For mysql binlog split, currently when checkpointing, we only print the
> binlog offset in logs, and difficult to know which table the offset belongs
> to, especially when there are multiple mysql cdc sources in one job.
> {code:java}
> 2024-09-27 18:16:10,488 INFO
> org.apache.flink.cdc.connectors.mysql.source.reader.MySqlSourceReader [] -
> Binlog offset on checkpoint 4: {transaction_id=null, ts_sec=0,
> file=mysql-binlog.000666, pos=147223270, kind=SPECIFIC,
> gtids=e6e6e8e3-6351-11ef-8340-52540bd1ce99:81433397-81559559, row=0,
> event=20, server_id=1}
> {code}
> So it's very useful to add table information of binlog offsets, to help user
> clearly see the binlog reading progress of each table. Another benefit is
> that when the job topology modification causes job state incompatibility,
> users can use the binlog file and position information here to restart from
> the specific-offsets mode to recover.
> {code:java}
> 2024-09-27 18:16:10,488 INFO
> org.apache.flink.cdc.connectors.mysql.source.reader.MySqlSourceReader [] -
> Binlog offset for tables [test_db.test] on checkpoint 4:
> {transaction_id=null, ts_sec=0, file=mysql-binlog.000666, pos=147223270,
> kind=SPECIFIC, gtids=e6e6e8e3-6351-11ef-8340-52540bd1ce99:81433397-81559559,
> row=0, event=20, server_id=1}
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
