[
https://issues.apache.org/jira/browse/FLINK-35876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Youle updated FLINK-35876:
--------------------------
Description:
I use mysql cdc to read data from mysql to write to doris, mysql cdc may
change from sanpshot mode to binlog mode failed when I set up 2 flink job(20
parallelism of source) and the cpu load of mysql server is 100%. It can't
change even the cpu load drop and just hang up, won't consume latest binlog
data and there is no exception information in logs. But when I only set up one
flink job(20 parallelism of source) and the cpu load of mysql server is less
than 100%, that mistake never happens.
mysql table: about 250 million data
flink version: 1.18.1
mysql cdc connector version: 3.1.0
doris connector version :1.18-1.6.2
Is there anything to do to avoid this mistake if we can't promise the cpu load
always in a healthy state?
!image-2024-07-23-15-47-10-376.png!
was:
I use mysql cdc to read data from mysql to write to doris, mysql cdc may
change from sanpshot mode to binlog mode failed when I set up 2 flink job(20
parallism of source) and the cpu load of mysql server is 100%. It can't change
even the cpu load drop and just hang up, won't consume latest binlog data and
there is no exception information in logs. But when I only set up one flink
job(20 parallism of source) and the cpu load of mysql server is less than
100%, that mistake never happens.
mysql table: about 250 million data
flink version: 1.18.1
mysql cdc connector version: 3.1.0
doris connector version :1.18-1.6.2
Is there anything to do to avoid this mistake if we can't promise the cpu load
always in a healthy state?
!image-2024-07-23-15-47-10-376.png!
> mysql cdc may change from snapshot mode to binlog mode failed when cpu load
> of mysql server is 100%
> ---------------------------------------------------------------------------------------------------
>
> Key: FLINK-35876
> URL: https://issues.apache.org/jira/browse/FLINK-35876
> Project: Flink
> Issue Type: Bug
> Components: Flink CDC
> Affects Versions: cdc-3.1.0
> Reporter: Youle
> Priority: Major
> Attachments: image-2024-07-23-15-47-10-376.png
>
>
> I use mysql cdc to read data from mysql to write to doris, mysql cdc may
> change from sanpshot mode to binlog mode failed when I set up 2 flink job(20
> parallelism of source) and the cpu load of mysql server is 100%. It can't
> change even the cpu load drop and just hang up, won't consume latest binlog
> data and there is no exception information in logs. But when I only set up
> one flink job(20 parallelism of source) and the cpu load of mysql server is
> less than 100%, that mistake never happens.
> mysql table: about 250 million data
> flink version: 1.18.1
> mysql cdc connector version: 3.1.0
> doris connector version :1.18-1.6.2
>
> Is there anything to do to avoid this mistake if we can't promise the cpu
> load always in a healthy state?
> !image-2024-07-23-15-47-10-376.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
