[
https://issues.apache.org/jira/browse/FLINK-37539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17937790#comment-17937790
]
Yanquan Lv commented on FLINK-37539:
------------------------------------
Table commit will cost many times.
2025-03-23 19:14:52,353 INFO
org.apache.flink.cdc.connectors.paimon.sink.v2.PaimonWriter [] -
prepareCommit: 13, elapsed time: 38886 ms
!image-2025-03-24-15-07-01-191.png!
> performance optimization in Paimon Sink to reduce end-to-end checkpoint time
> ----------------------------------------------------------------------------
>
> Key: FLINK-37539
> URL: https://issues.apache.org/jira/browse/FLINK-37539
> Project: Flink
> Issue Type: Improvement
> Components: Flink CDC
> Affects Versions: cdc-3.3.0
> Reporter: Yanquan Lv
> Priority: Blocker
> Attachments: image-2025-03-24-15-07-01-191.png
>
>
> At present, when using Paimon as a sink for jobs that require writing a large
> number of tables, the end-to-end checkpoint takes a long time because we need
> to perform commit operations at the checkpoint.
> Perhaps we can use multithreading to have each table submit separately and
> then process them together.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
