[
https://issues.apache.org/jira/browse/FLINK-37741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Leonard Xu resolved FLINK-37741.
--------------------------------
Resolution: Fixed
> Severe performance regression of transform operator node
> --------------------------------------------------------
>
> Key: FLINK-37741
> URL: https://issues.apache.org/jira/browse/FLINK-37741
> Project: Flink
> Issue Type: Bug
> Components: Flink CDC
> Affects Versions: cdc-3.4.0
> Reporter: yux
> Assignee: yux
> Priority: Blocker
> Labels: pull-request-available
> Fix For: cdc-3.4.0, cdc-3.5.0
>
> Attachments: image-2025-04-28-15-31-18-308.png
>
>
> FLINK-37104 refactored transform module with various code cleanup and logic
> simplification. However, a List<DataType> object is repeatedly created from
> pre/post transformed schemas, which significantly affects the RPS throughput.
> !image-2025-04-28-15-31-18-308.png!
> Based on my local testing, even a simple transform rule would cause a ~40%
> performance regression.
> |{color:#000000}*Version*{color}|*{color:#000000}Transform
> Rule{color}*|*{color:#000000}Time Elapsed{color}*|{color:#172b4d}*RPS*{color}|
> |{color:#000000}*3.4-SNAPSHOT*{color}|{color:#000000}Complex{color}|{color:#000000}4m
> 21s 86ms{color}|{color:#a30003}*38,314.18*{color}|
> |{color:#000000}*3.4-SNAPSHOT*{color}|{color:#000000}Simple{color}|{color:#000000}2m
> 45s 488ms{color}|{color:#a30003}*60,606.06*{color}|
> |{color:#000000}*3.4-SNAPSHOT*{color}|{color:#000000}Empty{color}|{color:#000000}1m
> 31s 869ms{color}|{color:#000000}109,890.11{color}|
> |{color:#000000}*3.4-SNAPSHOT*{color}|{color:#000000}Off{color}|{color:#000000}1m
> 45s 587ms{color}|{color:#000000}95,238.10{color}|
> |{color:#000000}*3.3*{color}|{color:#000000}Complex{color}|{color:#000000}1m
> 46s 54ms{color}|{color:#000000}94,339.62{color}|
> |{color:#000000}*3.3*{color}|{color:#000000}Simple{color}|{color:#000000}1m
> 34s 971ms{color}|{color:#000000}106,382.98{color}|
> |{color:#000000}*3.3*{color}|{color:#000000}Empty{color}|{color:#000000}1m
> 21s 546ms{color}|{color:#000000}123,456.79{color}|
> |{color:#000000}*3.3*{color}|{color:#000000}Off{color}|{color:#000000}1m 34s
> 5ms{color}|{color:#000000}106,382.98{color}|
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
