yux created FLINK-37741:
---------------------------
Summary: Severe performance regression of transform operator node
Key: FLINK-37741
URL: https://issues.apache.org/jira/browse/FLINK-37741
Project: Flink
Issue Type: Bug
Components: Flink CDC
Reporter: yux
Attachments: image-2025-04-28-15-31-18-308.png
FLINK-37104 refactored transform module with various code cleanup and logic
simplification. However, a List<DataType> object is repeatedly created from
pre/post transformed schemas, which significantly affects the RPS throughput.
!image-2025-04-28-15-31-18-308.png!
Based on my local testing, even a simple transform rule would cause a ~40%
performance regression.
|{color:#000000}*Version*{color}|*{color:#000000}Transform
Rule{color}*|*{color:#000000}Time Elapsed{color}*|{color:#172b4d}*RPS*{color}|
|{color:#000000}{*}3.4-SNAPSHOT{*}{*}{*}{color}|{color:#000000}Complex{color}|{color:#000000}4m
21s 86ms{color}|{color:#a30003}{*}38,314.18{*}{*}{*}{color}|
|{color:#000000}{*}3.4-SNAPSHOT{*}{*}{*}{color}|{color:#000000}Simple{color}|{color:#000000}2m
45s 488ms{color}|{color:#a30003}{*}60,606.06{*}{*}{*}{color}|
|{color:#000000}{*}3.4-SNAPSHOT{*}{*}{*}{color}|{color:#000000}Empty{color}|{color:#000000}1m
31s 869ms{color}|{color:#000000}109,890.11{color}|
|{color:#000000}{*}3.4-SNAPSHOT{*}{*}{*}{color}|{color:#000000}Off{color}|{color:#000000}1m
45s 587ms{color}|{color:#000000}95,238.10{color}|
|{color:#000000}{*}3.3{*}{*}{*}{color}|{color:#000000}Complex{color}|{color:#000000}1m
46s 54ms{color}|{color:#000000}94,339.62{color}|
|{color:#000000}{*}3.3{*}{*}{*}{color}|{color:#000000}Simple{color}|{color:#000000}1m
34s 971ms{color}|{color:#000000}106,382.98{color}|
|{color:#000000}{*}3.3{*}{*}{*}{color}|{color:#000000}Empty{color}|{color:#000000}1m
21s 546ms{color}|{color:#000000}123,456.79{color}|
|{color:#000000}{*}3.3{*}{*}{*}{color}|{color:#000000}Off{color}|{color:#000000}1m
34s 5ms{color}|{color:#000000}106,382.98{color}|
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
