[
https://issues.apache.org/jira/browse/FLINK-35740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lee SeungMin updated FLINK-35740:
---------------------------------
Description:
PR: [[FLINK-35740][mysql] Allow column as chunk key even if not in Primary Keys
by SML0127 · Pull Request #3448 · apache/flink-cdc
(github.com)|https://github.com/apache/flink-cdc/pull/3448]
Allow column as chunk key even if it is not in the primary keys.
There are cases where the primary key is not a numeric type, such as varchar or
varbinary.
In this case, the `distributed factor` and `chunk range` may be calculated
incorrectly, resulting in one chunk containing too many records.
But there was no conditions for restrict type for primary keys or chunk key
columns, so this may cause out of memory in the task manager.
Actually, in our company, there was a mysql tables that PK is a varbinary
column and the above situation occurred.
was:
PR: [Allow column as chunk key even if not in Primary Keys by SML0127 · Pull
Request #3448 · apache/flink-cdc
(github.com)|https://github.com/apache/flink-cdc/pull/3448]
Allow column as chunk key even if it is not in the primary keys.
There are cases where the primary key is not a numeric type, such as varchar or
varbinary.
In this case, the `distributed factor` and `chunk range` may be calculated
incorrectly, resulting in one chunk containing too many records.
But there was no conditions for restrict type for primary keys or chunk key
columns, so this may cause out of memory in the task manager.
Actually, in our company, there was a mysql tables that PK is a varbinary
column and the above situation occurred.
> [mysql] Allow column as chunk key even if not in Primary Keys
> -------------------------------------------------------------
>
> Key: FLINK-35740
> URL: https://issues.apache.org/jira/browse/FLINK-35740
> Project: Flink
> Issue Type: Improvement
> Components: Flink CDC
> Reporter: Lee SeungMin
> Priority: Major
>
> PR: [[FLINK-35740][mysql] Allow column as chunk key even if not in Primary
> Keys by SML0127 · Pull Request #3448 · apache/flink-cdc
> (github.com)|https://github.com/apache/flink-cdc/pull/3448]
> Allow column as chunk key even if it is not in the primary keys.
>
> There are cases where the primary key is not a numeric type, such as varchar
> or varbinary.
> In this case, the `distributed factor` and `chunk range` may be calculated
> incorrectly, resulting in one chunk containing too many records.
> But there was no conditions for restrict type for primary keys or chunk key
> columns, so this may cause out of memory in the task manager.
> Actually, in our company, there was a mysql tables that PK is a varbinary
> column and the above situation occurred.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
