[
https://issues.apache.org/jira/browse/FLINK-36913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-36913:
-----------------------------------
Labels: pull-request-available (was: )
> Add an option in Kafka Sink to manual map table to topic name.
> --------------------------------------------------------------
>
> Key: FLINK-36913
> URL: https://issues.apache.org/jira/browse/FLINK-36913
> Project: Flink
> Issue Type: New Feature
> Components: Flink CDC
> Affects Versions: cdc-3.3.0
> Reporter: Yanquan Lv
> Assignee: Yanquan Lv
> Priority: Major
> Labels: pull-request-available
> Fix For: cdc-3.3.0
>
>
> For many users who used Kafka as DataSink in YAML job, them usually want to
> send changelog to one topic, in current framework, we can use `route` module
> to do that, like what the following YAML file does:
> {noformat}
> source:
> type: mysql
> hostname: xxxxxx
> port: 3306
> username: flink
> password: Flinkxxx
> tables: flink_source.news_[0-9]
> sink:
> type: kafka
> properties.bootstrap.servers: xxx:9092
> value.format: canal-json
> route:
> - source-table: flink_source.news_[0-9]
> sink-table: my_source
> {noformat}
> However, the out put of Kafka in canal-json format doesn't contain the
> original database/table information, instead, it only contains the
> database/table information after routing. Although this is in line with the
> functionality of route, it does not meet the needs of users.
> Therefore, I suggest adding a parameter in the sink to let Kafka determine
> how to handle the mapping from table to topic name, so we can create one
> topic for many source tables, and keep all database/table information of
> source tables when output records.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
