[ 
https://issues.apache.org/jira/browse/FLINK-36913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated FLINK-36913:
-----------------------------------
    Labels: pull-request-available  (was: )

> Add an option in Kafka Sink to manual map table to topic name.
> --------------------------------------------------------------
>
>                 Key: FLINK-36913
>                 URL: https://issues.apache.org/jira/browse/FLINK-36913
>             Project: Flink
>          Issue Type: New Feature
>          Components: Flink CDC
>    Affects Versions: cdc-3.3.0
>            Reporter: Yanquan Lv
>            Assignee: Yanquan Lv
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: cdc-3.3.0
>
>
> For many users who used Kafka as DataSink in YAML job, them usually want to 
> send changelog to one topic, in current framework, we can use `route` module 
> to do that, like what the following YAML file does:
> {noformat}
> source:
> type: mysql
> hostname: xxxxxx
> port: 3306
> username: flink
> password: Flinkxxx
> tables: flink_source.news_[0-9]
> sink:
> type: kafka
> properties.bootstrap.servers: xxx:9092
> value.format: canal-json
> route:
> - source-table: flink_source.news_[0-9]
> sink-table: my_source
> {noformat}
> However, the out put of Kafka in canal-json format doesn't contain the 
> original database/table information, instead, it only contains the 
> database/table information after routing. Although this is in line with the 
> functionality of route, it does not meet the needs of users.
> Therefore, I suggest adding a parameter in the sink to let Kafka determine 
> how to handle the mapping from table to topic name, so we can create one 
> topic for many source tables, and keep all database/table information of 
> source tables when output records.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Reply via email to