[ https://issues.apache.org/jira/browse/FLINK-36913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ASF GitHub Bot updated FLINK-36913: ----------------------------------- Labels: pull-request-available (was: ) > Add an option in Kafka Sink to manual map table to topic name. > -------------------------------------------------------------- > > Key: FLINK-36913 > URL: https://issues.apache.org/jira/browse/FLINK-36913 > Project: Flink > Issue Type: New Feature > Components: Flink CDC > Affects Versions: cdc-3.3.0 > Reporter: Yanquan Lv > Assignee: Yanquan Lv > Priority: Major > Labels: pull-request-available > Fix For: cdc-3.3.0 > > > For many users who used Kafka as DataSink in YAML job, them usually want to > send changelog to one topic, in current framework, we can use `route` module > to do that, like what the following YAML file does: > {noformat} > source: > type: mysql > hostname: xxxxxx > port: 3306 > username: flink > password: Flinkxxx > tables: flink_source.news_[0-9] > sink: > type: kafka > properties.bootstrap.servers: xxx:9092 > value.format: canal-json > route: > - source-table: flink_source.news_[0-9] > sink-table: my_source > {noformat} > However, the out put of Kafka in canal-json format doesn't contain the > original database/table information, instead, it only contains the > database/table information after routing. Although this is in line with the > functionality of route, it does not meet the needs of users. > Therefore, I suggest adding a parameter in the sink to let Kafka determine > how to handle the mapping from table to topic name, so we can create one > topic for many source tables, and keep all database/table information of > source tables when output records. -- This message was sent by Atlassian Jira (v8.20.10#820010)