jmbkeyes opened a new issue, #9666:
URL: https://github.com/apache/seatunnel/issues/9666
It's can be done by the following config file if there's no requirement for
specify table columns.
source {
MySQL-CDC {
base-url="jdbc:mysql://10.0.0.12:3306/bocai"
username = "root"
password = "Forzadata1234"
database-names=[
"bocai"
]
table-names=[
"bocai.account",
]
# 初始化模式(全量+增量)
startup.mode = "initial"
# 启用 ddl 变更
#schema-changes.enabled = true
# 并行读取配置
snapshot.split.size = 8096
snapshot.fetch.size = 1024
}
}
sink {
Jdbc {
url =
"jdbc:mysql://10.0.0.181:3306/bocai?useUnicode=true&characterEncoding=UTF-8&rewriteBatchedStatements=true"
driver = "com.mysql.cj.jdbc.Driver"
user = "root"
password = "Forzadata1234"
database = "bocai"
table = "${table_name}"
schema_save_mode = "CREATE_SCHEMA_WHEN_NOT_EXIST"
generate_sink_sql = true
data_save_mode = "APPEND_DATA"
# enable_upsert = false
# support_upsert_by_query_primary_key_exist = true
# 精确一次语义(可选)
#is_exactly_once = true
#xa_data_source_class_name = "com.mysql.cj.jdbc.MysqlXADataSource"
}
}
**BUT If I want to sync several tables(each table needs to sync specific
columns), how to specify several tables' columns.** right now I just add source
for one table, and transform for the table.
source {
MySQL-CDC {
base-url="jdbc:mysql://10.0.0.12:3306/bocai"
username = "root"
password = "Forzadata1234"
database-names=[
"bocai"
]
table-names=[
"bocai.access_log",
]
# 初始化模式(全量+增量)
startup.mode = "initial"
# 启用 ddl 变更
#schema-changes.enabled = true
# 并行读取配置
snapshot.split.size = 8096
snapshot.fetch.size = 1024
plugin_output = "my_access_log";
}
}
transform {
# 可选的数据转换处理
Sql {
query = "select id, createTime, requestMethod,requestTime from
access_log"
result_table_name = "access_log"
plugin_input = "my_access_log"
}
}
If there're 10 tables to sync with specifying table columns. Shoud I need to
add ten more sources and tranforms for each table?
And for MySQL-CDC, each source acts as one mysql slaver or use the same
mysql slave?
Thanks in advance!!!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
