complone opened a new pull request, #42: URL: https://github.com/apache/flink-connector-jdbc/pull/42
related to [FLINK-31820](https://issues.apache.org/jira/browse/FLINK-31820) #### Background & Motivation Hi, community. As I develop full-database synchronization syntax in the company to reduce the increase in the number of connections when multiple data sources are synchronized, I find that traditional databases (Mysql, Postgres, Oracle) need to support the function of sub-database and sub-table. It supports data source sub-database sub-table on top of existing projects #### Goals - Support data source sub-database sub-table - When the sub-database and table functions are realized, the original data fragments allocated according to the partition are guaranteed to be available - Prioritize to ensure that Mysql, Postgres, Oracle support sub-database sub-table #### Change - Every time JdbcRowDataInputFormat switches fragments, it will call open(InputSplit inputSplit) once (corresponding to closing the current data fragmentation method: close()), the value of inputSplit corresponds to the value of x in Serializable[x][y] is incremented, and each parallel The instance will not be executed repeatedly. For example, if there are 1024 split tables and each table has 2 data splits, then the value range of inputSplit.getSplitNumber() is: [0~2047]. Therefore, when the database is divided into tables, it should be ensured that the query logic of each table after the database and table is consistent with the query logic of the original single database and single table. - JdbcMultiTableProvider needs to split the Serializable[][] twice based on the results of the original data fragmentation, according to the sub-database and sub-table, to ensure that JdbcRowDataInputFormat can obtain the split table information #### Design According to the configured url and table_name expressions, the basic encoding steps are as follows: - Query all schemas in the database - Match the schema by regular - Query matches the table under the schema - By regular matching table - Return the corresponding relationship between the database url and the table: List<TableItem> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
