DanielLeens commented on PR #10340: URL: https://github.com/apache/seatunnel/pull/10340#issuecomment-6369022278
I prepared a contributor-applicable conflict/XA/E2E cleanup patch for head `9445ffdbc5220b416fb146b74a839e50bac0c981` merged with upstream `dev` at `14a27ac26a0a71b56722f86cdf8ec2779094ceaf`. The local repository was shallow, so I first fetched the missing history and verified merge-base `81d873301d5d0bad16a3ef1680a3172d0968b8d9`. There is no unrelated-history merge in this proposal. The patch resolves the three content conflicts and one independently evidenced test cleanup defect: - Preserve every current upstream JDBC documentation row and its layout, adding only the Oscar row. Keep both `OSCAR` and newer `YASHANDB` database identifiers. All newer upstream dependency/module registrations remain present; no POM changes are included in this small resolution delta. - Correct the EN/ZH XA class to `com.oscar.xa.Jdbc3XADataSource`. I inspected the actual PR-selected Maven artifact `com.shentongdata:oscarJDBC8:4.1.152` (SHA256 `f53e80ec4bd1c32cd62b2bfe2f7f83b3e7608735f36c3a017f6c31263cd9b564`). Its class declaration implements `javax.sql.XADataSource`; the currently documented `Jdbc3XAConnection` implements `XAConnection`/`XAResource` instead. This verifies the type correction, not an end-to-end XA transaction claim. - Override `JdbcOscarUpsetIT.clearTable(database, schema, table)` so it clears `SYSDBA2` via the existing schema-aware overload. The base three-argument implementation forwards the database (`OSRDB`) as the schema, while this test creates its source/sink in `SYSDBA2`; otherwise old sink rows can remain between engine runs. All assertions are retained. Save the diff below as `pr10340-sync-resolve-conflicts.patch`. It applies to the conflicted working files **after** the pinned merge, not directly to a clean source checkout: ```bash git fetch https://github.com/apache/seatunnel.git dev git -c merge.conflictStyle=merge -c rerere.enabled=false merge --no-commit --no-ff 14a27ac26a0a71b56722f86cdf8ec2779094ceaf # Three expected content conflicts remain; the patch resolves them and the cleanup method. git apply --check pr10340-sync-resolve-conflicts.patch git apply pr10340-sync-resolve-conflicts.patch ``` Stage the four patched files, inspect the complete merge, then commit/push normally from your source branch. If the pinned source or target changes, regenerate the resolution. Do not apply the separate original-head cleanup patch on top of this combined one. Validation completed: scoped JDBC and JDBC E2E part-5 Spotless; exact saved-marker forward patch application; no remaining unmerged entries; target-relative scoped whitespace; byte comparison preserving all non-Oscar documentation; and retention of all upstream database identities/registrations. No local compile, UT, E2E, container or runtime validation was run. The old CI job log is expired (HTTP 410), so I do not infer its original cause. Fresh functional CI remains required, including the real Oscar tests and XA behavior if exactly-once support is claimed. Numeric mapping remains outside this patch. The earlier `Decimal(1000,38)` discussion needs a precise evidence boundary: SeaTunnel's `DecimalType` constructor stores p/s rather than itself rejecting that pair; Oscar's JDBC ResultSetMetaData and this connector's direct `v_sys_columns` catalog query are different metadata paths. The driver bytecode does not establish what a live catalog row contains. An unconditional precision cap could change represented values, so I have not inserted one without real metadata/value samples and an agreed overflow policy. This proposal therefore does not clear every review or merge gate. ```diff --- a/docs/en/connectors/sink/Jdbc.md +++ b/docs/en/connectors/sink/Jdbc.md @@ -610,34 +610,6 @@ The following values are starting points. Confirm the driver artifact and version against the database vendor's compatibility matrix. -<<<<<<< HEAD -| datasource | driver | url | xa_data_source_class_name | maven | -|-------------------|----------------------------------------------|---------------------------------------------------------------------|----------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------| -| MySQL | com.mysql.cj.jdbc.Driver | jdbc:mysql://localhost:3306/test | com.mysql.cj.jdbc.MysqlXADataSource | https://mvnrepository.com/artifact/mysql/mysql-connector-java | -| PostgreSQL | org.postgresql.Driver | jdbc:postgresql://localhost:5432/postgres | org.postgresql.xa.PGXADataSource | https://mvnrepository.com/artifact/org.postgresql/postgresql | -| DM | dm.jdbc.driver.DmDriver | jdbc:dm://localhost:5236 | dm.jdbc.driver.DmdbXADataSource | https://mvnrepository.com/artifact/com.dameng/DmJdbcDriver18 | -| Phoenix | org.apache.phoenix.queryserver.client.Driver | jdbc:phoenix:thin:url=http://localhost:8765;serialization=PROTOBUF | / | https://mvnrepository.com/artifact/com.aliyun.phoenix/ali-phoenix-shaded-thin-client | -| SQL Server | com.microsoft.sqlserver.jdbc.SQLServerDriver | jdbc:sqlserver://localhost:1433 | com.microsoft.sqlserver.jdbc.SQLServerXADataSource | https://mvnrepository.com/artifact/com.microsoft.sqlserver/mssql-jdbc | -| Oracle | oracle.jdbc.OracleDriver | jdbc:oracle:thin:@localhost:1521/xepdb1 | oracle.jdbc.xa.OracleXADataSource | https://mvnrepository.com/artifact/com.oracle.database.jdbc/ojdbc8 | -| Oscar | com.oscar.Driver | jdbc:oscar://localhost:2003/osrdb | com.oscar.xa.Jdbc3XAConnection | https://repo1.maven.org/maven2/com/shentongdata/oscarJDBC8 | -| sqlite | org.sqlite.JDBC | jdbc:sqlite:test.db | / | https://mvnrepository.com/artifact/org.xerial/sqlite-jdbc | -| GBase8a | com.gbase.jdbc.Driver | jdbc:gbase://e2e_gbase8aDb:5258/test | / | https://cdn.gbase.cn/products/30/p5CiVwXBKQYIUGN8ecHvk/gbase-connector-java-9.5.0.7-build1-bin.jar | -| StarRocks | com.mysql.cj.jdbc.Driver | jdbc:mysql://localhost:3306/test | / | https://mvnrepository.com/artifact/mysql/mysql-connector-java | -| db2 | com.ibm.db2.jcc.DB2Driver | jdbc:db2://localhost:50000/testdb | com.ibm.db2.jcc.DB2XADataSource | https://mvnrepository.com/artifact/com.ibm.db2.jcc/db2jcc/db2jcc4 | -| saphana | com.sap.db.jdbc.Driver | jdbc:sap://localhost:39015 | / | https://mvnrepository.com/artifact/com.sap.cloud.db.jdbc/ngdbc | -| Doris | com.mysql.cj.jdbc.Driver | jdbc:mysql://localhost:3306/test | / | https://mvnrepository.com/artifact/mysql/mysql-connector-java | -| teradata | com.teradata.jdbc.TeraDriver | jdbc:teradata://localhost/DBS_PORT=1025,DATABASE=test | / | https://mvnrepository.com/artifact/com.teradata.jdbc/terajdbc | -| Redshift | com.amazon.redshift.jdbc42.Driver | jdbc:redshift://localhost:5439/testdb | com.amazon.redshift.xa.RedshiftXADataSource | https://mvnrepository.com/artifact/com.amazon.redshift/redshift-jdbc42 | -| Snowflake | net.snowflake.client.jdbc.SnowflakeDriver | jdbc:snowflake://<account_name>.snowflakecomputing.com | / | https://mvnrepository.com/artifact/net.snowflake/snowflake-jdbc | -| Vertica | com.vertica.jdbc.Driver | jdbc:vertica://localhost:5433 | / | https://repo1.maven.org/maven2/com/vertica/jdbc/vertica-jdbc/12.0.3-0/vertica-jdbc-12.0.3-0.jar | -| Kingbase | com.kingbase8.Driver | jdbc:kingbase8://localhost:54321/db_test | / | https://repo1.maven.org/maven2/cn/com/kingbase/kingbase8/8.6.0/kingbase8-8.6.0.jar | -| OceanBase | com.oceanbase.jdbc.Driver | jdbc:oceanbase://localhost:2881 | / | https://repo1.maven.org/maven2/com/oceanbase/oceanbase-client/2.4.12/oceanbase-client-2.4.12.jar | -| xugu | com.xugu.cloudjdbc.Driver | jdbc:xugu://localhost:5138 | / | https://repo1.maven.org/maven2/com/xugudb/xugu-jdbc/12.2.0/xugu-jdbc-12.2.0.jar | -| InterSystems IRIS | com.intersystems.jdbc.IRISDriver | jdbc:IRIS://localhost:1972/%SYS | / | https://raw.githubusercontent.com/intersystems-community/iris-driver-distribution/main/JDBC/JDK18/intersystems-jdbc-3.8.4.jar | -| opengauss | org.opengauss.Driver | jdbc:opengauss://localhost:5432/postgres | / | https://repo1.maven.org/maven2/org/opengauss/opengauss-jdbc/5.1.0-og/opengauss-jdbc-5.1.0-og.jar | -| Highgo | com.highgo.jdbc.Driver | jdbc:highgo://localhost:5866/highgo | / | https://repo1.maven.org/maven2/com/highgo/HgdbJdbc/6.2.3/HgdbJdbc-6.2.3.jar | -| Dsql | org.postgresql.Driver | jdbc:postgresql://Amazon Aurora DSQL Cluster Endpoint:5432/postgres | org.postgresql.xa.PGXADataSource | https://mvnrepository.com/artifact/org.postgresql/postgresql | -======= | datasource | driver | url | xa_data_source_class_name | maven | |-------------------|----------------------------------------------|---------------------------------------------------------------------|----------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------| | MySQL | com.mysql.cj.jdbc.Driver | jdbc:mysql://localhost:3306/test | com.mysql.cj.jdbc.MysqlXADataSource | https://mvnrepository.com/artifact/mysql/mysql-connector-java | @@ -646,6 +618,7 @@ | Phoenix | org.apache.phoenix.queryserver.client.Driver | jdbc:phoenix:thin:url=http://localhost:8765;serialization=PROTOBUF | / | https://mvnrepository.com/artifact/com.aliyun.phoenix/ali-phoenix-shaded-thin-client | | SQL Server | com.microsoft.sqlserver.jdbc.SQLServerDriver | jdbc:sqlserver://localhost:1433 | com.microsoft.sqlserver.jdbc.SQLServerXADataSource | https://mvnrepository.com/artifact/com.microsoft.sqlserver/mssql-jdbc | | Oracle | oracle.jdbc.OracleDriver | jdbc:oracle:thin:@localhost:1521/xepdb1 | oracle.jdbc.xa.OracleXADataSource | https://mvnrepository.com/artifact/com.oracle.database.jdbc/ojdbc8 | +| Oscar | com.oscar.Driver | jdbc:oscar://localhost:2003/osrdb | com.oscar.xa.Jdbc3XADataSource | https://mvnrepository.com/artifact/com.shentongdata/oscarJDBC8 | | sqlite | org.sqlite.JDBC | jdbc:sqlite:test.db | / | https://mvnrepository.com/artifact/org.xerial/sqlite-jdbc | | GBase8a | com.gbase.jdbc.Driver | jdbc:gbase://localhost:5258/test | / | https://cdn.gbase.cn/products/30/p5CiVwXBKQYIUGN8ecHvk/gbase-connector-java-9.5.0.7-build1-bin.jar | | StarRocks | com.mysql.cj.jdbc.Driver | jdbc:mysql://localhost:3306/test | / | https://mvnrepository.com/artifact/mysql/mysql-connector-java | @@ -664,7 +637,6 @@ | Highgo | com.highgo.jdbc.Driver | jdbc:highgo://localhost:5866/highgo | / | https://repo1.maven.org/maven2/com/highgo/HgdbJdbc/6.2.3/HgdbJdbc-6.2.3.jar | | Dsql | org.postgresql.Driver | jdbc:postgresql://Amazon Aurora DSQL Cluster Endpoint:5432/postgres | org.postgresql.xa.PGXADataSource | https://mvnrepository.com/artifact/org.postgresql/postgresql | | YashanDB | com.yashandb.jdbc.Driver | jdbc:yasdb://localhost:1688/SYS | / | https://repo1.maven.org/maven2/com/yashandb/yashandb-jdbc/1.10.7/yashandb-jdbc-1.10.7.jar | ->>>>>>> 14a27ac26a0a71b56722f86cdf8ec2779094ceaf ## Common patterns --- a/docs/zh/connectors/sink/Jdbc.md +++ b/docs/zh/connectors/sink/Jdbc.md @@ -605,32 +605,6 @@ 下表仅作为起点；驱动制品和版本应以数据库厂商的兼容矩阵为准。 -<<<<<<< HEAD -| 数据源 | driver | url | xa_data_source_class_name | maven | -|------------|----------------------------------------------|---------------------------------------------------------------------|----------------------------------------------------|----------------------------------------------------------------------------------------------------| -| MySQL | com.mysql.cj.jdbc.Driver | jdbc:mysql://localhost:3306/test | com.mysql.cj.jdbc.MysqlXADataSource | https://mvnrepository.com/artifact/mysql/mysql-connector-java | -| PostgreSQL | org.postgresql.Driver | jdbc:postgresql://localhost:5432/postgres | org.postgresql.xa.PGXADataSource | https://mvnrepository.com/artifact/org.postgresql/postgresql | -| DM | dm.jdbc.driver.DmDriver | jdbc:dm://localhost:5236 | dm.jdbc.driver.DmdbXADataSource | https://mvnrepository.com/artifact/com.dameng/DmJdbcDriver18 | -| Phoenix | org.apache.phoenix.queryserver.client.Driver | jdbc:phoenix:thin:url=http://localhost:8765;serialization=PROTOBUF | / | https://mvnrepository.com/artifact/com.aliyun.phoenix/ali-phoenix-shaded-thin-client | -| SQL Server | com.microsoft.sqlserver.jdbc.SQLServerDriver | jdbc:sqlserver://localhost:1433 | com.microsoft.sqlserver.jdbc.SQLServerXADataSource | https://mvnrepository.com/artifact/com.microsoft.sqlserver/mssql-jdbc | -| Oracle | oracle.jdbc.OracleDriver | jdbc:oracle:thin:@localhost:1521/xepdb1 | oracle.jdbc.xa.OracleXADataSource | https://mvnrepository.com/artifact/com.oracle.database.jdbc/ojdbc8 | -| Oscar | com.oscar.Driver | jdbc:oscar://localhost:2003/osrdb | com.oscar.xa.Jdbc3XAConnection | https://repo1.maven.org/maven2/com/shentongdata/oscarJDBC8 | -| sqlite | org.sqlite.JDBC | jdbc:sqlite:test.db | / | https://mvnrepository.com/artifact/org.xerial/sqlite-jdbc | -| GBase8a | com.gbase.jdbc.Driver | jdbc:gbase://e2e_gbase8aDb:5258/test | / | https://cdn.gbase.cn/products/30/p5CiVwXBKQYIUGN8ecHvk/gbase-connector-java-9.5.0.7-build1-bin.jar | -| StarRocks | com.mysql.cj.jdbc.Driver | jdbc:mysql://localhost:3306/test | / | https://mvnrepository.com/artifact/mysql/mysql-connector-java | -| db2 | com.ibm.db2.jcc.DB2Driver | jdbc:db2://localhost:50000/testdb | com.ibm.db2.jcc.DB2XADataSource | https://mvnrepository.com/artifact/com.ibm.db2.jcc/db2jcc/db2jcc4 | -| saphana | com.sap.db.jdbc.Driver | jdbc:sap://localhost:39015 | / | https://mvnrepository.com/artifact/com.sap.cloud.db.jdbc/ngdbc | -| Doris | com.mysql.cj.jdbc.Driver | jdbc:mysql://localhost:3306/test | / | https://mvnrepository.com/artifact/mysql/mysql-connector-java | -| teradata | com.teradata.jdbc.TeraDriver | jdbc:teradata://localhost/DBS_PORT=1025,DATABASE=test | / | https://mvnrepository.com/artifact/com.teradata.jdbc/terajdbc | -| Redshift | com.amazon.redshift.jdbc42.Driver | jdbc:redshift://localhost:5439/testdb | com.amazon.redshift.xa.RedshiftXADataSource | https://mvnrepository.com/artifact/com.amazon.redshift/redshift-jdbc42 | -| Snowflake | net.snowflake.client.jdbc.SnowflakeDriver | jdbc:snowflake://<account_name>.snowflakecomputing.com | / | https://mvnrepository.com/artifact/net.snowflake/snowflake-jdbc | -| Vertica | com.vertica.jdbc.Driver | jdbc:vertica://localhost:5433 | / | https://repo1.maven.org/maven2/com/vertica/jdbc/vertica-jdbc/12.0.3-0/vertica-jdbc-12.0.3-0.jar | -| Kingbase | com.kingbase8.Driver | jdbc:kingbase8://localhost:54321/db_test | / | https://repo1.maven.org/maven2/cn/com/kingbase/kingbase8/8.6.0/kingbase8-8.6.0.jar | -| OceanBase | com.oceanbase.jdbc.Driver | jdbc:oceanbase://localhost:2881 | / | https://repo1.maven.org/maven2/com/oceanbase/oceanbase-client/2.4.12/oceanbase-client-2.4.12.jar | -| opengauss | org.opengauss.Driver | jdbc:opengauss://localhost:5432/postgres | / | https://repo1.maven.org/maven2/org/opengauss/opengauss-jdbc/5.1.0-og/opengauss-jdbc-5.1.0-og.jar | -| Highgo | com.highgo.jdbc.Driver | jdbc:highgo://localhost:5866/highgo | / | https://repo1.maven.org/maven2/com/highgo/HgdbJdbc/6.2.3/HgdbJdbc-6.2.3.jar | -| Dsql | org.postgresql.Driver | jdbc:postgresql://Amazon Aurora DSQL Cluster Endpoint:5432/postgres | org.postgresql.xa.PGXADataSource | https://mvnrepository.com/artifact/org.postgresql/postgresql | -======= | 数据源 | driver | url | xa_data_source_class_name | maven | |------------|----------------------------------------------|--------------------------------------------------------------------|----------------------------------------------------|----------------------------------------------------------------------------------------------------| | MySQL | com.mysql.cj.jdbc.Driver | jdbc:mysql://localhost:3306/test | com.mysql.cj.jdbc.MysqlXADataSource | https://mvnrepository.com/artifact/mysql/mysql-connector-java | @@ -639,6 +613,7 @@ | Phoenix | org.apache.phoenix.queryserver.client.Driver | jdbc:phoenix:thin:url=http://localhost:8765;serialization=PROTOBUF | / | https://mvnrepository.com/artifact/com.aliyun.phoenix/ali-phoenix-shaded-thin-client | | SQL Server | com.microsoft.sqlserver.jdbc.SQLServerDriver | jdbc:sqlserver://localhost:1433 | com.microsoft.sqlserver.jdbc.SQLServerXADataSource | https://mvnrepository.com/artifact/com.microsoft.sqlserver/mssql-jdbc | | Oracle | oracle.jdbc.OracleDriver | jdbc:oracle:thin:@localhost:1521/xepdb1 | oracle.jdbc.xa.OracleXADataSource | https://mvnrepository.com/artifact/com.oracle.database.jdbc/ojdbc8 | +| Oscar | com.oscar.Driver | jdbc:oscar://localhost:2003/osrdb | com.oscar.xa.Jdbc3XADataSource | https://mvnrepository.com/artifact/com.shentongdata/oscarJDBC8 | | sqlite | org.sqlite.JDBC | jdbc:sqlite:test.db | / | https://mvnrepository.com/artifact/org.xerial/sqlite-jdbc | | GBase8a | com.gbase.jdbc.Driver | jdbc:gbase://localhost:5258/test | / | https://cdn.gbase.cn/products/30/p5CiVwXBKQYIUGN8ecHvk/gbase-connector-java-9.5.0.7-build1-bin.jar | | StarRocks | com.mysql.cj.jdbc.Driver | jdbc:mysql://localhost:3306/test | / | https://mvnrepository.com/artifact/mysql/mysql-connector-java | @@ -655,7 +630,6 @@ | Highgo | com.highgo.jdbc.Driver | jdbc:highgo://localhost:5866/highgo | / | https://repo1.maven.org/maven2/com/highgo/HgdbJdbc/6.2.3/HgdbJdbc-6.2.3.jar | | Dsql | org.postgresql.Driver | jdbc:postgresql://Amazon Aurora DSQL Cluster Endpoint:5432/postgres | org.postgresql.xa.PGXADataSource | https://mvnrepository.com/artifact/org.postgresql/postgresql | | YashanDB | com.yashandb.jdbc.Driver | jdbc:yasdb://localhost:1688/SYS | / | https://repo1.maven.org/maven2/com/yashandb/yashandb-jdbc/1.10.7/yashandb-jdbc-1.10.7.jar | ->>>>>>> 14a27ac26a0a71b56722f86cdf8ec2779094ceaf ## 常用模式 --- a/seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/internal/dialect/DatabaseIdentifier.java +++ b/seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/internal/dialect/DatabaseIdentifier.java @@ -28,11 +28,8 @@ public static final String MYSQL = "MySQL"; public static final String STARROCKS = "StarRocks"; public static final String ORACLE = "Oracle"; -<<<<<<< HEAD public static final String OSCAR = "Oscar"; -======= public static final String YASHANDB = "YashanDB"; ->>>>>>> 14a27ac26a0a71b56722f86cdf8ec2779094ceaf public static final String PHOENIX = "Phoenix"; public static final String POSTGRESQL = "Postgres"; public static final String REDSHIFT = "Redshift"; --- a/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-jdbc-e2e/connector-jdbc-e2e-part-5/src/test/java/org/apache/seatunnel/connectors/seatunnel/jdbc/JdbcOscarUpsetIT.java +++ b/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-jdbc-e2e/connector-jdbc-e2e-part-5/src/test/java/org/apache/seatunnel/connectors/seatunnel/jdbc/JdbcOscarUpsetIT.java @@ -256,6 +256,17 @@ return container; } + /** + * Clears the same schema used by the upsert job after each engine run. + * + * <p>The base implementation uses the database argument as a schema. Oscar keeps those + * identifiers separate, so passing OSRDB would leave SYSDBA2's sink rows behind. + */ + @Override + protected void clearTable(String database, String schema, String table) { + super.clearTable(schema, table); + } + @Override public String quoteIdentifier(String field) { return "\"" + field + "\""; ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
