DanielLeens commented on PR #11012: URL: https://github.com/apache/seatunnel/pull/11012#issuecomment-6368818043
I downloaded the actual logs for run `30559645732`, attempt **7**, on head `bca8da53ff0a8806e69fd010e2dc243c0e84d20c`. They change the diagnosis in the earlier thread: - The current test uses **`ifnazar/sybase_15_7:latest`**, and already configures a **five-minute startup timeout**. - In [JDK 11 job 91177773929](https://github.com/lazyfetch/seatunnel/actions/runs/30559645732/job/91177773929), ASE reports `Recovery complete`, the container starts in **31.12 seconds**, and `testCatalogWithCatalogUtils` passes. This execution is not stuck starting `nguoianphu/docker-sybase`. - The Flink/Zeta executions then fail with this direct call chain: ```text java.lang.AbstractMethodError net.sourceforge.jtds.jdbc.JtdsResultSet.getObject(JtdsResultSet.java:1800) JdbcFieldTypeUtils.getLocalDateTime(JdbcFieldTypeUtils.java:129) AbstractJdbcRowConverter.toInternal(AbstractJdbcRowConverter.java:119) ``` `getLocalDateTime()` first invokes the JDBC 4.2 typed getter and catches `SQLException`/`UnsupportedOperationException`, but not the `AbstractMethodError` thrown by the JDBC 3 jTDS driver. That shared timestamp path was introduced by #10724 (`872077f6480`) and is still present in fetched upstream `dev` `14a27ac26a0a71b56722f86cdf8ec2779094ceaf`. The local proposal below adds only `AbstractMethodError` to this existing typed-getter fallback; modern JDBC 4.2 reads, UTC-calendar fallback ordering, and unrelated linkage-error propagation are preserved. It has not been committed or pushed. The last captured operation before the three-hour cancellation is the Spark 2 launch. The log does not contain its completed result, so the cause of that remaining hang is still unproven. A startup-timeout change or disabling `JdbcSybaseIT` would not repair the timestamp exception above. I have not disabled tests or rerun the exhausted workflow. I also checked the current review requests against source: the docs are already in `docs/en/connectors/source/Sybase.md` and `docs/zh/connectors/source/Sybase.md`; the jTDS dependency in the JDBC E2E POM is already **test-scoped**. The attached docs-only patch makes the actual type/metadata limitations explicit and calls out the independently unresolved `Connection.isValid()` compatibility gap. It changes no driver, dependency or production code. Save as `sybase-docs.patch`, then run `git apply --check sybase-docs.patch` and `git apply sybase-docs.patch` on the exact head above. <details><summary>EN/ZH documentation patch</summary> ```diff diff --git a/docs/en/connectors/source/Sybase.md b/docs/en/connectors/source/Sybase.md index 52abb6470d2..3c9dba6ee04 100644 --- a/docs/en/connectors/source/Sybase.md +++ b/docs/en/connectors/source/Sybase.md @@ -50,6 +50,14 @@ Read external data source data through JDBC. > Please download the support list corresponding to 'Maven' and copy it to the '$SEATUNNEL_HOME/plugins/jdbc/lib/' working directory<br/> > For example Sybase datasource: cp jtds-xxx.jar $SEATUNNEL_HOME/plugins/jdbc/lib/ +## Current limitations + +This dialect reads Sybase through the generic JDBC metadata fallback; it does not provide a Sybase catalog. Exact `table_path` and `table_list` entries use that fallback. Catalog-backed regular-expression table discovery (`use_regex`) is not implemented for Sybase; use explicit table entries or `query` instead. + +Only the types in the mapping table below are supported. BINARY, VARBINARY, IMAGE, UNICHAR, UNIVARCHAR, UNITEXT, unsigned integer types, BIGDATETIME, BIGTIME, and the binary rowversion TIMESTAMP type are not mapped. Select supported columns explicitly or cast unsupported columns to a supported type in `query`. + +The examples use plaintext TDS. For encrypted connections, configure a driver and database deployment that supports the required encryption and certificate validation; this connector does not configure TLS itself. The jTDS driver is supplied by the user, and its dependency in this PR is test-scoped. The driver does not implement JDBC 4 `Connection.isValid()`, so `connection_check_timeout_sec` does not make that operation compatible; connection validation compatibility still requires a runtime fix. + ## Data Type Mapping | Sybase Data type | Seatunnel Data type | diff --git a/docs/zh/connectors/source/Sybase.md b/docs/zh/connectors/source/Sybase.md index d1db0458163..97b676c60f4 100644 --- a/docs/zh/connectors/source/Sybase.md +++ b/docs/zh/connectors/source/Sybase.md @@ -50,6 +50,14 @@ import ChangeLog from '../changelog/connector-jdbc.md'; > 请下载对应 'Maven' 的支持列表，并将其复制到 '$SEATUNNEL_HOME/plugins/jdbc/lib/' 工作目录<br/> > 例如 Sybase 数据源：cp jtds-xxx.jar $SEATUNNEL_HOME/plugins/jdbc/lib/ +## 当前限制 + +该方言通过通用 JDBC 元数据回退路径读取 Sybase，没有提供 Sybase Catalog。精确的 `table_path` 和 `table_list` 条目使用此回退路径。Sybase 尚未实现依赖 Catalog 的正则表达式表发现（`use_regex`）；请使用明确的表条目或 `query`。 + +仅支持下方映射表列出的类型。BINARY、VARBINARY、IMAGE、UNICHAR、UNIVARCHAR、UNITEXT、无符号整数、BIGDATETIME、BIGTIME 和二进制 rowversion TIMESTAMP 尚未映射。请显式选择支持的列，或在 `query` 中将不支持的列转换为支持的类型。 + +示例使用明文 TDS。需要加密连接时，必须配置支持所需加密和证书校验的驱动与数据库部署；连接器本身不配置 TLS。jTDS 驱动由用户提供，本 PR 中的依赖仅用于测试。该驱动未实现 JDBC 4 `Connection.isValid()`，因此 `connection_check_timeout_sec` 无法使此操作兼容；连接校验兼容性仍需运行时修复。 + ## 数据类型映射 | Sybase 数据类型 | Seatunnel 数据类型 | ``` </details> Validation: scoped formatting passed for `connector-jdbc` and JDBC E2E part 3; whitespace and reverse patch applicability checks passed. No local compilation/tests/runtime validation were run, and these unpushed patches have no CI result. The remaining actions are a reviewed JDBC compatibility correction, investigation of the Spark 2 hang, and a fresh successful CI result on the contributor's resulting head. <details><summary>Shared JDBC timestamp proposal with UT/E2E regression coverage</summary> This affects the common JDBC timestamp reader, so please review the generic scope before adopting it. The existing utility test class covers the modern exact-value path, jTDS-style `AbstractMethodError` fallback with nanosecond precision, SQL NULL preservation, and propagation of an unrelated `NoClassDefFoundError`. The existing Sybase IT now supplies a fixed timestamp and asserts that exact value in its sink; existing 100-row assertions remain intact. No tests are disabled or weakened. Save the patch below as `jdbc3-timestamp.patch` and run `git apply --check jdbc3-timestamp.patch` before applying on the exact PR head. Scoped formatting and reverse applicability checks passed; these behavioral tests have **not** been executed, and fresh CI is required. This proposal addresses the evidenced timestamp exception, while the connection-validation gap and unfinished Spark 2 execution remain separate. ```diff diff --git a/seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/utils/JdbcFieldTypeUtils.java b/seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/utils/JdbcFieldTypeUtils.java index 08e875a462e..b77e8961003 100644 --- a/seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/utils/JdbcFieldTypeUtils.java +++ b/seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/utils/JdbcFieldTypeUtils.java @@ -127,8 +127,8 @@ public final class JdbcFieldTypeUtils { // Prefer the modern JDBC 4.2 API — returns wall-clock value directly, no TZ involved try { return resultSet.getObject(columnIndex, LocalDateTime.class); - } catch (SQLException | UnsupportedOperationException ignored) { - // Driver does not support getObject(index, LocalDateTime.class) — fall back + } catch (SQLException | UnsupportedOperationException | AbstractMethodError ignored) { + // JDBC 3 drivers such as jTDS cannot implement the JDBC 4.2 typed getter. } // Try UTC Calendar to avoid JVM-default-timezone influence. // A new Calendar is created per call to avoid thread-safety issues with a shared instance. diff --git a/seatunnel-connectors-v2/connector-jdbc/src/test/java/org/apache/seatunnel/connectors/seatunnel/jdbc/utils/JdbcFieldTypeUtilsTest.java b/seatunnel-connectors-v2/connector-jdbc/src/test/java/org/apache/seatunnel/connectors/seatunnel/jdbc/utils/JdbcFieldTypeUtilsTest.java index f9ac9cb819b..debc20b3140 100644 --- a/seatunnel-connectors-v2/connector-jdbc/src/test/java/org/apache/seatunnel/connectors/seatunnel/jdbc/utils/JdbcFieldTypeUtilsTest.java +++ b/seatunnel-connectors-v2/connector-jdbc/src/test/java/org/apache/seatunnel/connectors/seatunnel/jdbc/utils/JdbcFieldTypeUtilsTest.java @@ -23,16 +23,64 @@ import java.sql.ResultSet; import java.sql.SQLException; import java.sql.Timestamp; import java.time.Instant; +import java.time.LocalDateTime; import java.time.OffsetDateTime; import java.time.ZoneOffset; +import java.util.Calendar; import java.util.Date; import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertNull; +import static org.junit.jupiter.api.Assertions.assertSame; +import static org.junit.jupiter.api.Assertions.assertThrows; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.ArgumentMatchers.eq; import static org.mockito.Mockito.mock; import static org.mockito.Mockito.when; public class JdbcFieldTypeUtilsTest { + @Test + public void testGetLocalDateTimeUsesTypedGetter() throws SQLException { + LocalDateTime expected = LocalDateTime.of(2026, 1, 2, 3, 4, 5, 123456789); + ResultSet rs = mock(ResultSet.class); + when(rs.getObject(1, LocalDateTime.class)).thenReturn(expected); + + assertEquals(expected, JdbcFieldTypeUtils.getLocalDateTime(rs, 1)); + } + + @Test + public void testGetLocalDateTimeFallsBackForJdbc3Driver() throws SQLException { + LocalDateTime expected = LocalDateTime.of(2026, 1, 2, 3, 4, 5, 123456789); + ResultSet rs = mock(ResultSet.class); + when(rs.getObject(1, LocalDateTime.class)).thenThrow(new AbstractMethodError()); + when(rs.getTimestamp(eq(1), any(Calendar.class))).thenReturn(Timestamp.valueOf(expected)); + + assertEquals(expected, JdbcFieldTypeUtils.getLocalDateTime(rs, 1)); + } + + @Test + public void testGetLocalDateTimePreservesNullForJdbc3Driver() throws SQLException { + ResultSet rs = mock(ResultSet.class); + when(rs.getObject(1, LocalDateTime.class)).thenThrow(new AbstractMethodError()); + when(rs.getTimestamp(eq(1), any(Calendar.class))).thenReturn(null); + + assertNull(JdbcFieldTypeUtils.getLocalDateTime(rs, 1)); + } + + @Test + public void testGetLocalDateTimePropagatesUnrelatedLinkageError() throws SQLException { + ResultSet rs = mock(ResultSet.class); + NoClassDefFoundError failure = new NoClassDefFoundError("driver dependency"); + when(rs.getObject(1, LocalDateTime.class)).thenThrow(failure); + + assertSame( + failure, + assertThrows( + NoClassDefFoundError.class, + () -> JdbcFieldTypeUtils.getLocalDateTime(rs, 1))); + } + @Test public void testGetOffsetDateTimeFromTimestampUsesInstant() throws SQLException { Instant instant = Instant.parse("2025-01-01T00:00:00Z"); diff --git a/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-jdbc-e2e/connector-jdbc-e2e-part-3/src/test/java/org/apache/seatunnel/connectors/seatunnel/jdbc/JdbcSybaseIT.java b/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-jdbc-e2e/connector-jdbc-e2e-part-3/src/test/java/org/apache/seatunnel/connectors/seatunnel/jdbc/JdbcSybaseIT.java index f4711e6a176..f532f35c0dc 100644 --- a/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-jdbc-e2e/connector-jdbc-e2e-part-3/src/test/java/org/apache/seatunnel/connectors/seatunnel/jdbc/JdbcSybaseIT.java +++ b/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-jdbc-e2e/connector-jdbc-e2e-part-3/src/test/java/org/apache/seatunnel/connectors/seatunnel/jdbc/JdbcSybaseIT.java @@ -158,7 +158,8 @@ public class JdbcSybaseIT extends AbstractJdbcIT { i % 2 == 0, // BIT_TEST "CharValue" + i, // CHAR_TEST java.sql.Date.valueOf(LocalDate.now()), // DATE_TEST - java.sql.Timestamp.valueOf(LocalDateTime.now()), // DATETIME_TEST + java.sql.Timestamp.valueOf( + LocalDateTime.of(2026, 1, 2, 3, 4, 5)), // DATETIME_TEST new BigDecimal("123.45"), // DECIMAL_TEST 3.14d, // FLOAT_TEST new BigDecimal("567.89"), // MONEY_TEST diff --git a/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-jdbc-e2e/connector-jdbc-e2e-part-3/src/test/resources/jdbc_sybase_source_and_assert.conf b/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-jdbc-e2e/connector-jdbc-e2e-part-3/src/test/resources/jdbc_sybase_source_and_assert.conf index a73fb78537e..ed8e0661444 100644 --- a/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-jdbc-e2e/connector-jdbc-e2e-part-3/src/test/resources/jdbc_sybase_source_and_assert.conf +++ b/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-jdbc-e2e/connector-jdbc-e2e-part-3/src/test/resources/jdbc_sybase_source_and_assert.conf @@ -46,6 +46,19 @@ sink { rule_value = 100 } ] + field_rules = [ + { + # Exercise jTDS timestamp extraction, not only the number of rows emitted. + field_name = DATETIME_TEST + field_type = timestamp + field_value = [ + { + rule_type = NOT_NULL + equals_to = "2026-01-02T03:04:05" + } + ] + } + ] } } } ``` </details> <!-- daniel-fcp-repair-20261023-pr11012 --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
