fatmanverse opened a new issue, #12914:
URL: https://github.com/apache/seatunnel/issues/12914
### Search before asking
- [x] I had searched in the issues and found no similar issues.
### What happened
`numeric_special_value_as_null = true` (#12790, #12792) does not cover
PostgreSQL `Infinity` and `-Infinity` once pgjdbc switches to the binary
transfer protocol. The job still fails with:
```
java.lang.IllegalArgumentException: invalid sign in "numeric" value
at org.postgresql.util.ByteConverter.numeric(ByteConverter.java:163)
at org.postgresql.jdbc.PgResultSet.getNumeric(PgResultSet.java:2683)
at org.postgresql.jdbc.PgResultSet.getBigDecimal(PgResultSet.java:2658)
at
org.apache.seatunnel.connectors.seatunnel.jdbc.utils.JdbcFieldTypeUtils.getBigDecimal(JdbcFieldTypeUtils.java:110)
at
org.apache.seatunnel.connectors.seatunnel.jdbc.internal.dialect.psql.PostgresJdbcRowConverter.getDecimalValue(PostgresJdbcRowConverter.java:231)
```
`JdbcFieldTypeUtils#getBigDecimal` only catches `SQLException` and
`ClassCastException`. In binary mode the driver cannot decode the sign word of
the infinities (`0xD000` / `0xF000`) and throws `IllegalArgumentException`, and
`getString` on the same column fails the same way, so `isNumericSpecialValue`
cannot identify the value either. `NaN` is not affected.
This also happens with the default URL. pgjdbc uses binary results for
server-prepared statements, and a statement is server-prepared after
`prepareThreshold` (default 5) executions on one connection. A partitioned read
runs the same split query on one connection, so from the 6th split on the rows
come back in binary. In the repro below the job wrote 51 of 100 rows and failed.
The existing e2e case
`jdbc_postgres_source_and_sink_numeric_special_value_binary.conf` only sets
`binaryTransfer=true`, which is already the driver default, so it runs in text
mode and does not cover this.
### How to reproduce (verified on `dev` e2957d2, PostgreSQL 16, pgjdbc
42.4.3)
```sql
CREATE TABLE public.num_special_part (id INT PRIMARY KEY, n_free NUMERIC);
INSERT INTO public.num_special_part SELECT n, n * 1.5 FROM
generate_series(1, 100) AS n;
UPDATE public.num_special_part SET n_free = 'Infinity' WHERE id = 52;
```
```hocon
source {
Jdbc {
url = "jdbc:postgresql://127.0.0.1:5432/postgres"
driver = "org.postgresql.Driver"
username = "postgres"
password = "..."
table_path = "postgres.public.num_special_part"
partition_column = "id"
split.size = 4
numeric_special_value_as_null = true
}
}
```
The same failure happens on the first row with `?prepareThreshold=-1` in the
URL, with both `table_path` and `query`.
### Expected behavior
With `numeric_special_value_as_null = true`, `Infinity` and `-Infinity` are
read as null in binary mode as they are in text mode.
### SeaTunnel Version
dev (3.0.1-SNAPSHOT, e2957d2)
### Zeta or Flink or Spark Version
Zeta
### Java or Scala Version
JDK 11
### Are you willing to submit PR?
- [x] Yes I am willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
