siwen-yu opened a new pull request, #12792:
URL: https://github.com/apache/seatunnel/pull/12792
### Purpose of this pull request
Reading a PostgreSQL `numeric`/`decimal` column that contains `NaN`,
`Infinity` or
`-Infinity` aborts the whole job, because these special values have no
`BigDecimal`
representation: the pgjdbc driver throws
`PSQLException: Bad value for type BigDecimal : NaN` in text transfer mode
and a
`ClassCastException` in binary transfer mode (where the driver decodes the
value into a
`Double`). A single such cell fails the entire job.
This PR adds an opt-in source option `numeric_special_value_as_null`
(default `false`,
following the existing `decimal_type_narrowing` / `int_type_narrowing`
pattern). When
enabled, `NaN`/`Infinity`/`-Infinity` values are read as `null` with a
warning logged
instead of aborting the job. Unrelated driver errors are still rethrown, and
the default
behavior is unchanged.
Closes #12790
### Does this PR introduce _any_ user-facing change?
Yes. A new JDBC source option `numeric_special_value_as_null` is added,
documented in
`docs/en/connectors/source/Jdbc.md` and `docs/zh/connectors/source/Jdbc.md`.
It only takes
effect for the PostgreSQL dialect (including the `postgresLow` compatible
mode). The
default behavior is unchanged, so existing jobs are not affected unless they
opt in.
### How was this patch tested?
Unit tests added:
- `JdbcFieldTypeUtilsTest`: normal value passthrough;
`NaN`/`Infinity`/`-Infinity` narrowed
to `null` when the option is enabled; binary-mode `ClassCastException`
path; error kept
when the option is disabled; unrelated errors still rethrown.
- `PostgresJdbcRowConverterTest`: DECIMAL column with a normal value; `NaN`
narrowed to
`null` when enabled; error thrown when the option is disabled.
The full `connector-jdbc` test suite was run locally: 543 tests, the only
failure
(`VerticaDialectTest.testUpsertStatementByTableSchema`) is pre-existing on a
clean tree and
unrelated to this change.
### Check list
- [ ] If any new Jar binary package adding in your PR, please add License
Notice according
[New License
Guide](https://github.com/apache/seatunnel/blob/dev/docs/en/contribution/new-license.md)
(N/A, no new dependency)
- [x] If necessary, please update the documentation to describe the new
feature. (en/zh
Jdbc.md updated)
- [ ] If necessary, please update `incompatible-changes.md` (N/A, no
incompatible change,
default behavior unchanged)
- [ ] If you are contributing the connector code, please check that the
following files are
updated (N/A, no new connector, only a new source option in connector-jdbc)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
