ryanmeowy opened a new pull request, #13143:
URL: https://github.com/apache/seatunnel/pull/13143

   Yes — I am contributing this change to Apache SeaTunnel.
   
   ### Purpose of this pull request
   
   Fixes #13142.
   
   A File sink job fails with `NullPointerException` on the first row when a 
`partition_by` column value is NULL. A nullable column is a legal schema: 
`FileSinkConfig` only validates that the partition columns exist in the sink 
columns, and the `partition_by` documentation does not require non-null values. 
Today the job starts and then dies at write time; on restart it fails on the 
same row.
   
   Root cause: `AbstractWriteStrategy.generatorPartitionDir()` renders the 
partition directory through `StringBuilder.append(Object)` — which already 
turns a null into the text `null` — but takes the values that are carried into 
the commit info via `.toString()` on the raw field, without a null check:
   
   - `partition_dir_expression` unset: `AbstractWriteStrategy.java:529`
   - `partition_dir_expression` set: the same `toString()` twice, for the 
`${vN}` substitution map (`:538`) and for `vals` (`:539`)
   
   Since the directory component already becomes the text `null`, this is an 
unguarded dereference, not an unrepresentable value.
   
   Fix: derive the partition value once with `String.valueOf(field)`. It keeps 
the existing rendering (null -> `"null"`, anything else -> `toString()`) and 
makes the directory name and the values in `FileCommitInfo` / 
`FileAggregatedCommitInfo` consistent. The values list is only carried through 
`toString()` and the committer's aggregation; no SQL is built from it.
   
   Compatibility notes:
   
   - Non-null partition values are unchanged.
   - Multi-column partitions: only the NULL column renders as `null`.
   - `partition_dir_expression`: `${vN}` substitutes the literal `null`, 
consistent with the rest of the expression behavior.
   - Hive-style `__HIVE_DEFAULT_PARTITION__` was intentionally not introduced 
here: that would be a user-visible output path change and does not belong in an 
NPE fix. Note that the Hive sink shares this writer through 
`WriteStrategyFactory`, and its aggregated committer registers partitions from 
the directory keys, so a NULL value becomes the Hive partition `col=null` 
(literal string), not a NULL partition. If the Hive convention should win, that 
is a separate decision that changes the directory name as well.
   
   ### Does this PR introduce _any_ user-facing change?
   
   Yes. A row whose partition column is NULL used to fail the job with a 
`NullPointerException`; it now writes to a partition directory `col=null` (the 
same text the directory name already used) and the committed partition values 
agree with it. Non-null values, directory layout for non-null values, and the 
`partition_dir_expression` behavior for non-null inputs are unchanged.
   
   The `partition_by` documentation does not currently state the NULL 
semantics. I kept this PR to the two source files; I can add a sentence to 
`docs/en` and `docs/zh` if maintainers prefer to document the behavior in the 
same change.
   
   ### How was this patch tested?
   
   `AbstractWriteStrategyTest` (extends the existing class and reuses its 
config helpers):
   
   - `generatorPartitionDirWithNullPartitionValue` — a null partition value 
yields key `c_null=null` and values `["null"]`; this test throws NPE on the 
parent commit.
   - `generatorPartitionDirWithNullPartitionValueAndExpression` — same with 
`partition_dir_expression = "${k0}/${v0}"`, asserting key `c_null/null`. The 
expression is deliberately not `${k0}=${v0}`: that renders `c_null=null`, 
identical to the non-expression branch, so the test could not tell whether the 
expression path ran at all. With `${k0}/${v0}` an ignored expression fails the 
assertion. Pre-fix this test errors at `:538`, which also proves the branch is 
exercised.
   - `generatorPartitionDirKeepsNonNullPartitionValue` — regression guard, 
non-null output identical to today.
   
   Commands:
   
   ```bash
   JAVA_TOOL_OPTIONS="-Dnet.bytebuddy.experimental=true" \
     ./mvnw -pl seatunnel-connectors-v2/connector-file/connector-file-base test 
\
     -Dtest='AbstractWriteStrategyTest,FileSinkConfigTest' 
-Dspotless.check.skip=true
   # Tests run: 13, Failures: 0, Errors: 0
   ```
   
   Also ran the whole `connector-file-base` module: 382 tests, 370 passed; the 
12 errors are `xerial.snappy: no native library ... Mac/aarch64` in 
`source/reader` Parquet tests (environmental on this machine, identical set on 
the parent commit).
   
   A standalone harness that drives the real `TextWriteStrategy` with a mocked 
`FileSinkConfig` prints `partition dir = {c_null=null=[null]}` after the fix 
(before: NPE at `:529`).
   
   ### Check list
   
   - [x] If any new Jar binary package adding in your PR, please add License 
Notice according [New License 
Guide](https://github.com/apache/seatunnel/blob/dev/docs/en/developer/new-license.md)
 — no new jar
   - [x] If necessary, please update the documentation to describe the new 
feature. — no new feature; documented NULL semantics offered above
   - [x] If necessary, please update `incompatible-changes.md` — not an 
incompatible change
   - [x] Connector checklist (plugin-mapping, seatunnel-dist pom, labeler conf, 
e2e, plugin_config) — not a new connector
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to