beetle0915 opened a new pull request, #29398: URL: https://github.com/apache/flink/pull/29398
## What is the purpose of the change Fix [FLINK-40658](https://issues.apache.org/jira/browse/FLINK-40658): UNNEST drops null ROW elements from collections. WITH ORDINALITY also produces an incorrectly shaped row for these elements, which can fail during execution. ## Brief change log - Record the original ROW element arity when specializing collection UNNEST functions. - Convert null ROW elements into internal rows with null fields before collection, preserving ARRAY elements and MULTISET multiplicities. - Apply the same normalization before appending ordinality. Derive the output type and element arity from the original element type inside the constructor. - Add regression coverage for repeated null ROW elements, LEFT JOIN UNNEST, MULTISET multiplicity, and WITH ORDINALITY. Retain control coverage for non-null ROWs whose fields are all null. MAP handling and the general TableFunction.collect() behavior are unchanged. ## Verifying this change The batch and streaming UnnestITCase suites pass locally: 95 tests, 0 failures, 0 errors, and 0 skipped (31 batch tests and 64 streaming test invocations). ```bash ./mvnw -pl flink-table/flink-table-planner -DskipITs -Dfast \ -Dcheckstyle.skip -Dspotless.check.skip \ '-Dtest=org.apache.flink.table.planner.runtime.batch.sql.UnnestITCase,org.apache.flink.table.planner.runtime.stream.sql.UnnestITCase' test ``` All eight added test methods also reproduce the defect with the three runtime implementation files restored to the unmodified baseline. Ordinary UNNEST loses rows; WITH ORDINALITY fails during execution. Restoring and rebuilding the fix makes the full 95-test suite pass again. After rebasing onto master at 953f843d578, the runtime and its dependencies were rebuilt and the same 95 tests passed. Checkstyle and Spotless passed with: ```bash ./mvnw -pl flink-table/flink-table-runtime,flink-table/flink-table-planner \ -DskipTests validate ``` Full-repository clean verify has not been run. This PR is initially a draft pending broader CI validation and review. ## Does this pull request potentially affect one of the following parts: - Dependencies: no. - The public API: no; the changed functions are internal. - The serializers: no state serializer changes. - Runtime per-record code paths: yes. Collection elements have an additional null/ROW check; null ROW elements allocate a GenericRowData and now produce the previously missing output. No performance benchmark has been run. - Deployment or recovery: no intended changes to protocols, checkpoints, or savepoints. - The S3 file system connector: no. ## Documentation - Does this pull request introduce a new feature? No, this corrects existing UNNEST behavior. - No DataFrame-specific workaround or unrelated planner changes are included. --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes (please specify the tool below) Generated-by: OpenAI Codex (GPT-5) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
