auroflow opened a new pull request, #29247: URL: https://github.com/apache/flink/pull/29247
## What is the purpose of the change Fix [FLINK-40705](https://issues.apache.org/jira/browse/FLINK-40705): pandas UDFs consuming MAP columns can fail across Arrow batches because map child writers retain their previous write positions. Resetting these writers with the parent keeps map offsets and child positions aligned. ## Brief change log - Override `MapWriter.reset()` to reset both key and value writers recursively. - Extend `ArrowReaderWriterTest` to verify MAP values across consecutive batches. ## Verifying this change The regression test checks Arrow IPC round trips with batch sizes 1 and 2, covering direct and nested maps, empty maps, null maps, null values, and a partial final batch. Both parameterized cases failed before the fix and passed afterward during the original implementation. Tests have not been rerun after the latest rebase. ```bash ./mvnw -pl flink-python \ -Dtest=ArrowReaderWriterTest#testMapsAcrossBatches test ``` ## Does this pull request potentially affect one of the following parts: - Dependencies: **no** - The public API: **no** - The serializers: **yes** — Arrow writer lifecycle; no serialization format changes. - The runtime per-record code paths: **yes** — resets child writers at batch boundaries. - Deployment or recovery components: **no** - The S3 file system connector: **no** ## Documentation - Does this pull request introduce a new feature? **no** - If yes, how is the feature documented? **not applicable** --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes Generated-by: Codex (GPT-6) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
