jiayuasu opened a new pull request, #29150: URL: https://github.com/apache/flink/pull/29150
## What is the purpose of the change Fix repeated Table API job submissions that use RAW types loaded from the job jar. Calcite's static type caches can replace a later job's type with one that retains an earlier job's classloader. The same class name and serializer snapshot do not make those Java classes interchangeable. This addresses [FLINK-20986](https://issues.apache.org/jira/browse/FLINK-20986) and the same failure reported in apache/sedona#1052, where the first REST submission succeeds and the second fails during `ST_S2CellIDs` type inference. ## Brief change log - Bypass both global canonicalization paths for modern and legacy RAW types, including RAW nested in arrays, maps, multisets, rows, and join types. - Preserve the factory's existing logical-type cache and global canonicalization for types without RAW fields. - Add tests with separate classloaders for both RAW representations, nullability, map keys, and nested rows. Check that primitive types still use the global cache. - Verify RAW JSON round trips preserve the logical type and serialized representation without requiring a shared object across factories. ## Verifying this change Added 42 classloader cases and 5 cache controls in `FlinkTypeFactoryTest`. The regressions were checked before and after the fix. On Java 17, a clean planner build with eight selected suites passed all 770 tests: `FlinkTypeFactoryTest`, `LogicalRelDataTypeConverterTest`, `RelDataTypeJsonSerdeTest`, `LogicalTypeJsonSerdeTest`, `TimeIndicatorRelDataTypeTest`, stream SQL `CalcTest`, and batch/stream Table API `CorrelateTest`. Spotless and Checkstyle passed. The full repository test suite was not run. The modern RAW cache change was also tested separately on Flink 1.19.3 with Java 11, one JobManager and two TaskManager JVMs on one host. Three consecutive REST submissions of the unchanged Sedona job completed. Each produced 2,048 ARRAY rows and 2,048 UNNEST rows, with both workers and network exchange verified. The unmodified cluster fails on the second submission. This cluster check used the Scala 1.19 implementation; the current Java implementation is covered by the native tests above. ## Does this pull request potentially affect one of the following parts - Dependencies: no - Public API: no - Serializers: no format changes - Runtime per-record code paths: no - Deployment or recovery: fixes repeated job planning; no deployment or recovery protocol changes - S3 file system connector: no ## Documentation - New feature: no - Feature documentation: not applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
