Hi,
I was investigating a problem in Apache Beam test infrastructure after
introduction of FlinkRunner for Flink 1.18 (see [1]). There was a
persistent failure of integration test which uses `mvn exec:java` to run
a word count example. The test (for FlinkRunner) uses MiniCluster and
works fine up to version 1.17. Running it against 1.18 results in
`ClassNotFoundException: org.apache.flink.api.common.ExecutionConfig`.
Digging deeper I was able to isolate the problem to a classloader issue,
some invocation path results in ClassLoaderObjectInputStream [2] being
instantiated with `AppClassLoader` (which holds only the maven jar)
instead of context classloader (URLClassLoader provided by maven to the
executed main() method). I was not yet able to find out exactly which
change introduced this regression, but it seems similar behavior has
been observed earlier [3]. I believe this is likely bug in Flink ([4])
as the MiniCluster should use provided context classloader under all
circumstances. Are there any known workarounds, should this be fixed in
Flink itself or should we introduce some measures to fix this in Beam?
Thanks for any suggestions,
Jan
[1] https://github.com/apache/beam/pull/31062#issuecomment-2106103389
[2]
https://github.com/apache/flink/blob/c7c1d78752836b96591e31422c65b85eca38bd50/flink-core/src/main/java/org/apache/flink/util/InstantiationUtil.java#L62
[3] https://www.mail-archive.com/user@flink.apache.org/msg52035.html
[4] https://github.com/apache/beam/pull/31328#issuecomment-2120011239
