Henning Schmiedehausen created FLINK-33363: ----------------------------------------------
Summary: docker bases images can't run the java compiler Key: FLINK-33363 URL: https://issues.apache.org/jira/browse/FLINK-33363 Project: Flink Issue Type: Bug Components: flink-docker Affects Versions: 1.17.1 Reporter: Henning Schmiedehausen I have set up a small cluster (Job Manager + 2 Task managers) using docker compose. When submitting a flink job that needs Calcite planning, it crashes with org.apache.flink.client.program.ProgramInvocationException: The main method caused an error: Unable to instantiate java compiler at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:372) ~[flink-dist-1.17.1.jar:1.17.1] at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:222) ~[flink-dist-1.17.1.jar:1.17.1] at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:105) ~[flink-dist-1.17.1.jar:1.17.1] at org.apache.flink.client.deployment.application.DetachedApplicationRunner.tryExecuteJobs(DetachedApplicationRunner.java:84) ~[flink-dist-1.17.1.jar:1.17.1] at org.apache.flink.client.deployment.application.DetachedApplicationRunner.run(DetachedApplicationRunner.java:70) ~[flink-dist-1.17.1.jar:1.17.1] at org.apache.flink.runtime.webmonitor.handlers.JarRunHandler.lambda$handleRequest$0(JarRunHandler.java:108) ~[flink-dist-1.17.1.jar:1.17.1] at java.util.concurrent.CompletableFuture$AsyncSupply.run(Unknown Source) [?:?] at java.lang.Thread.run(Unknown Source) [?:?] Caused by: java.lang.IllegalStateException: Unable to instantiate java compiler at org.apache.calcite.rel.metadata.JaninoRelMetadataProvider.compile(JaninoRelMetadataProvider.java:163) ~[?:?] at org.apache.calcite.rel.metadata.JaninoRelMetadataProvider.generateCompileAndInstantiate(JaninoRelMetadataProvider.java:141) ~[?:?] at org.apache.calcite.rel.metadata.JaninoRelMetadataProvider.lambda$static$0(JaninoRelMetadataProvider.java:73) ~[?:?] at org.apache.flink.calcite.shaded.com.google.common.cache.CacheLoader$FunctionToCacheLoader.load(CacheLoader.java:165) ~[?:?] at org.apache.flink.calcite.shaded.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3529) ~[?:?] at org.apache.flink.calcite.shaded.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2278) ~[?:?] at org.apache.flink.calcite.shaded.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2155) ~[?:?] at org.apache.flink.calcite.shaded.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045) ~[?:?] at org.apache.flink.calcite.shaded.com.google.common.cache.LocalCache.get(LocalCache.java:3951) ~[?:?] at org.apache.flink.calcite.shaded.com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974) ~[?:?] at org.apache.flink.calcite.shaded.com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4958) ~[?:?] at org.apache.calcite.rel.metadata.JaninoRelMetadataProvider.revise(JaninoRelMetadataProvider.java:197) ~[?:?] at org.apache.calcite.rel.metadata.RelMetadataQueryBase.revise(RelMetadataQueryBase.java:118) ~[?:?] at org.apache.calcite.rel.metadata.RelMetadataQuery.getPulledUpPredicates(RelMetadataQuery.java:844) ~[?:?] at org.apache.calcite.rel.rules.ReduceExpressionsRule$ProjectReduceExpressionsRule.onMatch(ReduceExpressionsRule.java:307) ~[?:?] at org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:337) ~[?:?] at org.apache.calcite.plan.hep.HepPlanner.applyRule(HepPlanner.java:565) ~[?:?] at org.apache.calcite.plan.hep.HepPlanner.applyRules(HepPlanner.java:428) ~[?:?] at org.apache.calcite.plan.hep.HepPlanner.executeInstruction(HepPlanner.java:251) ~[?:?] at org.apache.calcite.plan.hep.HepInstruction$RuleInstance.execute(HepInstruction.java:130) ~[?:?] at org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:208) ~[?:?] at org.apache.calcite.plan.hep.HepPlanner.findBestExp(HepPlanner.java:195) ~[?:?] at org.apache.flink.table.planner.plan.optimize.program.FlinkHepProgram.optimize(FlinkHepProgram.scala:64) ~[?:?] at org.apache.flink.table.planner.plan.optimize.program.FlinkHepRuleSetProgram.optimize(FlinkHepRuleSetProgram.scala:78) ~[?:?] at org.apache.flink.table.planner.plan.optimize.program.FlinkChainedProgram.$anonfun$optimize$1(FlinkChainedProgram.scala:59) ~[?:?] at scala.collection.TraversableOnce.$anonfun$foldLeft$1(TraversableOnce.scala:156) ~[flink-scala_2.12-1.17.1.jar:1.17.1] at scala.collection.TraversableOnce.$anonfun$foldLeft$1$adapted(TraversableOnce.scala:156) ~[flink-scala_2.12-1.17.1.jar:1.17.1] at scala.collection.Iterator.foreach(Iterator.scala:937) ~[flink-scala_2.12-1.17.1.jar:1.17.1] at scala.collection.Iterator.foreach$(Iterator.scala:937) ~[flink-scala_2.12-1.17.1.jar:1.17.1] at scala.collection.AbstractIterator.foreach(Iterator.scala:1425) ~[flink-scala_2.12-1.17.1.jar:1.17.1] at scala.collection.IterableLike.foreach(IterableLike.scala:70) ~[flink-scala_2.12-1.17.1.jar:1.17.1] at scala.collection.IterableLike.foreach$(IterableLike.scala:69) ~[flink-scala_2.12-1.17.1.jar:1.17.1] at scala.collection.AbstractIterable.foreach(Iterable.scala:54) ~[flink-scala_2.12-1.17.1.jar:1.17.1] at scala.collection.TraversableOnce.foldLeft(TraversableOnce.scala:156) ~[flink-scala_2.12-1.17.1.jar:1.17.1] at scala.collection.TraversableOnce.foldLeft$(TraversableOnce.scala:154) ~[flink-scala_2.12-1.17.1.jar:1.17.1] at scala.collection.AbstractTraversable.foldLeft(Traversable.scala:104) ~[flink-scala_2.12-1.17.1.jar:1.17.1] at org.apache.flink.table.planner.plan.optimize.program.FlinkChainedProgram.optimize(FlinkChainedProgram.scala:55) ~[?:?] at org.apache.flink.table.planner.plan.optimize.StreamCommonSubGraphBasedOptimizer.optimizeTree(StreamCommonSubGraphBasedOptimizer.scala:176) ~[?:?] at org.apache.flink.table.planner.plan.optimize.StreamCommonSubGraphBasedOptimizer.doOptimize(StreamCommonSubGraphBasedOptimizer.scala:83) ~[?:?] at org.apache.flink.table.planner.plan.optimize.CommonSubGraphBasedOptimizer.optimize(CommonSubGraphBasedOptimizer.scala:87) ~[?:?] at org.apache.flink.table.planner.delegation.PlannerBase.optimize(PlannerBase.scala:329) ~[?:?] at org.apache.flink.table.planner.delegation.PlannerBase.translate(PlannerBase.scala:195) ~[?:?] at org.apache.flink.table.api.bridge.internal.AbstractStreamTableEnvironmentImpl.toStreamInternal(AbstractStreamTableEnvironmentImpl.java:224) ~[flink-table-api-java-uber-1.17.1.jar:1.17.1] at org.apache.flink.table.api.bridge.internal.AbstractStreamTableEnvironmentImpl.toStreamInternal(AbstractStreamTableEnvironmentImpl.java:219) ~[flink-table-api-java-uber-1.17.1.jar:1.17.1] at org.apache.flink.table.api.bridge.java.internal.StreamTableEnvironmentImpl.toDataStream(StreamTableEnvironmentImpl.java:253) ~[flink-table-api-java-uber-1.17.1.jar:1.17.1] at org.apache.flink.table.api.bridge.java.internal.StreamTableEnvironmentImpl.toDataStream(StreamTableEnvironmentImpl.java:226) ~[flink-table-api-java-uber-1.17.1.jar:1.17.1] at com.appfolio.data.kafka.consumer.PropertyListings.getDataStream(PropertyListings.java:198) ~[?:?] at com.appfolio.data.kafka.consumer.KafkaIcebergConsumer.run(KafkaIcebergConsumer.java:80) ~[?:?] at com.appfolio.data.kafka.consumer.KafkaIcebergConsumer.main(KafkaIcebergConsumer.java:32) ~[?:?] at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?] at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[?:?] at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[?:?] at java.lang.reflect.Method.invoke(Unknown Source) ~[?:?] at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:355) ~[flink-dist-1.17.1.jar:1.17.1] ... 7 more Caused by: java.lang.ClassCastException: class org.codehaus.janino.CompilerFactory cannot be cast to class org.codehaus.commons.compiler.ICompilerFactory (org.codehaus.janino.CompilerFactory is in unnamed module of loader org.apache.flink.util.ChildFirstClassLoader @252ce819; org.codehaus.commons.compiler.ICompilerFactory is in unnamed module of loader 'app') at org.codehaus.commons.compiler.CompilerFactoryFactory.getCompilerFactory(CompilerFactoryFactory.java:129) ~[flink-table-runtime-1.17.1.jar:1.17.1] at org.codehaus.commons.compiler.CompilerFactoryFactory.getDefaultCompilerFactory(CompilerFactoryFactory.java:79) ~[flink-table-runtime-1.17.1.jar:1.17.1] at org.apache.calcite.rel.metadata.JaninoRelMetadataProvider.compile(JaninoRelMetadataProvider.java:161) ~[?:?] at org.apache.calcite.rel.metadata.JaninoRelMetadataProvider.generateCompileAndInstantiate(JaninoRelMetadataProvider.java:141) ~[?:?] at org.apache.calcite.rel.metadata.JaninoRelMetadataProvider.lambda$static$0(JaninoRelMetadataProvider.java:73) ~[?:?] at org.apache.flink.calcite.shaded.com.google.common.cache.CacheLoader$FunctionToCacheLoader.load(CacheLoader.java:165) ~[?:?] at org.apache.flink.calcite.shaded.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3529) ~[?:?] at org.apache.flink.calcite.shaded.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2278) ~[?:?] at org.apache.flink.calcite.shaded.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2155) ~[?:?] at org.apache.flink.calcite.shaded.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045) ~[?:?] at org.apache.flink.calcite.shaded.com.google.common.cache.LocalCache.get(LocalCache.java:3951) ~[?:?] at org.apache.flink.calcite.shaded.com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974) ~[?:?] at org.apache.flink.calcite.shaded.com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4958) ~[?:?] at org.apache.calcite.rel.metadata.JaninoRelMetadataProvider.revise(JaninoRelMetadataProvider.java:197) ~[?:?] at org.apache.calcite.rel.metadata.RelMetadataQueryBase.revise(RelMetadataQueryBase.java:118) ~[?:?] at org.apache.calcite.rel.metadata.RelMetadataQuery.getPulledUpPredicates(RelMetadataQuery.java:844) ~[?:?] at org.apache.calcite.rel.rules.ReduceExpressionsRule$ProjectReduceExpressionsRule.onMatch(ReduceExpressionsRule.java:307) ~[?:?] at org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:337) ~[?:?] at org.apache.calcite.plan.hep.HepPlanner.applyRule(HepPlanner.java:565) ~[?:?] at org.apache.calcite.plan.hep.HepPlanner.applyRules(HepPlanner.java:428) ~[?:?] at org.apache.calcite.plan.hep.HepPlanner.executeInstruction(HepPlanner.java:251) ~[?:?] at org.apache.calcite.plan.hep.HepInstruction$RuleInstance.execute(HepInstruction.java:130) ~[?:?] at org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:208) ~[?:?] at org.apache.calcite.plan.hep.HepPlanner.findBestExp(HepPlanner.java:195) ~[?:?] at org.apache.flink.table.planner.plan.optimize.program.FlinkHepProgram.optimize(FlinkHepProgram.scala:64) ~[?:?] at org.apache.flink.table.planner.plan.optimize.program.FlinkHepRuleSetProgram.optimize(FlinkHepRuleSetProgram.scala:78) ~[?:?] at org.apache.flink.table.planner.plan.optimize.program.FlinkChainedProgram.$anonfun$optimize$1(FlinkChainedProgram.scala:59) ~[?:?] at scala.collection.TraversableOnce.$anonfun$foldLeft$1(TraversableOnce.scala:156) ~[flink-scala_2.12-1.17.1.jar:1.17.1] at scala.collection.TraversableOnce.$anonfun$foldLeft$1$adapted(TraversableOnce.scala:156) ~[flink-scala_2.12-1.17.1.jar:1.17.1] at scala.collection.Iterator.foreach(Iterator.scala:937) ~[flink-scala_2.12-1.17.1.jar:1.17.1] at scala.collection.Iterator.foreach$(Iterator.scala:937) ~[flink-scala_2.12-1.17.1.jar:1.17.1] at scala.collection.AbstractIterator.foreach(Iterator.scala:1425) ~[flink-scala_2.12-1.17.1.jar:1.17.1] at scala.collection.IterableLike.foreach(IterableLike.scala:70) ~[flink-scala_2.12-1.17.1.jar:1.17.1] at scala.collection.IterableLike.foreach$(IterableLike.scala:69) ~[flink-scala_2.12-1.17.1.jar:1.17.1] at scala.collection.AbstractIterable.foreach(Iterable.scala:54) ~[flink-scala_2.12-1.17.1.jar:1.17.1] at scala.collection.TraversableOnce.foldLeft(TraversableOnce.scala:156) ~[flink-scala_2.12-1.17.1.jar:1.17.1] at scala.collection.TraversableOnce.foldLeft$(TraversableOnce.scala:154) ~[flink-scala_2.12-1.17.1.jar:1.17.1] at scala.collection.AbstractTraversable.foldLeft(Traversable.scala:104) ~[flink-scala_2.12-1.17.1.jar:1.17.1] at org.apache.flink.table.planner.plan.optimize.program.FlinkChainedProgram.optimize(FlinkChainedProgram.scala:55) ~[?:?] at org.apache.flink.table.planner.plan.optimize.StreamCommonSubGraphBasedOptimizer.optimizeTree(StreamCommonSubGraphBasedOptimizer.scala:176) ~[?:?] at org.apache.flink.table.planner.plan.optimize.StreamCommonSubGraphBasedOptimizer.doOptimize(StreamCommonSubGraphBasedOptimizer.scala:83) ~[?:?] at org.apache.flink.table.planner.plan.optimize.CommonSubGraphBasedOptimizer.optimize(CommonSubGraphBasedOptimizer.scala:87) ~[?:?] at org.apache.flink.table.planner.delegation.PlannerBase.optimize(PlannerBase.scala:329) ~[?:?] at org.apache.flink.table.planner.delegation.PlannerBase.translate(PlannerBase.scala:195) ~[?:?] at org.apache.flink.table.api.bridge.internal.AbstractStreamTableEnvironmentImpl.toStreamInternal(AbstractStreamTableEnvironmentImpl.java:224) ~[flink-table-api-java-uber-1.17.1.jar:1.17.1] at org.apache.flink.table.api.bridge.internal.AbstractStreamTableEnvironmentImpl.toStreamInternal(AbstractStreamTableEnvironmentImpl.java:219) ~[flink-table-api-java-uber-1.17.1.jar:1.17.1] at org.apache.flink.table.api.bridge.java.internal.StreamTableEnvironmentImpl.toDataStream(StreamTableEnvironmentImpl.java:253) ~[flink-table-api-java-uber-1.17.1.jar:1.17.1] at org.apache.flink.table.api.bridge.java.internal.StreamTableEnvironmentImpl.toDataStream(StreamTableEnvironmentImpl.java:226) ~[flink-table-api-java-uber-1.17.1.jar:1.17.1] at com.appfolio.data.kafka.consumer.PropertyListings.getDataStream(PropertyListings.java:198) ~[?:?] at com.appfolio.data.kafka.consumer.KafkaIcebergConsumer.run(KafkaIcebergConsumer.java:80) ~[?:?] at com.appfolio.data.kafka.consumer.KafkaIcebergConsumer.main(KafkaIcebergConsumer.java:32) ~[?:?] at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?] at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[?:?] at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[?:?] at java.lang.reflect.Method.invoke(Unknown Source) ~[?:?] at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:355) ~[flink-dist-1.17.1.jar:1.17.1] ... 7 more 2023-10-25 23:23:02,347 ERROR org.apache.flink.runtime.webmonitor.handlers.JarRunHandler [] - Exception occurred in REST handler: Could not execute application. As this image builds off eclipse-temurin:11-jre-jammy, this may be the problem -- This message was sent by Atlassian Jira (v8.20.10#820010)