Henning Schmiedehausen created FLINK-33363:
----------------------------------------------

             Summary: docker bases images can't run the java compiler
                 Key: FLINK-33363
                 URL: https://issues.apache.org/jira/browse/FLINK-33363
             Project: Flink
          Issue Type: Bug
          Components: flink-docker
    Affects Versions: 1.17.1
            Reporter: Henning Schmiedehausen


I have set up a small cluster (Job Manager + 2 Task managers) using docker 
compose. When submitting a flink job that needs Calcite planning, it crashes 
with

org.apache.flink.client.program.ProgramInvocationException: The main method 
caused an error: Unable to instantiate java compiler
        at 
org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:372)
 ~[flink-dist-1.17.1.jar:1.17.1]
        at 
org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:222)
 ~[flink-dist-1.17.1.jar:1.17.1]
        at 
org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:105) 
~[flink-dist-1.17.1.jar:1.17.1]
        at 
org.apache.flink.client.deployment.application.DetachedApplicationRunner.tryExecuteJobs(DetachedApplicationRunner.java:84)
 ~[flink-dist-1.17.1.jar:1.17.1]
        at 
org.apache.flink.client.deployment.application.DetachedApplicationRunner.run(DetachedApplicationRunner.java:70)
 ~[flink-dist-1.17.1.jar:1.17.1]
        at 
org.apache.flink.runtime.webmonitor.handlers.JarRunHandler.lambda$handleRequest$0(JarRunHandler.java:108)
 ~[flink-dist-1.17.1.jar:1.17.1]
        at java.util.concurrent.CompletableFuture$AsyncSupply.run(Unknown 
Source) [?:?]
        at java.lang.Thread.run(Unknown Source) [?:?]
Caused by: java.lang.IllegalStateException: Unable to instantiate java compiler
        at 
org.apache.calcite.rel.metadata.JaninoRelMetadataProvider.compile(JaninoRelMetadataProvider.java:163)
 ~[?:?]
        at 
org.apache.calcite.rel.metadata.JaninoRelMetadataProvider.generateCompileAndInstantiate(JaninoRelMetadataProvider.java:141)
 ~[?:?]
        at 
org.apache.calcite.rel.metadata.JaninoRelMetadataProvider.lambda$static$0(JaninoRelMetadataProvider.java:73)
 ~[?:?]
        at 
org.apache.flink.calcite.shaded.com.google.common.cache.CacheLoader$FunctionToCacheLoader.load(CacheLoader.java:165)
 ~[?:?]
        at 
org.apache.flink.calcite.shaded.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3529)
 ~[?:?]
        at 
org.apache.flink.calcite.shaded.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2278)
 ~[?:?]
        at 
org.apache.flink.calcite.shaded.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2155)
 ~[?:?]
        at 
org.apache.flink.calcite.shaded.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045)
 ~[?:?]
        at 
org.apache.flink.calcite.shaded.com.google.common.cache.LocalCache.get(LocalCache.java:3951)
 ~[?:?]
        at 
org.apache.flink.calcite.shaded.com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974)
 ~[?:?]
        at 
org.apache.flink.calcite.shaded.com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4958)
 ~[?:?]
        at 
org.apache.calcite.rel.metadata.JaninoRelMetadataProvider.revise(JaninoRelMetadataProvider.java:197)
 ~[?:?]
        at 
org.apache.calcite.rel.metadata.RelMetadataQueryBase.revise(RelMetadataQueryBase.java:118)
 ~[?:?]
        at 
org.apache.calcite.rel.metadata.RelMetadataQuery.getPulledUpPredicates(RelMetadataQuery.java:844)
 ~[?:?]
        at 
org.apache.calcite.rel.rules.ReduceExpressionsRule$ProjectReduceExpressionsRule.onMatch(ReduceExpressionsRule.java:307)
 ~[?:?]
        at 
org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:337)
 ~[?:?]
        at 
org.apache.calcite.plan.hep.HepPlanner.applyRule(HepPlanner.java:565) ~[?:?]
        at 
org.apache.calcite.plan.hep.HepPlanner.applyRules(HepPlanner.java:428) ~[?:?]
        at 
org.apache.calcite.plan.hep.HepPlanner.executeInstruction(HepPlanner.java:251) 
~[?:?]
        at 
org.apache.calcite.plan.hep.HepInstruction$RuleInstance.execute(HepInstruction.java:130)
 ~[?:?]
        at 
org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:208) 
~[?:?]
        at 
org.apache.calcite.plan.hep.HepPlanner.findBestExp(HepPlanner.java:195) ~[?:?]
        at 
org.apache.flink.table.planner.plan.optimize.program.FlinkHepProgram.optimize(FlinkHepProgram.scala:64)
 ~[?:?]
        at 
org.apache.flink.table.planner.plan.optimize.program.FlinkHepRuleSetProgram.optimize(FlinkHepRuleSetProgram.scala:78)
 ~[?:?]
        at 
org.apache.flink.table.planner.plan.optimize.program.FlinkChainedProgram.$anonfun$optimize$1(FlinkChainedProgram.scala:59)
 ~[?:?]
        at 
scala.collection.TraversableOnce.$anonfun$foldLeft$1(TraversableOnce.scala:156) 
~[flink-scala_2.12-1.17.1.jar:1.17.1]
        at 
scala.collection.TraversableOnce.$anonfun$foldLeft$1$adapted(TraversableOnce.scala:156)
 ~[flink-scala_2.12-1.17.1.jar:1.17.1]
        at scala.collection.Iterator.foreach(Iterator.scala:937) 
~[flink-scala_2.12-1.17.1.jar:1.17.1]
        at scala.collection.Iterator.foreach$(Iterator.scala:937) 
~[flink-scala_2.12-1.17.1.jar:1.17.1]
        at scala.collection.AbstractIterator.foreach(Iterator.scala:1425) 
~[flink-scala_2.12-1.17.1.jar:1.17.1]
        at scala.collection.IterableLike.foreach(IterableLike.scala:70) 
~[flink-scala_2.12-1.17.1.jar:1.17.1]
        at scala.collection.IterableLike.foreach$(IterableLike.scala:69) 
~[flink-scala_2.12-1.17.1.jar:1.17.1]
        at scala.collection.AbstractIterable.foreach(Iterable.scala:54) 
~[flink-scala_2.12-1.17.1.jar:1.17.1]
        at scala.collection.TraversableOnce.foldLeft(TraversableOnce.scala:156) 
~[flink-scala_2.12-1.17.1.jar:1.17.1]
        at 
scala.collection.TraversableOnce.foldLeft$(TraversableOnce.scala:154) 
~[flink-scala_2.12-1.17.1.jar:1.17.1]
        at scala.collection.AbstractTraversable.foldLeft(Traversable.scala:104) 
~[flink-scala_2.12-1.17.1.jar:1.17.1]
        at 
org.apache.flink.table.planner.plan.optimize.program.FlinkChainedProgram.optimize(FlinkChainedProgram.scala:55)
 ~[?:?]
        at 
org.apache.flink.table.planner.plan.optimize.StreamCommonSubGraphBasedOptimizer.optimizeTree(StreamCommonSubGraphBasedOptimizer.scala:176)
 ~[?:?]
        at 
org.apache.flink.table.planner.plan.optimize.StreamCommonSubGraphBasedOptimizer.doOptimize(StreamCommonSubGraphBasedOptimizer.scala:83)
 ~[?:?]
        at 
org.apache.flink.table.planner.plan.optimize.CommonSubGraphBasedOptimizer.optimize(CommonSubGraphBasedOptimizer.scala:87)
 ~[?:?]
        at 
org.apache.flink.table.planner.delegation.PlannerBase.optimize(PlannerBase.scala:329)
 ~[?:?]
        at 
org.apache.flink.table.planner.delegation.PlannerBase.translate(PlannerBase.scala:195)
 ~[?:?]
        at 
org.apache.flink.table.api.bridge.internal.AbstractStreamTableEnvironmentImpl.toStreamInternal(AbstractStreamTableEnvironmentImpl.java:224)
 ~[flink-table-api-java-uber-1.17.1.jar:1.17.1]
        at 
org.apache.flink.table.api.bridge.internal.AbstractStreamTableEnvironmentImpl.toStreamInternal(AbstractStreamTableEnvironmentImpl.java:219)
 ~[flink-table-api-java-uber-1.17.1.jar:1.17.1]
        at 
org.apache.flink.table.api.bridge.java.internal.StreamTableEnvironmentImpl.toDataStream(StreamTableEnvironmentImpl.java:253)
 ~[flink-table-api-java-uber-1.17.1.jar:1.17.1]
        at 
org.apache.flink.table.api.bridge.java.internal.StreamTableEnvironmentImpl.toDataStream(StreamTableEnvironmentImpl.java:226)
 ~[flink-table-api-java-uber-1.17.1.jar:1.17.1]
        at 
com.appfolio.data.kafka.consumer.PropertyListings.getDataStream(PropertyListings.java:198)
 ~[?:?]
        at 
com.appfolio.data.kafka.consumer.KafkaIcebergConsumer.run(KafkaIcebergConsumer.java:80)
 ~[?:?]
        at 
com.appfolio.data.kafka.consumer.KafkaIcebergConsumer.main(KafkaIcebergConsumer.java:32)
 ~[?:?]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
~[?:?]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) 
~[?:?]
        at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown 
Source) ~[?:?]
        at java.lang.reflect.Method.invoke(Unknown Source) ~[?:?]
        at 
org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:355)
 ~[flink-dist-1.17.1.jar:1.17.1]
        ... 7 more
Caused by: java.lang.ClassCastException: class 
org.codehaus.janino.CompilerFactory cannot be cast to class 
org.codehaus.commons.compiler.ICompilerFactory 
(org.codehaus.janino.CompilerFactory is in unnamed module of loader 
org.apache.flink.util.ChildFirstClassLoader @252ce819; 
org.codehaus.commons.compiler.ICompilerFactory is in unnamed module of loader 
'app')
        at 
org.codehaus.commons.compiler.CompilerFactoryFactory.getCompilerFactory(CompilerFactoryFactory.java:129)
 ~[flink-table-runtime-1.17.1.jar:1.17.1]
        at 
org.codehaus.commons.compiler.CompilerFactoryFactory.getDefaultCompilerFactory(CompilerFactoryFactory.java:79)
 ~[flink-table-runtime-1.17.1.jar:1.17.1]
        at 
org.apache.calcite.rel.metadata.JaninoRelMetadataProvider.compile(JaninoRelMetadataProvider.java:161)
 ~[?:?]
        at 
org.apache.calcite.rel.metadata.JaninoRelMetadataProvider.generateCompileAndInstantiate(JaninoRelMetadataProvider.java:141)
 ~[?:?]
        at 
org.apache.calcite.rel.metadata.JaninoRelMetadataProvider.lambda$static$0(JaninoRelMetadataProvider.java:73)
 ~[?:?]
        at 
org.apache.flink.calcite.shaded.com.google.common.cache.CacheLoader$FunctionToCacheLoader.load(CacheLoader.java:165)
 ~[?:?]
        at 
org.apache.flink.calcite.shaded.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3529)
 ~[?:?]
        at 
org.apache.flink.calcite.shaded.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2278)
 ~[?:?]
        at 
org.apache.flink.calcite.shaded.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2155)
 ~[?:?]
        at 
org.apache.flink.calcite.shaded.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045)
 ~[?:?]
        at 
org.apache.flink.calcite.shaded.com.google.common.cache.LocalCache.get(LocalCache.java:3951)
 ~[?:?]
        at 
org.apache.flink.calcite.shaded.com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974)
 ~[?:?]
        at 
org.apache.flink.calcite.shaded.com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4958)
 ~[?:?]
        at 
org.apache.calcite.rel.metadata.JaninoRelMetadataProvider.revise(JaninoRelMetadataProvider.java:197)
 ~[?:?]
        at 
org.apache.calcite.rel.metadata.RelMetadataQueryBase.revise(RelMetadataQueryBase.java:118)
 ~[?:?]
        at 
org.apache.calcite.rel.metadata.RelMetadataQuery.getPulledUpPredicates(RelMetadataQuery.java:844)
 ~[?:?]
        at 
org.apache.calcite.rel.rules.ReduceExpressionsRule$ProjectReduceExpressionsRule.onMatch(ReduceExpressionsRule.java:307)
 ~[?:?]
        at 
org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:337)
 ~[?:?]
        at 
org.apache.calcite.plan.hep.HepPlanner.applyRule(HepPlanner.java:565) ~[?:?]
        at 
org.apache.calcite.plan.hep.HepPlanner.applyRules(HepPlanner.java:428) ~[?:?]
        at 
org.apache.calcite.plan.hep.HepPlanner.executeInstruction(HepPlanner.java:251) 
~[?:?]
        at 
org.apache.calcite.plan.hep.HepInstruction$RuleInstance.execute(HepInstruction.java:130)
 ~[?:?]
        at 
org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:208) 
~[?:?]
        at 
org.apache.calcite.plan.hep.HepPlanner.findBestExp(HepPlanner.java:195) ~[?:?]
        at 
org.apache.flink.table.planner.plan.optimize.program.FlinkHepProgram.optimize(FlinkHepProgram.scala:64)
 ~[?:?]
        at 
org.apache.flink.table.planner.plan.optimize.program.FlinkHepRuleSetProgram.optimize(FlinkHepRuleSetProgram.scala:78)
 ~[?:?]
        at 
org.apache.flink.table.planner.plan.optimize.program.FlinkChainedProgram.$anonfun$optimize$1(FlinkChainedProgram.scala:59)
 ~[?:?]
        at 
scala.collection.TraversableOnce.$anonfun$foldLeft$1(TraversableOnce.scala:156) 
~[flink-scala_2.12-1.17.1.jar:1.17.1]
        at 
scala.collection.TraversableOnce.$anonfun$foldLeft$1$adapted(TraversableOnce.scala:156)
 ~[flink-scala_2.12-1.17.1.jar:1.17.1]
        at scala.collection.Iterator.foreach(Iterator.scala:937) 
~[flink-scala_2.12-1.17.1.jar:1.17.1]
        at scala.collection.Iterator.foreach$(Iterator.scala:937) 
~[flink-scala_2.12-1.17.1.jar:1.17.1]
        at scala.collection.AbstractIterator.foreach(Iterator.scala:1425) 
~[flink-scala_2.12-1.17.1.jar:1.17.1]
        at scala.collection.IterableLike.foreach(IterableLike.scala:70) 
~[flink-scala_2.12-1.17.1.jar:1.17.1]
        at scala.collection.IterableLike.foreach$(IterableLike.scala:69) 
~[flink-scala_2.12-1.17.1.jar:1.17.1]
        at scala.collection.AbstractIterable.foreach(Iterable.scala:54) 
~[flink-scala_2.12-1.17.1.jar:1.17.1]
        at scala.collection.TraversableOnce.foldLeft(TraversableOnce.scala:156) 
~[flink-scala_2.12-1.17.1.jar:1.17.1]
        at 
scala.collection.TraversableOnce.foldLeft$(TraversableOnce.scala:154) 
~[flink-scala_2.12-1.17.1.jar:1.17.1]
        at scala.collection.AbstractTraversable.foldLeft(Traversable.scala:104) 
~[flink-scala_2.12-1.17.1.jar:1.17.1]
        at 
org.apache.flink.table.planner.plan.optimize.program.FlinkChainedProgram.optimize(FlinkChainedProgram.scala:55)
 ~[?:?]
        at 
org.apache.flink.table.planner.plan.optimize.StreamCommonSubGraphBasedOptimizer.optimizeTree(StreamCommonSubGraphBasedOptimizer.scala:176)
 ~[?:?]
        at 
org.apache.flink.table.planner.plan.optimize.StreamCommonSubGraphBasedOptimizer.doOptimize(StreamCommonSubGraphBasedOptimizer.scala:83)
 ~[?:?]
        at 
org.apache.flink.table.planner.plan.optimize.CommonSubGraphBasedOptimizer.optimize(CommonSubGraphBasedOptimizer.scala:87)
 ~[?:?]
        at 
org.apache.flink.table.planner.delegation.PlannerBase.optimize(PlannerBase.scala:329)
 ~[?:?]
        at 
org.apache.flink.table.planner.delegation.PlannerBase.translate(PlannerBase.scala:195)
 ~[?:?]
        at 
org.apache.flink.table.api.bridge.internal.AbstractStreamTableEnvironmentImpl.toStreamInternal(AbstractStreamTableEnvironmentImpl.java:224)
 ~[flink-table-api-java-uber-1.17.1.jar:1.17.1]
        at 
org.apache.flink.table.api.bridge.internal.AbstractStreamTableEnvironmentImpl.toStreamInternal(AbstractStreamTableEnvironmentImpl.java:219)
 ~[flink-table-api-java-uber-1.17.1.jar:1.17.1]
        at 
org.apache.flink.table.api.bridge.java.internal.StreamTableEnvironmentImpl.toDataStream(StreamTableEnvironmentImpl.java:253)
 ~[flink-table-api-java-uber-1.17.1.jar:1.17.1]
        at 
org.apache.flink.table.api.bridge.java.internal.StreamTableEnvironmentImpl.toDataStream(StreamTableEnvironmentImpl.java:226)
 ~[flink-table-api-java-uber-1.17.1.jar:1.17.1]
        at 
com.appfolio.data.kafka.consumer.PropertyListings.getDataStream(PropertyListings.java:198)
 ~[?:?]
        at 
com.appfolio.data.kafka.consumer.KafkaIcebergConsumer.run(KafkaIcebergConsumer.java:80)
 ~[?:?]
        at 
com.appfolio.data.kafka.consumer.KafkaIcebergConsumer.main(KafkaIcebergConsumer.java:32)
 ~[?:?]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
~[?:?]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) 
~[?:?]
        at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown 
Source) ~[?:?]
        at java.lang.reflect.Method.invoke(Unknown Source) ~[?:?]
        at 
org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:355)
 ~[flink-dist-1.17.1.jar:1.17.1]
        ... 7 more
2023-10-25 23:23:02,347 ERROR 
org.apache.flink.runtime.webmonitor.handlers.JarRunHandler   [] - Exception 
occurred in REST handler: Could not execute application.

As this image builds off eclipse-temurin:11-jre-jammy, this may be the problem



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Reply via email to