Hi Hang,
Once I exclude file-core from the fat jar I get this error:
I believe org.apache.flink.util.OutputTag is part of flink-core itself.
Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/flink/util/OutputTag
at java.base/java.lang.Class.forName0(Native Method)
at java.base/java.lang.Class.forName(Class.java:467)
at org.apache.hadoop.util.RunJar.run(RunJar.java:321)
at org.apache.hadoop.util.RunJar.main(RunJar.java:241)
Caused by: java.lang.ClassNotFoundException: org.apache.flink.util.OutputTag
at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:445)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:592)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:525)
... 4 more
Thanks
Sachin
On Tue, Mar 12, 2024 at 2:11 PM Sachin Mittal <sjmit...@gmail.com> wrote:
> Ok. Actually it’s version 1.18. I will try to remove flink-core from the
> fat jar.
>
> On Tue, 12 Mar 2024 at 1:51 PM, Hang Ruan <ruanhang1...@gmail.com> wrote:
>
>> Hi, Sachin.
>>
>> This error occurs when there is class conflict. There is no need to
>> package flink-core in your own jar. It is already contained in flink-dist.
>> And Flink version 1.8 is too old. It is better to update your flink
>> version.
>>
>> Best,
>> Hang
>>
>>
>>
>> Sachin Mittal <sjmit...@gmail.com> 于2024年3月12日周二 16:04写道：
>>
>>> Hi,
>>> We have installed a flink cluster version 1.8.0 on AWS EMR.
>>> However when we submit a job we get the following error:
>>>
>>> (Do note that when we submit the same job on a local instance of Flink
>>> 1.8.1 it is working fine.
>>> The fat jar we submit has all the flink dependencies from 1.8.0
>>> including the class org.apache.flink.api.common.ExecutionConfig).
>>>
>>> Caused by: java.lang.RuntimeException:
>>> org.apache.flink.runtime.client.JobInitializationException: Could not start
>>> the JobMaster.
>>> at org.apache.flink.util.ExceptionUtils.rethrow(ExceptionUtils.java:321)
>>> at
>>> org.apache.flink.util.function.FunctionUtils.lambda$uncheckedFunction$2(FunctionUtils.java:75)
>>> at
>>> java.base/java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:646)
>>> at
>>> java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483)
>>> at
>>> java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373)
>>> at
>>> java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182)
>>> at
>>> java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655)
>>> at
>>> java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622)
>>> at
>>> java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165)
>>> Caused by: org.apache.flink.runtime.client.JobInitializationException:
>>> Could not start the JobMaster.
>>> at
>>> org.apache.flink.runtime.jobmaster.DefaultJobMasterServiceProcess.lambda$new$0(DefaultJobMasterServiceProcess.java:97)
>>> at
>>> java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863)
>>> at
>>> java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841)
>>> at
>>> java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
>>> at
>>> java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1773)
>>> at
>>> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
>>> at
>>> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
>>> at java.base/java.lang.Thread.run(Thread.java:840)
>>> Caused by: java.util.concurrent.CompletionException:
>>> java.lang.RuntimeException: java.lang.ClassNotFoundException:
>>> org.apache.flink.api.common.ExecutionConfig
>>> at
>>> java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:315)
>>> at
>>> java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:320)
>>> at
>>> java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1770)
>>> ... 3 more
>>> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
>>> org.apache.flink.api.common.ExecutionConfig
>>> at org.apache.flink.util.ExceptionUtils.rethrow(ExceptionUtils.java:321)
>>> at
>>> org.apache.flink.util.function.FunctionUtils.lambda$uncheckedSupplier$4(FunctionUtils.java:114)
>>> at
>>> java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1768)
>>> ... 3 more
>>> Caused by: java.lang.ClassNotFoundException:
>>> org.apache.flink.api.common.ExecutionConfig
>>> at
>>> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
>>> at
>>> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
>>> at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:525)
>>> at java.base/java.lang.Class.forName0(Native Method)
>>> at java.base/java.lang.Class.forName(Class.java:467)
>>>
>>>
