I had been excluding all transitive dependencies from the lib dir; it seems to
be working when I added the following deps:
<dependency>
<groupId>commons-configuration</groupId>
<artifactId>commons-configuration</artifactId>
<version>1.7</version>
</dependency>
<dependency>
<groupId>commons-lang</groupId>
<artifactId>commons-lang</artifactId>
<version>2.6</version>
</dependency>
--
Jared Stehler
Chief Architect - Intellify Learning
o: 617.701.6330 x703
> On Dec 12, 2017, at 2:10 PM, Chesnay Schepler <ches...@apache.org> wrote:
>
> Could you look into the flink-shaded-hadoop jar to check whether the missing
> class is actually contained?
>
> Where did the flink-shaded-hadoop jar come from? I'm asking because when
> building flink-dist from source the jar is called
> flink-shaded-hadoop2-uber-1.4.0.jar, which does indeed contain the jar. (the
> uber jar is created by building flink-shaded-hadoop2-uber)
>
> On 12.12.2017 19:28, Jared Stehler wrote:
>> After upgrading to flink 1.4.0 using the hadoop-free build option, I’m
>> seeing the following error on startup in the app master:
>>
>> 2017-12-12 18:23:15.473 [main] ERROR
>> o.a.f.m.r.clusterframework.MesosApplicationMasterRunner - Mesos JobManager
>> initialization failed
>>
>> <https://internal.dev3.master.us-west-2.prod.aws.intellify.io/singularity/task/flink-appmaster-1513102980-1513102985422-1-10.80.53.70-us_west_2b/tail/flink-appmaster-1513102980-1513102985422-1-10.80.53.70-us_west_2b/tail_of_finished_service.log?offset=34208>java.lang.NoClassDefFoundError:
>> Could not initialize class org.apache.hadoop.security.UserGroupInformation
>>
>> <https://internal.dev3.master.us-west-2.prod.aws.intellify.io/singularity/task/flink-appmaster-1513102980-1513102985422-1-10.80.53.70-us_west_2b/tail/flink-appmaster-1513102980-1513102985422-1-10.80.53.70-us_west_2b/tail_of_finished_service.log?offset=34315>
>> at
>> org.apache.flink.runtime.clusterframework.overlays.HadoopUserOverlay$Builder.fromEnvironment(HadoopUserOverlay.java:74)
>>
>> <https://internal.dev3.master.us-west-2.prod.aws.intellify.io/singularity/task/flink-appmaster-1513102980-1513102985422-1-10.80.53.70-us_west_2b/tail/flink-appmaster-1513102980-1513102985422-1-10.80.53.70-us_west_2b/tail_of_finished_service.log?offset=34439>
>> at
>> org.apache.flink.mesos.entrypoint.MesosEntrypointUtils.applyOverlays(MesosEntrypointUtils.java:145)
>> Looking at the code, it appears that the HadoopUserOverlay always tries to
>> init the UserGroupInformation class, and is failing. Same error with or
>> without the flink-shaded-hadoop2 library included.
>>
>> This is my lib dir:
>>
>> flink-appmaster-1.0-SNAPSHOT.jar flink-s3-fs-presto-1.4.0.jar
>> jul-to-slf4j-1.7.25.jar sentry-1.5.3.jar
>> flink-dist_2.11-1.4.0.jar flink-shaded-hadoop2-1.4.0.jar
>> log4j-over-slf4j-1.7.25.jar sentry-logback-1.5.3.jar
>> flink-metrics-prometheus-1.4.0.jar jackson-core-2.8.10.jar
>> logback-classic-1.1.11.jar
>> flink-python_2.11-1.4.0.jar jcl-over-slf4j-1.7.25.jar
>> logback-core-1.1.11.jar
>>
>>
>> --
>> Jared Stehler
>> Chief Architect - Intellify Learning
>> o: 617.701.6330 x703
>>
>>
>>
>
