Filesystems are supposed to be used as plugins (by putting the jars
under plugins/ instead of lib/), in which case they are loaded
separately from other classes, specifically user-code.
https://ci.apache.org/projects/flink/flink-docs-release-1.11/ops/plugins.html
On 14/08/2020 20:25, Satish Saley wrote:
Hi team,
Was there a reason for not shading hadoop-common
https://github.com/apache/flink/commit/e1e7d7f7ecc080c850a264021bf1b20e3d27d373#diff-e7b798a682ee84ab804988165e99761cR38-R44
? This is leaking lots of classes such as guava and causing issues in
our flink application.
I see that hadoop-common classes were shaded in earlier versions
https://mvnrepository.com/artifact/org.apache.flink/flink-s3-fs-hadoop/1.9.0
Stacktrace :
Caused by: java.lang.NoSuchMethodError:
com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;CLjava/lang/Object;)V
at io.grpc.Metadata$Key.validateName(Metadata.java:742)
at io.grpc.Metadata$Key.<init>(Metadata.java:750)
at io.grpc.Metadata$Key.<init>(Metadata.java:668)
at io.grpc.Metadata$AsciiKey.<init>(Metadata.java:959)
at io.grpc.Metadata$AsciiKey.<init>(Metadata.java:954)
at io.grpc.Metadata$Key.of(Metadata.java:705)
at io.grpc.Metadata$Key.of(Metadata.java:701)
at io.grpc.internal.GrpcUtil.<clinit>(GrpcUtil.java:80)
at
io.grpc.internal.AbstractManagedChannelImplBuilder.<clinit>(AbstractManagedChannelImplBuilder.java:90)
at
io.grpc.netty.shaded.io.grpc.netty.NettyChannelProvider.builderForTarget(NettyChannelProvider.java:42)
at
io.grpc.netty.shaded.io.grpc.netty.NettyChannelProvider.builderForTarget(NettyChannelProvider.java:23)
at io.grpc.ManagedChannelBuilder.forTarget(ManagedChannelBuilder.java:76)
