[
https://issues.apache.org/jira/browse/FLINK-13044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chesnay Schepler closed FLINK-13044.
------------------------------------
Resolution: Fixed
master: 5e5f5bec72d30397253fc1f11105ca385ca8681f
1.9: 16f519a1e7efae9210725f741c27a647ac3f3271
> Shading of AWS SDK in flink-s3-fs-hadoop results in ClassNotFoundExceptions
> ---------------------------------------------------------------------------
>
> Key: FLINK-13044
> URL: https://issues.apache.org/jira/browse/FLINK-13044
> Project: Flink
> Issue Type: Bug
> Components: FileSystems
> Affects Versions: 1.8.0
> Reporter: Sebastian J.
> Assignee: Chesnay Schepler
> Priority: Blocker
> Labels: pull-request-available
> Fix For: 1.9.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> The current build configuration shades the AWS SDK into
> {{flink-s3-fs-hadoop.jar}}
> Due to [MSHAD-156|https://issues.apache.org/jira/browse/MSHADE-156] the
> shading itself does not only add the classes to the JAR but also alters the
> string literal {{PACKAGE_PREFIXES_TO_SHADE}} in
> {{org.apache.flink.fs.s3hadoop.S3FileSystemFactory}} from {{com.amazonaws.}}
> to {{org.apache.flink.fs.s3base.shaded.com.amazonaws}} - which causes
> settings like
> {noformat}fs.s3a.aws.credentials.provider:
> com.amazonaws.auth.DefaultAWSCredentialsProviderChain{noformat}
> to not be remapped properly as the {{startsWith}} in
> {{org.apache.flink.fs.s3.common.HadoopConfigLoader.shadeClassConfig()}}
> doesn't match the beginning of the FQCN.
> This is a known issue in the Maven shade plugin (see
> [MSHAD-156|https://issues.apache.org/jira/browse/MSHADE-156]) - a possible
> workaround is named in the ticket, too.
> This issue is only visible looking into the compiled {{class}} file after the
> shading happened!
> Fixing the {{PACKAGE_PREFIXES_TO_SHADE}} value alone doesn't help as based on
> {{FLINK_SHADING_PREFIX}} the FQCN is assembled as
> {noformat}org.apache.flink.fs.s3hadoop.shaded.com.amazonaws.auth.DefaultAWSCredentialsProviderChain{noformat}
> which can't be found either;
> the class is located at
> {noformat}org.apache.flink.fs.s3base.shaded.com.amazonaws.auth.DefaultAWSCredentialsProviderChain{noformat}
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
