[
https://issues.apache.org/jira/browse/FLINK-9441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aljoscha Krettek updated FLINK-9441:
------------------------------------
Fix Version/s: 1.7.0
> Hadoop Required Dependency List Not Clear
> -----------------------------------------
>
> Key: FLINK-9441
> URL: https://issues.apache.org/jira/browse/FLINK-9441
> Project: Flink
> Issue Type: Bug
> Components: Build System, Configuration, Documentation
> Affects Versions: 1.5.0, 1.4.2, 1.6.0
> Reporter: Razvan
> Priority: Blocker
> Fix For: 1.7.0
>
>
> To be able to use Apache Flink with S3, a few libraries from Hadoop
> distribution are required to be added to lib/ folder.
> This list is partially documented in
> [https://ci.apache.org/projects/flink/flink-docs-master/ops/deployment/aws.html]
> (Provide S3 FileSystem Dependency). But it refers to Hadoop 2.7 whereas
> Flink supports 2.8 also.
> How to compose the dependecy list for Hadoop 2.8?
> Is it possible to bundle the dependencies in a separate archive that users
> can download?
>
> UPDATE:
> Downloaded Apache Flink 1.4.2 for Hadoop 2.7 and it seems it was compiled
> with Hadoop 2.8. I get the error here:
>
> {{"java.lang.NumberFormatException: For input string: "100M" at
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.lang.Long.parseLong(Long.java:589) at
> java.lang.Long.parseLong(Long.java:631) at
> org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1319) at
> org.apache.hadoop.fs.s3a.S3AFileSystem.initialize(S3AFileSystem.java:248) at
> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2811) at
> org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:100) }}
> {{..."}}
> [https://stackoverflow.com/questions/48149929/hive-1-2-metastore-service-doesnt-start-after-configuring-it-to-s3-storage-inst?rq=1]
>
> So I cannot start the cluster.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
