[
https://issues.apache.org/jira/browse/FLINK-5706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16202425#comment-16202425
]
ASF GitHub Bot commented on FLINK-5706:
---------------------------------------
Github user steveloughran commented on the issue:
https://github.com/apache/flink/pull/4818
1. I hope you pick up Hadoop 2.8.1 for this, as it's got a lot of the
optimisations
1. And equally importantly: a later SDK
1. Though not one of the more recent 1.11 SDKs, where support is yet to
ship. That's a big single shaded aws-sdk JAR so things like joda-time, jackson,
guava, etc, don't cause problems, just the detail of 50+MB more of .jar on the
CP.
Test wise, see how well your client works with a v4 endpoint like
frankfurt, as there you also have to change the endpoint used.
Otherwise, nothing obvious I'd flag up as dangerous
> Implement Flink's own S3 filesystem
> -----------------------------------
>
> Key: FLINK-5706
> URL: https://issues.apache.org/jira/browse/FLINK-5706
> Project: Flink
> Issue Type: New Feature
> Components: filesystem-connector
> Reporter: Stephan Ewen
> Assignee: Stephan Ewen
>
> As part of the effort to make Flink completely independent from Hadoop, Flink
> needs its own S3 filesystem implementation. Currently Flink relies on
> Hadoop's S3a and S3n file systems.
> An own S3 file system can be implemented using the AWS SDK. As the basis of
> the implementation, the Hadoop File System can be used (Apache Licensed,
> should be okay to reuse some code as long as we do a proper attribution).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
