Github user etiennecarriere commented on the issue:
https://github.com/apache/flink/pull/6149
Hi @pnowojski,
Unfortunately, this patch is not working because :
* The implementation of FileSystem is using a temporary file to store it
** s3 : it is mandatory to know the file of the file before sending it
(Content-Length is needed and Chunked mode is not supported)
https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/model/PutObjectRequest.html
*** flink-s3-fs-hadoop :
https://github.com/Aloisius/hadoop-s3a/blob/master/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java#L808
*** flink-s3-fs-presto :
https://github.com/prestodb/presto/blob/master/presto-hive/src/main/java/com/facebook/presto/hive/s3/PrestoS3FileSystem.java#L991
* Swift : even if possible to upload file via chunked the hadoop
implementation use tempFile
** flink-swift-fs-hadoop :
https://github.com/c9n/hadoop/blob/master/hadoop-tools/hadoop-openstack/src/main/java/org/apache/hadoop/fs/swift/snative/SwiftNativeOutputStream.java#L78
so I propose to close the PR as it is not the right level to handle that.
---
