Re: S3 StreamingFileSink never completes multipart uploads

2019-01-07 Thread Kostas Kloudas
No problem and thanks for looking into the problem! I also commented on the JIRA. Cheers, Kostas On Mon, Jan 7, 2019, 18:13 Addison Higham The not seeing the final post from the logs was what initially clued me > into the same issue, so I figured it was the same issue, but I should have > worde

Re: S3 StreamingFileSink never completes multipart uploads

2019-01-07 Thread Addison Higham
The not seeing the final post from the logs was what initially clued me into the same issue, so I figured it was the same issue, but I should have worded it better that it is one possible explanation. Sorry for the confusion! Addison On Fri, Jan 4, 2019 at 11:24 PM Kostas Kloudas wrote: >

Re: S3 StreamingFileSink never completes multipart uploads

2019-01-04 Thread Kostas Kloudas
Hi Addison, >From the information that Nick provides, how can you be sure that the root cause is the same? Cheers, Kostas On Fri, Jan 4, 2019, 22:10 Addison Higham Hi Nick, > > This is a known issue with 1.7.0, I have an issue opened up here: > https://issues.apache.org/jira/browse/FLINK-11187

Re: S3 StreamingFileSink never completes multipart uploads

2019-01-04 Thread Addison Higham
Hi Nick, This is a known issue with 1.7.0, I have an issue opened up here: https://issues.apache.org/jira/browse/FLINK-11187 On Wed, Jan 2, 2019 at 5:00 PM Martin, Nick wrote: > I’m running on Flink 1.7.0 trying to use the StreamingFileSink with an S3A > URI. What I’m seeing is that whenever

S3 StreamingFileSink never completes multipart uploads

2019-01-02 Thread Martin, Nick
I'm running on Flink 1.7.0 trying to use the StreamingFileSink with an S3A URI. What I'm seeing is that whenever the RollingPolicy determines that it's time to roll to a new part file, the whole Sink just hangs, and the in progress MultiPart Upload never gets completed. I've looked at the traffi