Hi Galen, I think that adding support for GCS using the StreamingFileSink sounds like a very good idea to me. Looking at FLINK-11838 I believe that this effort has been abandoned. I think that you could take this ticket over if you want. Maybe you could update this ticket with your solution proposal.
I will check whether I can find a committer who could help you with this effort. Cheers, Till On Sat, Jan 30, 2021 at 7:43 PM Galen Warren <ga...@cvillewarrens.com> wrote: > Hi -- I'm wondering if you would be interested in a contribution to add a > HadoopFileSystem implementation, with associated RecoverableWriter, for > Google Cloud Storage. This would be similar to what's already in place for > S3, and it would allow writing to GCS using a StreamingFileSink. The > implementation would be similar to what's already in place for S3. > > I see there's been some work on this before (FLINK-11838 Add GCS > RecoverableWriter by Fokko · Pull Request #7915 · apache/flink (github.com > ) > <https://github.com/apache/flink/pull/7915>, but the original people > working on it have put it on hold, and the last activity was over six > months ago. > > I need this for my own purposes and I have an implementation that I'm > working on locally. I'd be interested to contribute this if you'd be > interested. Let me know if so and I'll create a Jira ticket. > > Thanks, > Galen Warren >
