Hi Bobby, You can already use it for writing in Flink 1.14 [1] as long as you're aware of the implications in case of a failure / need to recover.
Best regards, Martijn [1] https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/filesystems/gcs/ On Mon, 14 Mar 2022 at 16:00, Bobby Richard <bobby.rich...@broadcom.com> wrote: > Thanks Martijn, are there any alternatives to write to GCS using Flink > 1.14? Recoverability isn't important for my current use case. > > On Mon, Mar 14, 2022 at 10:58 AM Martijn Visser <martijnvis...@apache.org> > wrote: > >> Hi Bobby, >> >> That's because Flink 1.14 currently doesn't support the RecoverableWriter >> for GCS. This will be supported as of Flink 1.15, which you can see in the >> relevant Flink Jira ticket [1]. For more details on why RecoverableWriter >> is important, the JavaDoc [2] is probably the best source of information. >> >> Best regards, >> >> Martijn Visser >> https://twitter.com/MartijnVisser82 >> >> [1] https://issues.apache.org/jira/browse/FLINK-11838 >> [2] >> https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/core/fs/RecoverableWriter.html >> >> >> On Mon, 14 Mar 2022 at 15:21, Bobby Richard <bobby.rich...@broadcom.com> >> wrote: >> >>> I am receiving the following exception when attempting to write to GCS >>> with the FileSink in FLink 14.3. Using flink hadoop shaded 2.8.3-10.0 >>> and gcs connector hadoop2-2.1.1. >>> >>> >>> java.lang.UnsupportedOperationException: Recoverable writers on Hadoop >>> are only supported for HDFS >>> >>> I am able to write checkpoints and savepoints to GCS successfully. >>> >>> This electronic communication and the information and any files >>> transmitted with it, or attached to it, are confidential and are intended >>> solely for the use of the individual or entity to whom it is addressed and >>> may contain information that is confidential, legally privileged, protected >>> by privacy laws, or otherwise restricted from disclosure to anyone else. If >>> you are not the intended recipient or the person responsible for delivering >>> the e-mail to the intended recipient, you are hereby notified that any use, >>> copying, distributing, dissemination, forwarding, printing, or copying of >>> this e-mail is strictly prohibited. If you received this e-mail in error, >>> please return the e-mail to the sender, delete it from your computer, and >>> destroy any printed copy of it. >> >> > This electronic communication and the information and any files > transmitted with it, or attached to it, are confidential and are intended > solely for the use of the individual or entity to whom it is addressed and > may contain information that is confidential, legally privileged, protected > by privacy laws, or otherwise restricted from disclosure to anyone else. If > you are not the intended recipient or the person responsible for delivering > the e-mail to the intended recipient, you are hereby notified that any use, > copying, distributing, dissemination, forwarding, printing, or copying of > this e-mail is strictly prohibited. If you received this e-mail in error, > please return the e-mail to the sender, delete it from your computer, and > destroy any printed copy of it.
