xintongsong commented on a change in pull request #18430: URL: https://github.com/apache/flink/pull/18430#discussion_r791330799
########## File path: docs/content/docs/deployment/filesystems/gcs.md ########## @@ -50,57 +48,66 @@ env.getCheckpointConfig().setCheckpointStorage("gs://<bucket>/<endpoint>"); ``` -### Libraries +Note that these examples are *not* exhaustive and you can use GCS in other places as well, including your [high availability setup]({{< ref "docs/deployment/ha/overview" >}}) or the [EmbeddedRocksDBStateBackend]({{< ref "docs/ops/state/state_backends" >}}#the-rocksdbstatebackend); everywhere that Flink expects a FileSystem URI. + +### GCS File System plugin + +Flink provides the `flink-gs-fs-hadoop` file system to write to GCS. +This implementation is self-contained with no dependency footprint, so there is no need to add Hadoop to the classpath to use it. -You must include the following jars in Flink's `lib` directory to connect Flink with gcs: +`flink-gs-fs-hadoop` registers a `FileSystem` wrapper for URIs with the *gs://* scheme. It uses Google's [gcs-connector](https://mvnrepository.com/artifact/com.google.cloud.bigdataoss/gcs-connector) Hadoop library to access GCS. It also uses Google's [google-cloud-storage](https://mvnrepository.com/artifact/com.google.cloud/google-cloud-storage) library to provide `RecoverableWriter` support. -```xml -<dependency> - <groupId>org.apache.flink</groupId> - <artifactId>flink-shaded-hadoop2-uber</artifactId> - <version>${flink.shared_hadoop_latest_version}</version> -</dependency> +This file system supports being used in [file sinks]({{< ref "docs/content/docs/connectors/datastream/filesystem.md" >}}). Review comment: My bad provided the wrong path in the previous comment. You may build and check the doc locally, by running `docs/build_docs.sh`. ```suggestion This file system supports being used in [file sinks]({{< ref "docs/connectors/datastream/filesystem" >}}). ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org