github-actions[bot] commented on code in PR #68540:
URL: https://github.com/apache/doris/pull/68540#discussion_r4236490418
##########
fe/fe-connector/fe-connector-iceberg/src/main/java/org/apache/doris/connector/iceberg/IcebergCatalogFactory.java:
##########
@@ -419,6 +422,36 @@ public static Map<String, String>
buildCatalogProperties(IcebergCatalogPropertie
// s3tables: bespoke instantiation. Preserve the skeleton's
base+impl routing.
break;
}
+ chosenS3.ifPresent(storage ->
storage.toBackendProperties().ifPresent(backend -> {
+ Optional<GcsAuth> auth = GcsAuthResolver.resolve(backend.toMap());
+ if (auth.filter(GcsAuth::isAnonymous).isPresent()) {
+ opts.put(AwsClientProperties.CLIENT_CREDENTIALS_PROVIDER,
+
"software.amazon.awssdk.auth.credentials.AnonymousCredentialsProvider");
+ }
+ auth.flatMap(GcsAuth::getNativeCredential).ifPresent(credential ->
{
+ // HadoopCatalog resolves its namespace filesystem
independently of S3FileIO.
+ // Native GCS credentials configure fs.gs.*, so compatibility
warehouse schemes
+ // must select that same filesystem before HadoopCatalog
initializes.
+ String warehouse =
opts.get(CatalogProperties.WAREHOUSE_LOCATION);
+ if (IcebergCatalogProperties.TYPE_HADOOP.equals(flavor) &&
warehouse != null) {
+ if (warehouse.regionMatches(true, 0, "s3://", 0, 5)) {
+ opts.put(CatalogProperties.WAREHOUSE_LOCATION, "gs://"
+ warehouse.substring(5));
+ } else if (warehouse.regionMatches(true, 0, "s3a://", 0,
6)) {
+ opts.put(CatalogProperties.WAREHOUSE_LOCATION, "gs://"
+ warehouse.substring(6));
+ }
+ }
+ putS3FileIODialect(opts, storage);
+ opts.put("provider", "GCP");
+ opts.put(GcpCredential.CREDENTIAL_PROVIDER_TYPE,
credential.getCredentialProviderType().name());
+ putIfNotBlank(opts,
GcpCredential.IMPERSONATION_SERVICE_ACCOUNT,
+ credential.getImpersonationServiceAccount());
+ // Keep explicit FileIO choices. By default, S3 aliases use
the GCP client factory,
+ // while gs/HDFS paths use HadoopFileIO with the native
fs.gs.* configuration
+ // (the plugin does not bundle iceberg-gcp's GCSFileIO).
+ opts.putIfAbsent(CatalogProperties.FILE_IO_IMPL,
"org.apache.iceberg.io.ResolvingFileIO");
Review Comment:
[P2] Pass the native GCS Hadoop configuration to Iceberg Glue catalogs. This
new `ResolvingFileIO` routes `gs://` through `HadoopFileIO` when `iceberg-gcp`
is absent, but `IcebergConnector.createCatalog` builds `storageHadoopConfig`
and then passes `conf=null` for Glue. [Iceberg 1.11.0's
resolver](https://github.com/apache/iceberg/blob/apache-iceberg-1.11.0/core/src/main/java/org/apache/iceberg/io/ResolvingFileIO.java)
forwards that null configuration to the fallback, so a private `gs://` table
using Compute Engine credentials or service-account impersonation lacks
`fs.gs.impl` and `fs.gs.auth.*` and cannot read metadata with the selected
identity. Pass the storage configuration for Glue and cover a private GCS Glue
table.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
