Bert,
Also set the GDAL_DISABLE_READDIR_ON_OPEN=EMPTY_DIR config option,
otherwise the generic open mechanism of GDAL tries to list the content
of the VNP02IMG/ directory and it seems there are tons of files there
When doing that, I get a result within a few seconds
Even
Le 20/07/2023 à 09:59, b.coerver--- via gdal-dev a écrit :
Hello,
I'm trying to access data from NASA's Earthdata S3 buckets, but I get
a `"<filename> does not exist in the file system, and is not
recognized as a supported dataset name."` error after waiting a long
time (± 50 minutes, the process is downloading some data the whole
time) doing the following:
from osgeo import gdal
gdal_config_options = {
"AWS_ACCESS_KEY_ID": creds["accessKeyId"],
"AWS_SESSION_TOKEN": creds["sessionToken"],
"AWS_SECRET_ACCESS_KEY": creds["secretAccessKey"],
"AWS_REGION": "us-west-2",
}
url =
"/vsis3/prod-lads/VNP02IMG/VNP02IMG.A2023193.1942.002.2023194025636.nc"
for k, v in gdal_config_options.items():
gdal.SetConfigOption(k, v)
out =gdal.Info(url)
The `creds` variable is a dictionary with temporary credential
information that I get from
[here](https://data.laadsdaac.earthdatacloud.nasa.gov/s3credentials
<https://data.laadsdaac.earthdatacloud.nasa.gov/s3credentials>), you
need a free account to get them.
When I introduce an error in one of the keys/tokens (e.g.
`"AWS_ACCESS_KEY_ID": creds["accessKeyId"] + "x"`, I do get a message
immediately saying my credentials are unknown. So I do think they are
being ingested correctly. I’m using GDAL version 3.7.1.
I also managed to download the entire file using `boto3`, by doing the
following:
import boto3
client = boto3.client(
's3',
aws_access_key_id=creds["accessKeyId"],
aws_secret_access_key=creds["secretAccessKey"],
aws_session_token=creds["sessionToken"]
)
client.download_file('prod-lads',
'VNP02IMG/VNP02IMG.A2023193.1942.002.2023194025636.nc', 'test.nc')
Any ideas what I'm doing wrong or how to make this work? In the end
I'm interested in accessing the files metadata without downloading the
entire file.
Regards,
Bert
_______________________________________________
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev
--
http://www.spatialys.com
My software is free, but my time generally not.
_______________________________________________
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev
