Thanks for that! The sites you listed worked will with the gdal vsi list
client.
Interestingly, the following worked with your sites as well, but not the
NOAA site:
wget --spider -r --no-parent
https://cdn.star.nesdis.noaa.gov/GOES19/ABI/MESO/18N-76W/GEOCOLOR/ 2>&1
| grep "^--"
On 10/28/25 11:44, Laurențiu Nicola via gdal-dev wrote:
I was wrong, GDAL actually has code to deal with the autoindex pages:
https://github.com/OSGeo/gdal/blob/0f757d0/port/cpl_vsil_curl.cpp#L5024-L5340.
It just doesn't work very well with your site:
gdal.ReadDir("/vsicurl/https://cscie12.dce.harvard.edu/apache/autoindex/";)
['client-server.gif', 'raspberry.gif', 'raspberry.jpg', 'raspberry.png',
'slide1.html', 'slide2.html', 'slide3.html']
gdal.ReadDir("/vsicurl/https://nginx.org/packages/";)
['aix', 'alpine', 'amzn', 'centos', 'debian', 'keys', 'rhel', 'sles', 'ubuntu',
'mainline']
gdal.ReadDir("/vsicurl/https://cdn.star.nesdis.noaa.gov/GOES18/ABI/MESO/M1/GEOCOLOR/";)
(nothing)
Laurentiu
On Tue, Oct 28, 2025, at 20:31, Scott via gdal-dev wrote:
But, the docs say:
gdal vsi list list files of GDAL Virtual File Systems (compressed,
network hosted, etc...): /vsimem, /vsizip, /vsitar, /vsicurl, ....
...so this implies http dir listings via /vsicurl. I may be interpreting
that incorrectly, though.
Scott
On 10/28/25 11:17, Laurențiu Nicola via gdal-dev wrote:
I'd be quite surprised if GDAL's vsicurl was able to understand a directory
index.
Take https://cscie12.dce.harvard.edu/apache/autoindex/, for example, it's clear
that's a directory with 7 files, but it also has a bunch of column sorting
links you'd want to ignore.
To be fair, there aren't that many directory index formats, but I'm not sure
that parsing them is a job for GDAL.
Laurentiu
On Tue, Oct 28, 2025, at 18:28, Scott via gdal-dev wrote:
Using list with /vsicurl only returns results if I specify a file. No
results when specifying a directory. /vsis3 works as expected. Am I
doing this correctly?
Works:
gdal vsi list -lR
"/vsicurl/https://cdn.star.nesdis.noaa.gov/GOES18/ABI/MESO/M1/GEOCOLOR/2025292000425_GOES18-ABI-MESO-GEOCOLOR-39N-119W-1000x1000.jpg";
No workee:
gdal vsi list -lR
"/vsicurl/https://cdn.star.nesdis.noaa.gov/GOES18/ABI/MESO/M1/GEOCOLOR/";
/vsicurl_streaming only returns directory name:
gdal vsi list -lR
"/vsicurl_streaming/https://cdn.star.nesdis.noaa.gov/GOES18/ABI/MESO/M1/GEOCOLOR/";
--depth has no effect.
Debian 12/GDAL 3.12.0beta1
Thanks!
Scott
--
www.postholer.com
_______________________________________________
gdal-dev mailing list
[email protected]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
_______________________________________________
gdal-dev mailing list
[email protected]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
_______________________________________________
gdal-dev mailing list
[email protected]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
_______________________________________________
gdal-dev mailing list
[email protected]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
_______________________________________________
gdal-dev mailing list
[email protected]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
![https://cdn.star.nesdis.noaa.gov/GOES18/ABI/MESO/M1/GEOCOLOR/2025292000425_GOES18-ABI-MESO-GEOCOLOR-39N-119W-1000x1000.jpg"](https://cdn.star.nesdis.noaa.gov/GOES18/ABI/MESO/M1/GEOCOLOR/2025292000425_GOES18-ABI-MESO-GEOCOLOR-39N-119W-1000x1000.jpg"){kind=link}