I think GDAL doesn't parse it because the URL doesn't match the directory name on the page (/ma-cdn03/GOES/data/GOES18/ABI/MESO/M1/GEOCOLOR/).
On Tue, Oct 28, 2025, at 20:58, Scott via gdal-dev wrote: > Thanks for that! The sites you listed worked will with the gdal vsi list > client. > > Interestingly, the following worked with your sites as well, but not the > NOAA site: > > wget --spider -r --no-parent > https://cdn.star.nesdis.noaa.gov/GOES19/ABI/MESO/18N-76W/GEOCOLOR/ 2>&1 > | grep "^--" > > > On 10/28/25 11:44, Laurențiu Nicola via gdal-dev wrote: >> I was wrong, GDAL actually has code to deal with the autoindex pages: >> https://github.com/OSGeo/gdal/blob/0f757d0/port/cpl_vsil_curl.cpp#L5024-L5340. >> It just doesn't work very well with your site: >> >>>>> gdal.ReadDir("/vsicurl/https://cscie12.dce.harvard.edu/apache/autoindex/";) >> ['client-server.gif', 'raspberry.gif', 'raspberry.jpg', 'raspberry.png', >> 'slide1.html', 'slide2.html', 'slide3.html'] >> >>>>> gdal.ReadDir("/vsicurl/https://nginx.org/packages/";) >> ['aix', 'alpine', 'amzn', 'centos', 'debian', 'keys', 'rhel', 'sles', >> 'ubuntu', 'mainline'] >> >>>>> gdal.ReadDir("/vsicurl/https://cdn.star.nesdis.noaa.gov/GOES18/ABI/MESO/M1/GEOCOLOR/";) >> (nothing) >> >> Laurentiu >> >> On Tue, Oct 28, 2025, at 20:31, Scott via gdal-dev wrote: >>> But, the docs say: >>> >>> gdal vsi list list files of GDAL Virtual File Systems (compressed, >>> network hosted, etc...): /vsimem, /vsizip, /vsitar, /vsicurl, .... >>> >>> ...so this implies http dir listings via /vsicurl. I may be interpreting >>> that incorrectly, though. >>> >>> Scott >>> >>> On 10/28/25 11:17, Laurențiu Nicola via gdal-dev wrote: >>>> I'd be quite surprised if GDAL's vsicurl was able to understand a >>>> directory index. >>>> >>>> Take https://cscie12.dce.harvard.edu/apache/autoindex/, for example, it's >>>> clear that's a directory with 7 files, but it also has a bunch of column >>>> sorting links you'd want to ignore. >>>> >>>> To be fair, there aren't that many directory index formats, but I'm not >>>> sure that parsing them is a job for GDAL. >>>> >>>> Laurentiu >>>> >>>> On Tue, Oct 28, 2025, at 18:28, Scott via gdal-dev wrote: >>>>> Using list with /vsicurl only returns results if I specify a file. No >>>>> results when specifying a directory. /vsis3 works as expected. Am I >>>>> doing this correctly? >>>>> >>>>> Works: >>>>> gdal vsi list -lR >>>>> "/vsicurl/https://cdn.star.nesdis.noaa.gov/GOES18/ABI/MESO/M1/GEOCOLOR/2025292000425_GOES18-ABI-MESO-GEOCOLOR-39N-119W-1000x1000.jpg"; >>>>> >>>>> No workee: >>>>> gdal vsi list -lR >>>>> "/vsicurl/https://cdn.star.nesdis.noaa.gov/GOES18/ABI/MESO/M1/GEOCOLOR/"; >>>>> >>>>> /vsicurl_streaming only returns directory name: >>>>> gdal vsi list -lR >>>>> "/vsicurl_streaming/https://cdn.star.nesdis.noaa.gov/GOES18/ABI/MESO/M1/GEOCOLOR/"; >>>>> >>>>> --depth has no effect. >>>>> >>>>> Debian 12/GDAL 3.12.0beta1 >>>>> >>>>> Thanks! >>>>> Scott >>>>> >>>>> -- >>>>> www.postholer.com >>>>> >>>>> _______________________________________________ >>>>> gdal-dev mailing list >>>>> [email protected] >>>>> https://lists.osgeo.org/mailman/listinfo/gdal-dev >>>> _______________________________________________ >>>> gdal-dev mailing list >>>> [email protected] >>>> https://lists.osgeo.org/mailman/listinfo/gdal-dev >>> >>> _______________________________________________ >>> gdal-dev mailing list >>> [email protected] >>> https://lists.osgeo.org/mailman/listinfo/gdal-dev >> _______________________________________________ >> gdal-dev mailing list >> [email protected] >> https://lists.osgeo.org/mailman/listinfo/gdal-dev > > _______________________________________________ > gdal-dev mailing list > [email protected] > https://lists.osgeo.org/mailman/listinfo/gdal-dev _______________________________________________ gdal-dev mailing list [email protected] https://lists.osgeo.org/mailman/listinfo/gdal-dev
![https://cdn.star.nesdis.noaa.gov/GOES18/ABI/MESO/M1/GEOCOLOR/2025292000425_GOES18-ABI-MESO-GEOCOLOR-39N-119W-1000x1000.jpg"](https://cdn.star.nesdis.noaa.gov/GOES18/ABI/MESO/M1/GEOCOLOR/2025292000425_GOES18-ABI-MESO-GEOCOLOR-39N-119W-1000x1000.jpg"){kind=link}