Re: [gdal-dev] Proper usage of gdal vsi list

Tue, 28 Oct 2025 11:44:40 -0700 

I was wrong, GDAL actually has code to deal with the autoindex pages: 
https://github.com/OSGeo/gdal/blob/0f757d0/port/cpl_vsil_curl.cpp#L5024-L5340. 
It just doesn't work very well with your site:

>>> gdal.ReadDir("/vsicurl/https://cscie12.dce.harvard.edu/apache/autoindex/";)
['client-server.gif', 'raspberry.gif', 'raspberry.jpg', 'raspberry.png', 
'slide1.html', 'slide2.html', 'slide3.html']

>>> gdal.ReadDir("/vsicurl/https://nginx.org/packages/";)
['aix', 'alpine', 'amzn', 'centos', 'debian', 'keys', 'rhel', 'sles', 'ubuntu', 
'mainline']

>>> gdal.ReadDir("/vsicurl/https://cdn.star.nesdis.noaa.gov/GOES18/ABI/MESO/M1/GEOCOLOR/";)
(nothing)

Laurentiu

On Tue, Oct 28, 2025, at 20:31, Scott via gdal-dev wrote:
> But, the docs say:
>
> gdal vsi list list files of GDAL Virtual File Systems (compressed, 
> network hosted, etc...): /vsimem, /vsizip, /vsitar, /vsicurl, ....
>
> ...so this implies http dir listings via /vsicurl. I may be interpreting 
> that incorrectly, though.
>
> Scott
>
> On 10/28/25 11:17, Laurențiu Nicola via gdal-dev wrote:
>> I'd be quite surprised if GDAL's vsicurl was able to understand a directory 
>> index.
>> 
>> Take https://cscie12.dce.harvard.edu/apache/autoindex/, for example, it's 
>> clear that's a directory with 7 files, but it also has a bunch of column 
>> sorting links you'd want to ignore.
>> 
>> To be fair, there aren't that many directory index formats, but I'm not sure 
>> that parsing them is a job for GDAL.
>> 
>> Laurentiu
>> 
>> On Tue, Oct 28, 2025, at 18:28, Scott via gdal-dev wrote:
>>> Using list with /vsicurl only returns results if I specify a file. No
>>> results when specifying a directory. /vsis3 works as expected. Am I
>>> doing this correctly?
>>>
>>> Works:
>>> gdal vsi list -lR
>>> "/vsicurl/https://cdn.star.nesdis.noaa.gov/GOES18/ABI/MESO/M1/GEOCOLOR/2025292000425_GOES18-ABI-MESO-GEOCOLOR-39N-119W-1000x1000.jpg";
>>>
>>> No workee:
>>> gdal vsi list -lR
>>> "/vsicurl/https://cdn.star.nesdis.noaa.gov/GOES18/ABI/MESO/M1/GEOCOLOR/";
>>>
>>> /vsicurl_streaming only returns directory name:
>>> gdal vsi list -lR
>>> "/vsicurl_streaming/https://cdn.star.nesdis.noaa.gov/GOES18/ABI/MESO/M1/GEOCOLOR/";
>>>
>>> --depth has no effect.
>>>
>>> Debian 12/GDAL 3.12.0beta1
>>>
>>> Thanks!
>>> Scott
>>>
>>> -- 
>>> www.postholer.com
>>>
>>> _______________________________________________
>>> gdal-dev mailing list
>>> [email protected]
>>> https://lists.osgeo.org/mailman/listinfo/gdal-dev
>> _______________________________________________
>> gdal-dev mailing list
>> [email protected]
>> https://lists.osgeo.org/mailman/listinfo/gdal-dev
>
> _______________________________________________
> gdal-dev mailing list
> [email protected]
> https://lists.osgeo.org/mailman/listinfo/gdal-dev
_______________________________________________
gdal-dev mailing list
[email protected]
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Reply via email to