Tim,
as far as I can see this error message comes from the specific
ReadMultiRange() method that has no retry capabilities. It should
probably, although I'm unsure from an implementation point of view how
that would work with the multi request curl API. I guess it can. Anway,
just to tell that you're going to be out of luck there.
A workaround would be to set GDAL_HTTP_MULTIRANGE=SERIAL to go back to
the classic code path (obviously you'll loose the parallel network
request aspect!), and there GDAL_HTTP_RETRY_CODES=ALL should hopefully
work even for response_code=0
Even
Le 19/12/2024 à 20:13, Tim Harris via gdal-dev a écrit :
I am using Python and translating small chunks of imagery out of S3
and occasionally run into errors like this:
2024-12-17 18:06:16.201 MST: ERROR 1: Request for 372390946-372700449
failed with response_code=0
2024-12-17 18:06:16.201 MST: ERROR 1: Request for
2028508162-2028817665 failed with response_code=0
2024-12-17 18:06:16.201 MST: ERROR 1: Request for
2030984194-2031293697 failed with response_code=0
2024-12-17 18:06:16.202 MST: ERROR 1: Request for
1476778594-1477088097 failed with response_code=0
2024-12-17 18:06:16.202 MST: ERROR 1: Request for 374247970-374557473
failed with response_code=0
Even with gdal.UseExceptions() and GDAL_HTTP_MAX_RETRY /
GDAL_HTTP_RETRY_DELAY set, it doesn’t seem to do anything other than
log this error and proceed. The end result is that my output file has
blank pixels in it.
I have been staring at the code a bit to understand what it means,
particularly the response_code being 0. I see this in the libcurl docs:
> The stored value is zero if no server response code has been received.
I still don’t quite know why this happens but could be that some AWS
server closed a connection early or something. But is there any way to
force a retry? I see that GDAL_HTTP_RETRY_CODES was added recently. If
I set it to “ALL” will that ensure that a situation like this results
in a retry even on a closed connection? I am not set up to easily
reproduce this or even to see a stack trace, so I’m not sure what
leads to this error. It doesn’t happen very often, but it’s disruptive
when it does happen because it’s hard to catch.
_______________________________________________
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev
--
http://www.spatialys.com
My software is free, but my time generally not.
Butcher of all kinds of standards, open or closed formats. At the end, this is
just about bytes.
Mood of the day: "Bien entendu, on peut sauter sur sa chaise comme un cabri en
disant : les standards ! les standards ! les standards ! Mais ça n’aboutit à rien et ça
ne signifie rien." ~ dixit De Gaulle
_______________________________________________
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev
