Thanks for the announcement, Even! I wonder if we should track such issues in a list? Or maybe give them a unique GitHub label?
We don't plan to release a 3.6.5, correct? I'm going to make a Rasterio post release that patches 3.6.4 by tomorrow: https://github.com/rasterio/rasterio/issues/2943. On Mon, Oct 16, 2023 at 9:26 AM Even Rouault via gdal-dev < gdal-dev@lists.osgeo.org> wrote: > > Le 16/10/2023 à 17:14, Kurt Schwehr a écrit : > > Thanks for the heads up! > > Can you share that SHAs of the fix and cause? > > master: d083af1ec9c38e79bfb0532885570de6e5b8a410 > > 3.7 branch (should apply to 3.6 as well): > b5858ed5bc5004c97f7cd6000674015bdc70b33b > > cause: a worker thread of the multithreaded decoding could, in a situation > where the block cache is full, cause a "dirty" (ie modified but not yet > serialized to disk) block to be written, resulting in either a deadlock > between the lock of the multithreaded decoder and the lock of the job queue > mechanism, or other decoding threads could see their file handle being > seek() by another thread. In short: multithreading is hard. > > > On Mon, Oct 16, 2023, 7:44 AM Even Rouault via gdal-dev < > gdal-dev@lists.osgeo.org> wrote: > >> Hi, >> >> For GDAL 3.6.0 to 3.7.2, use of multi-threaded GTiff >> compression+decompression, in particular within the same file, as for >> example in a "gdalwarp -co COMPRESS=... -co NUM_THREADS=..." workflow >> can lead to deadlocks (process stalled forever) and/or data corruption >> (sometimes without errors at generation). Cf >> https://github.com/OSGeo/gdal/issues/8470 for a reproducer. The fix is >> in https://github.com/OSGeo/gdal/pull/8561 >> >> The issue is particularly visible on Windows, or more generally any >> operating system (or file system where the output file is located) which >> has no VSIVirtualHandle::PRead() implementation, but it can also be >> occasionally reproduced on Linux (at least as a deadlock). >> >> Even >> >> -- Sean Gillies
_______________________________________________ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
