Even, I'm actually surprised that it is considered impractical to make some/most drivers thread safe (read-only) natively and I thought key ones already were mostly there in the past though I'm not so confident in my memory. Looking through the GTiff code I find myself not able to work out where locking is done and how we are managing the directory loading/switching as block reads are done.
I would certainly like to work towards read-only thread safety for some drivers with: - a shared block cache for all threads reading from a dataset - no duplication of file handles per-thread - a way of checking that a dataset is considered to be thread safe. I'm curious if you can talk a bit more about the challenges of accomplishing the above. I had been hoping to take advantage of my (assumption) that this already mostly existed in GDAL for our new high performance environment at Planet so it is of more than academic interest to me. Also, doesn't gdalwarp's multithreading support already depend on read-only thread safety for source files? The above said, I can also see some utility in RFC 101 and have no objection to it. Best regards, Frank On Fri, Sep 6, 2024 at 12:45 PM Kurt Schwehr via gdal-dev < gdal-dev@lists.osgeo.org> wrote: > My take would be to not offer the force option unless there is someone who > states they really need it. > > On Fri, Sep 6, 2024 at 6:52 AM Even Rouault via gdal-dev < > gdal-dev@lists.osgeo.org> wrote: > >> Hi, >> >> No other reactions? In particular w.r.t. the open question whether it is >> worth to offer a "force" mode in the situations where a dataset cannot >> be re-opened (or for a band whose belonging to a dataset cannot be >> determined) where the thread-safe wrapper would just put a mutex around >> the calls to that dataset/band (so obviously this wouldn't scale with >> the number of threads). I'm probably leaning on not offering this option >> for now, if there's no obvious need it is useful. It can always been >> added afterwards. >> >> Even >> >> Le 29/08/2024 à 14:26, Even Rouault via gdal-dev a écrit : >> > Hi, >> > >> > I've worked on a new RFC 101: Raster dataset read-only thread-safety: >> > https://github.com/OSGeo/gdal/pull/10676 >> > >> > Summary: This RFC enables users to get instances of GDALDataset (and >> > their related objects such as GDALRasterBand) that are thread-safe for >> > read-only raster operations, that is such instances can be safely used >> > from multiple threads without locking. >> > >> > It is backed by a preliminary implementation (fairly complete, apart >> > from writing new tests...). There's at least one open question raised >> > in the RFC. >> > >> > PS: you can see one of the benefits of the migration of the doc to >> > ReadTheDocs hosting, with the preview of the updated documentation in >> > pull requests. So this is a way to mention that the move to RtD >> > hosting has now been completed, with the default version ("latest") >> > reflecting the state of the master branch, as well as the version for >> > the release/3.9 branch. Thanks to Dan Baston (and other people >> > involved) for this work! There might be a few adjustments needed, so >> > if you notice something unusual w.r.t. to the previous state, please >> > file a ticket. >> > >> > Even >> > >> -- >> http://www.spatialys.com >> My software is free, but my time generally not. >> >> _______________________________________________ >> gdal-dev mailing list >> gdal-dev@lists.osgeo.org >> https://lists.osgeo.org/mailman/listinfo/gdal-dev >> > _______________________________________________ > gdal-dev mailing list > gdal-dev@lists.osgeo.org > https://lists.osgeo.org/mailman/listinfo/gdal-dev > -- ---------------------------------------+-------------------------------------- I set the clouds in motion - turn up | Frank Warmerdam, warmer...@pobox.com light and sound - activate the windows | USA: +1 650-701-7823 <http://voice.google.com/calls?a=nc,%2B16507017823> <http://voice.google.com/calls?a=nc,%2B16507017823> and watch the world go round - Rush | CAN: +1 343-550-9984
_______________________________________________ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
