hum... debugging in the C layers, that'll be interesting... Ok, I'll continue to dig in, thanks
--- Gregory Bataille On Tue, Mar 21, 2017 at 8:44 AM, Even Rouault <[email protected]> wrote: > On mardi 21 mars 2017 05:51:18 CET Grégory Bataille wrote: > > > Hey Even, > > > > > > reaching out again, because I just don't know... > > > > > > so here is gdal2tiles_parallel strategy (as far as I understand it) > > > - in each process, open the input file, compute the autowarped vrt (in > the > > > case of the dataset I use at least) loop over tiles to generate and work > > > only depending on the process number (so that this work is made in > > > parallel). We therefore end up with, in each process > > > - the input file opened as a dataset (but not touched after the > > > autowarped vrt is generated) > > > - a different autowarped vrt in memory (for each process) but pointing > > > to the same source (as far as I can see) > > > > > > > > > My strategy: > > > - open the input file. > > > - generate the autowarped vrt and save it to disk > > > - compute all the tile details and store them in a data structure. > > > Then in each process: > > > > I'm confused. You're talking about process, but is it a real operating > system process or a thread ? I guess the later. > > > > > - take one tile detail > > > - open the vrt (same file for each process, pointing to the same source > > > TIFF) > > > - read the vrt and write the tile > > > And that's the read that is failing as mentioned in my first email. > > > > > > I can't see much difference. To try and be complete, I tried to (or > think I > > > tried to): > > > - open the input file in each process > > > - generate a different vrt file for each thread > > > > > > I even copy/pasted the code that gdal2tiles_parallel uses in their > > > processes, but with no success, always the same error. The > > > VRT_SHARED_SOURCE option > > > does not seem to change anything (but looking at the wiki, it's not clear > > > whether it applies to autowarped vrt too) > > > > VRT_SHARED_SOURCE indeed only works for "regular" mosaicing VRTs. So not > to warped VRT. > > > > Warped VRT opens the source dataset as a shared dataset in > > https://github.com/OSGeo/gdal/blob/trunk/gdal/alg/gdalwarper.cpp#L1675 > > So you could have an issue if you open the VRT twice in the same thread, > and then use each handle in a different thread, since both VRT handles > would point to the same shared dataset. But it doesn't look that's what you > are doing. > > > > As a workaround to investigate, you could try to remove the GDAL_OF_SHARED > | flag in the line I mentionned and see if that makes a difference (but we > could potentially - not sure - have issues at closing time since different > call sites might make an assumption on the dataset being opened with shared > semantics) > > > > What you could do too is run the script through your prefered debugger and > set a breakpoint in GDALOpenEx and look at the calling sites. If you are > using 2 threads, then if we want things to work correctly, GDALOpenEx() > should be called twice on the TIFF dataset and return a different handle. > > > > Even > > > > -- > > Spatialys - Geospatial professional services > > http://www.spatialys.com >
_______________________________________________ gdal-dev mailing list [email protected] https://lists.osgeo.org/mailman/listinfo/gdal-dev
