Hi,,
I’m creating vector tiles from a PostGIS database using ogr2ogr with
the MVT driver. The region is quite large and I’m creating tiles for
levels 0-15, so the process takes quite some time (hours). The tiles
are written directly to a S3 storage using the /vsis3/ virtual file
system (see:
https://gdal.org/user/virtual_file_systems.html#vsis3-aws-s3-files).
Looking at the logs, I can see that, firstly, a “temporary database”
file (*.temp.db) is created. Some things are notable:
- this step seems to be CPU intensive compared to the creation of the
actual pbf files
- the temporary database file is not being written to the output file
directory (S3 storage in this case) as stated in the documentation
I don't observe this:
$ ogr2ogr /vsis3/XXXXX/foo.mvt poly.shp
ERROR 1: wb+ not supported for /vsis3/, unless
CPL_VSIL_USE_TEMP_FILE_FOR_RANDOM_WRITE is set to YES
vs
$ ogr2ogr /vsis3/spatialys/foo.mvt poly.shp -dsco
TEMPORARY_DB=/tmp/my.temp.db
- the dataset creation option `TEMPORARY_DB` (see:
https://gdal.org/drivers/vector/mvt.html#dataset-creation-options)
seems to be ignored (I tried writing the file to a local directory but
no such file shows up)
On Unix, the file is immediately deleted after being opened, so that if
you kill the process there's no remaining temporary file. You can set
the OGR_MVT_REMOVE_TEMP_FILE env variable to NO to disable this
- my guess is that the file is only kept in memory (which would be
/vsimem/ if I’m correct)
Is it possible to skip the step of first creating the temp db and
directly create tiles from PostGIS or any other data source?
No
Where/how exactly is the database file stored? Any ideas on optimizing
the process?
You could possibly paralle
Also, the MVT generator is intended to be basic. You might perhaps have
more functionality with https://github.com/felt/tippecanoe
I’m looking forward to some discussion on this topic. I will happily
try to contribute where I can.
Regards,
Leo
_______________________________________________
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev
--
http://www.spatialys.com
My software is free, but my time generally not.
_______________________________________________
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev
