Hi 20% more speed into GeoJSON writing is notable. I do not use GeoJSON so much but some other people in the community may be happy. Thanks for having a look.
-Jukka- Lähettäjä: Even Rouault <even.roua...@spatialys.com> Lähetetty: tiistai 3. joulukuuta 2024 20.22 Vastaanottaja: Rahkonen Jukka <jukka.rahko...@maanmittauslaitos.fi>; 'gdal-dev@lists.osgeo.org' (gdal-dev@lists.osgeo.org) <gdal-dev@lists.osgeo.org> Aihe: Re: [gdal-dev] Does writing GeoJSON need to be so slow? Jukka, well, we have used up to now the same trick as a famous vendor did with their flagship text processing editor for Mac decades ago: add explicit sleep() to make the process slower, to discourage users from creating too large GeoJSON files, which are difficult to read if too big. More seriously, some modest enhancements for GML and GeoJSON in https://github.com/OSGeo/gdal/pull/11428 With them, I get 1m56s for whole file GeoJSON conversion (2m20s before) and 1m36s for GML (1m45s before). I found on my Linux system that MIF export was the fastest of the 4 text formats, not sure why that isn't the case on Windows. Why is ExportGeoJSON so fast? Completely hand-written compared to the OGR GeoJSON driver which constructs a json_object* hierarchical representation of each feature before serializing it to string, the fact that the OGR GeoJSON driver implements "smart" rounding/truncation logic, and possibly (didn't check) the fact the the sqlite3_mprintf() routine is faster than standard library printf(). Even Le 28/11/2024 à 14:43, Rahkonen Jukka via gdal-dev a écrit : Hi, I was comparing some alternative scenarios for data exports, and I was a bit surprised when I noticed that GeoJSON output from ogr2ogr is really slow. I used these lake polygons as test data https://wwwd3.ymparisto.fi/d3/gis_data/spesific/ranta10jarvet.zip and I tested on Windows with GDAL 3.11.0dev-181b6b9991, released 2024/11/21. I was thinking that maybe it is slow to write JSON just because it is text based format so I made tests also with other text formats (GML, MapInfo MIF, and CSV). My commands and timings: ogr2ogr -f geojson lakes.json jarvi10.shp --config cpl_debug on --config cpl_timestamp on 220 sec - 1000 features/sec ogr2ogr -f "mapinfo file" lakes.mif jarvi10.shp --config cpl_debug on --config cpl_timestamp on 110 sec - 2000 features/sec ogr2ogr -f gml lakes.gml jarvi10.shp --config cpl_debug on --config cpl_timestamp on 92 sec - 2300 features/sec ogr2ogr -f csv lakes.csv jarvi10.shp -lco geometry=as_wkt --config cpl_debug on --config cpl_timestamp on 77 sec - 2800 featurs/sec Then I pondered if I know any other tools for exporting GeoJSON, and SpatiaLite came into my mind. ExportGeoJSON https://www.gaia-gis.it/gaia-sins/spatialite-sql-5.1.0.html from GeoPackage into GeoJSON file was 4 times faster than ogr2ogr. select exportgeojson('vgpkg_jarvi10','geom','c:\data\jarvet\fromspatialite.json'); 54 sec - 4000 features/sec For calibrating the speedometer, I converted data also from shapefile into GeoPackage ogr2ogr -f gpkg lakes.gpkg jarvi10.shp --config cpl_debug on --config cpl_timestamp on 12 sec - 18000 features/sec I made also a couple of tests with geojsonseq output but I did not notice much difference. Does writing GeoJSON require some tricks that other formats do not require, or why it is so slow? -Jukka Rahkonen- _______________________________________________ gdal-dev mailing list gdal-dev@lists.osgeo.org<mailto:gdal-dev@lists.osgeo.org> https://lists.osgeo.org/mailman/listinfo/gdal-dev -- http://www.spatialys.com<http://www.spatialys.com/> My software is free, but my time generally not. Butcher of all kinds of standards, open or closed formats. At the end, this is just about bytes.
_______________________________________________ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
