Re: [gdal-dev] a possible memory leak in ogr\ogrsf_frmts\sqlite
Hello, it may be needed for sure for the whole session or at least if you need the already registered extension to be available at the next db open attempt. A deeper look in sqlite3 gave me the sqlite3_shutdown() API that actually frees the extension attached via sqlite3_reset_auto_extension() together with various other sqlite3 stuff. I do not see GDAL to call sqlite3_shutdown() anywhere and I think it is quite reasonable to do it in some GDAL sqlite driver finalization steps - in the OGRSQLiteBaseDataSource destructor, in the Close() method after closing db, etc. Deyan On Thu, Feb 6, 2025 at 12:52 PM Even Rouault wrote: > > Deyan > > this sounds more like "still reachable" memory in Valgrind terminology > (https://valgrind.org/docs/manual/faq.html#faq.deflost), that is memory > that is allocated by some component, still reachable by it but never > explicitly freed before the end of the process, and the amount of it > doesn't grow over time. There are certainly many similar such patterns > in GDAL or other libraries it uses, and I don't see that as a bug or > something that requires code changes. > > Even > > Le 06/02/2025 à 11:34, Deyan Vasilev via gdal-dev a écrit : > > Hello, > > > > I was chasing for a long time a small 8 byte memory leak and was able > > to identify it in the OGR sqlite extension binding, here's what I > > observed: > > > > A call to OGRSQLiteBaseDataSource::OpenOrCreateDB() would invoke > > OGR2SQLITE_Register() and then sqlite3_auto_extension(). The latter is > > a sqlite3 call that would malloc an 8 byte buffer for the extension > > function entry point if the extension is unknown to sqlite3 (was not > > previously registered). > > > > This allocation seems to be unmanaged neither by sqlite3 nor by OGR > > later causing an 8 byte memory leak. I was able to resolve this by > > adding a OGR2SQLITE_UnRegister() stub in ogrsqlitevirtualogr.cpp that > > would call in a succession: > > > > sqlite3_cancel_auto_extension ((void (*)(void)) OGR2SQLITE_static_register); > > sqlite3_reset_auto_extension (); > > < > > > > and then have the OGR2SQLITE_UnRegister() stub called in > > ogrsqlitedatasource.cpp \ OGRSQLiteBaseDataSource::CloseDB() right > > after closing the database: > > > > sqlite3_close( hDB ); > > hDB = nullptr; > > OGR2SQLITE_UnRegister(); > > < > > > > This way the memory leak was gone. My app runs on VS2015/2019 with an > > older GDAL release (2.4.4.) so the fix will be a bit different in 3.10 > > as the HAVE_SQLITE3EXT_H conditionals were not present in 2.4.4.. > > > > Best regards, > > Deyan > > ___ > > gdal-dev mailing list > > gdal-dev@lists.osgeo.org > > https://lists.osgeo.org/mailman/listinfo/gdal-dev > > -- > http://www.spatialys.com > My software is free, but my time generally not. > ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] a possible memory leak in ogr\ogrsf_frmts\sqlite
Le 06/02/2025 à 12:30, Deyan Vasilev a écrit : Hello, it may be needed for sure for the whole session or at least if you need the already registered extension to be available at the next db open attempt. A deeper look in sqlite3 gave me the sqlite3_shutdown() API that actually frees the extension attached via sqlite3_reset_auto_extension() together with various other sqlite3 stuff. I do not see GDAL to call sqlite3_shutdown() anywhere and I think it is quite reasonable to do it in some GDAL sqlite driver finalization steps - in the OGRSQLiteBaseDataSource destructor, in the Close() method after closing db, etc. GDAL is just a library. It may be integrated in an application that would use sqlite3 itself, and GDAL calling sqlite3_shutdown(), could case chaos in the application. -- http://www.spatialys.com My software is free, but my time generally not. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
[gdal-dev] GDAL open flags
Hi GDAL devs, does anyone know why GDAL_OF_ALL and GDAL_READONLY consts have the same value? Both are used by GDALOpenEx as nOpenFlags param so shouldn't they be distinguishable? *#define GDAL_OF_READONLY 0x00* *#define GDAL_OF_ALL 0x00* https://github.com/OSGeo/gdal/blob/5248a30e9a84998dd0416ab7b43fb1a3b1a18cb2/gcore/gdal.h#L999 https://github.com/OSGeo/gdal/blob/5248a30e9a84998dd0416ab7b43fb1a3b1a18cb2/gcore/gdal.h#L1011 Have a good day! Michał Kowalczuk ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] GDAL open flags
It make sense. If I find an unusual case, I'll write here :-) Thanks! czw., 6 lut 2025 o 09:19 Javier Jimenez Shaw napisał(a): > I would say that 0 is the default. Read only and all drivers. > If you want anything else, then specify it. > > On Thu, 6 Feb 2025, 09:10 Michał Kowalczuk via gdal-dev, < > gdal-dev@lists.osgeo.org> wrote: > >> Hi GDAL devs, >> does anyone know why GDAL_OF_ALL and GDAL_READONLY consts have the same >> value? >> Both are used by GDALOpenEx as nOpenFlags param so shouldn't they be >> distinguishable? >> >> *#define GDAL_OF_READONLY 0x00* >> *#define GDAL_OF_ALL 0x00* >> >> >> https://github.com/OSGeo/gdal/blob/5248a30e9a84998dd0416ab7b43fb1a3b1a18cb2/gcore/gdal.h#L999 >> >> https://github.com/OSGeo/gdal/blob/5248a30e9a84998dd0416ab7b43fb1a3b1a18cb2/gcore/gdal.h#L1011 >> >> Have a good day! >> Michał Kowalczuk >> ___ >> gdal-dev mailing list >> gdal-dev@lists.osgeo.org >> https://lists.osgeo.org/mailman/listinfo/gdal-dev >> > ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] GDAL open flags
I would say that 0 is the default. Read only and all drivers. If you want anything else, then specify it. On Thu, 6 Feb 2025, 09:10 Michał Kowalczuk via gdal-dev, < gdal-dev@lists.osgeo.org> wrote: > Hi GDAL devs, > does anyone know why GDAL_OF_ALL and GDAL_READONLY consts have the same > value? > Both are used by GDALOpenEx as nOpenFlags param so shouldn't they be > distinguishable? > > *#define GDAL_OF_READONLY 0x00* > *#define GDAL_OF_ALL 0x00* > > > https://github.com/OSGeo/gdal/blob/5248a30e9a84998dd0416ab7b43fb1a3b1a18cb2/gcore/gdal.h#L999 > > https://github.com/OSGeo/gdal/blob/5248a30e9a84998dd0416ab7b43fb1a3b1a18cb2/gcore/gdal.h#L1011 > > Have a good day! > Michał Kowalczuk > ___ > gdal-dev mailing list > gdal-dev@lists.osgeo.org > https://lists.osgeo.org/mailman/listinfo/gdal-dev > ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
[gdal-dev] a possible memory leak in ogr\ogrsf_frmts\sqlite
Hello, I was chasing for a long time a small 8 byte memory leak and was able to identify it in the OGR sqlite extension binding, here's what I observed: A call to OGRSQLiteBaseDataSource::OpenOrCreateDB() would invoke OGR2SQLITE_Register() and then sqlite3_auto_extension(). The latter is a sqlite3 call that would malloc an 8 byte buffer for the extension function entry point if the extension is unknown to sqlite3 (was not previously registered). This allocation seems to be unmanaged neither by sqlite3 nor by OGR later causing an 8 byte memory leak. I was able to resolve this by adding a OGR2SQLITE_UnRegister() stub in ogrsqlitevirtualogr.cpp that would call in a succession: > sqlite3_cancel_auto_extension ((void (*)(void)) OGR2SQLITE_static_register); sqlite3_reset_auto_extension (); < and then have the OGR2SQLITE_UnRegister() stub called in ogrsqlitedatasource.cpp \ OGRSQLiteBaseDataSource::CloseDB() right after closing the database: > sqlite3_close( hDB ); hDB = nullptr; OGR2SQLITE_UnRegister(); < This way the memory leak was gone. My app runs on VS2015/2019 with an older GDAL release (2.4.4.) so the fix will be a bit different in 3.10 as the HAVE_SQLITE3EXT_H conditionals were not present in 2.4.4.. Best regards, Deyan ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] a possible memory leak in ogr\ogrsf_frmts\sqlite
Deyan this sounds more like "still reachable" memory in Valgrind terminology (https://valgrind.org/docs/manual/faq.html#faq.deflost), that is memory that is allocated by some component, still reachable by it but never explicitly freed before the end of the process, and the amount of it doesn't grow over time. There are certainly many similar such patterns in GDAL or other libraries it uses, and I don't see that as a bug or something that requires code changes. Even Le 06/02/2025 à 11:34, Deyan Vasilev via gdal-dev a écrit : Hello, I was chasing for a long time a small 8 byte memory leak and was able to identify it in the OGR sqlite extension binding, here's what I observed: A call to OGRSQLiteBaseDataSource::OpenOrCreateDB() would invoke OGR2SQLITE_Register() and then sqlite3_auto_extension(). The latter is a sqlite3 call that would malloc an 8 byte buffer for the extension function entry point if the extension is unknown to sqlite3 (was not previously registered). This allocation seems to be unmanaged neither by sqlite3 nor by OGR later causing an 8 byte memory leak. I was able to resolve this by adding a OGR2SQLITE_UnRegister() stub in ogrsqlitevirtualogr.cpp that would call in a succession: sqlite3_cancel_auto_extension ((void (*)(void)) OGR2SQLITE_static_register); sqlite3_reset_auto_extension (); < and then have the OGR2SQLITE_UnRegister() stub called in ogrsqlitedatasource.cpp \ OGRSQLiteBaseDataSource::CloseDB() right after closing the database: sqlite3_close( hDB ); hDB = nullptr; OGR2SQLITE_UnRegister(); < This way the memory leak was gone. My app runs on VS2015/2019 with an older GDAL release (2.4.4.) so the fix will be a bit different in 3.10 as the HAVE_SQLITE3EXT_H conditionals were not present in 2.4.4.. Best regards, Deyan ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev -- http://www.spatialys.com My software is free, but my time generally not. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
[gdal-dev] Call for review on RFC 107: Add OGRLayer::IGetExtent() and OGRLayer::ISetSpatialFilter()
Hi, Please review https://github.com/OSGeo/gdal/pull/11814: Add OGRLayer::IGetExtent() and OGRLayer::ISetSpatialFilter() It could have gone through a pull request business-as-usual, but as this impacts a number of drivers, including out-of-tree ones, this is worth this small RFC. Summary --- This RFC changes the prototype of the OGRLayer::GetExtent(), GetExtent3D(), SetSpatialFilter() and SetSpatialFilterRect() methods. Motivation -- Originally GetExtent(), SetSpatialFilter() and SetSpatialFilterRect() were designed for a single geometry field. When support for multiple geometry fields was added per :ref:`rfc-41`, alternate virtual methods were added to accept a ``int iGeomField`` argument, but this causes slightly repeating code patterns in most drivers, and omissions of the boilerplate can cause bugs. Even -- http://www.spatialys.com My software is free, but my time generally not. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
[gdal-dev] GDAL-GRASS GIS driver 1.0.3 released
The GDAL-GRASS driver 1.0.3 provides more than 25 improvements and fixes with respect to the release 1.0.2. Highlights: - CMake support added - Autoconf configure updated to version 2.71, with a number of fixes - Migrate use of deprecated GDAL driver registration API - Remove use of obsolete CPL_CVSID() macro - CI supported tests and code formatted with ClangFormat More complete information on changes can be found at: https://github.com/OSGeo/gdal-grass/releases/tag/1.0.3 The release can be downloaded from: https://download.osgeo.org/gdal-grass/gdal-grass-1.0.3.tar.gz https://download.osgeo.org/gdal-grass/gdal-grass-1.0.3.tar.gz.md5 https://download.osgeo.org/gdal-grass/gdal-grass-1.0.3.tar.gz.sha256 Best regards, Nicklas___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
