Looks to me like you are reading a lot of rows under a lock, and never releasing the lock, so the rows remain in memory.
I don’t know the internals of the SQLite very well, but my understanding is that it is not really a “driver” in the traditional sense that communicates with a db - but rather it is the implementation as well. Since it is the implementation, holding the lock seems reasonable to also hold the rows. > On Mar 9, 2025, at 2:59 PM, Gavra <gav...@gmail.com> wrote: > > Hi, > So we have been trying really hard to understand a major leak in our product. > We are using this sqlite3 driver: https://github.com/mattn/go-sqlite3 > The pprof heap profile indicates the source is the call to SQLiteRows.Columns > func and two additional calls, one on SQLiteStmt and the other on SQLiteConn. > According to the code > 1. SQLiteRows references SQLiteStmt which then references to SQLiteConn > 2. The SQLiteRows instance is the sole object holding a ref to the allocation > by Columns(). > This is a strong indication that refs to SQLiteRows are leaked. > We can confirm the leak is increasing over time and does not appear to > reflect a burst or large data. > We thought we forgot to close rows or somehow appened refs to rows etc but we > ruled it out completely since. Note the heap profile only shows SQLite > allocations, no app objects allocated. We verified that forgetting to call > SQLiteRows.Close leaks only memory in CGO which means it is not visible in > the heap profile. > So on one hand we are convinced someone is holding a reference to SQLiteRows > but on the other hand it is not our application and not the SQLite driver. > > We tracked back our source code changes and noticed that correlate to the > appearance of this issue: > 1. Upgrading go: 1.22.6 to 1.22.9 > 2. Upgrading the sqlite3 driver which had one non-negligible change: adding > SetFinalizer to all these objects. > > This is a very weird thing to suggest, but we think this could be caused by > the go runtime, somehow. > I attached a screenshot of the heap profile, focused on the major leak around > the sqlite3 driver. > (our current plans is to use goref or dlv on a core dump to understand who is > holding these references but that could take a few more days). > > Thank you. > > -- > You received this message because you are subscribed to the Google Groups > "golang-nuts" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to golang-nuts+unsubscr...@googlegroups.com > <mailto:golang-nuts+unsubscr...@googlegroups.com>. > To view this discussion visit > https://groups.google.com/d/msgid/golang-nuts/8a8b608b-e501-403a-b7a7-0d0bda657e4cn%40googlegroups.com > > <https://groups.google.com/d/msgid/golang-nuts/8a8b608b-e501-403a-b7a7-0d0bda657e4cn%40googlegroups.com?utm_medium=email&utm_source=footer>. > <Screenshot 2025-03-09 at 21.40.22.png> -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/golang-nuts/367EB2B6-F4F3-42B6-8B5E-F04ABA76CE42%40ix.netcom.com.
