Hey, Vlad, it's not that. we have a similar or rather the exactly the same problem... where the OS doesn't claim the memory back.. this is also annoying to the point we had to disable memory under pressure notifications for that specific host. Also valgrind doesn't report any leaks... simulating the same with C app the memory is returned to the OS almost immediately.
Fortunately for us this is in the worker processes where we can restart them after about 1000 iterations. BR, Miha On Monday, August 24, 2020 at 11:05:00 AM UTC+2 vlad...@varank.in wrote: > Hey, > > I haven't looked deep but I recall there had been a note about runtime > change in Go 1.13's https://golang.org/doc/go1.13#runtime That is > > > The runtime is now more aggressive at returning memory to the operating > system to make it available to co-tenant applications [..] However, on many > OSes, including Linux, the OS itself reclaims memory lazily, so process RSS > will not decrease until the system is under memory pressure. > > Could that be the behaviour you observe (although, since you don't see the > same in C implementation, I might be confused, sorry in advance). > > On Sunday, August 23, 2020 at 5:05:22 PM UTC+2 Manish Rai Jain wrote: > >> Hey Gophers, >> >> I'm puzzled by a mysterious RSS memory spike in my Go program, when all >> memory allocations are happening via Cgo. I assert that there are no memory >> leaks in the program. And have written another C program with similar logic >> which does NOT show RSS memory spiking. So, I suspect this is something to >> do with Go memory. >> >> <https://github.com/dgraph-io/ristretto/pull/186> >> Program: >> >> https://github.com/dgraph-io/ristretto/pull/186 >> >> This PR creates a Go memtest program, which does this: >> - Uses z.Calloc and z.Free to allocate Go struct (S) and a byte slice >> inside it. All allocations are happening in Cgo, and being type casted into >> Go. No allocations are happening in Go (except a 32 MB fill slice). >> - z.NumAllocBytes is tracking memory allocated and freed by these calls. >> - Increases memory usage to 16 GB (as reported by z.NumAllocBytes). >> - Decreases it back to 1 GB. >> - Repeats this cycle. >> - On Ctrl+C, it deallocates everything and asserts that Cgo memory >> allocated is zero. >> >> I was concerned about memory fragmentation, so created a very similar C >> program which does the same thing (memtestc). >> >> Please feel free to run either of the Go or C programs. They should >> compile and run easily. >> >> Behavior: >> >> Run the program with: `go build . && ./memtest` . Go pprof heap shows 32 >> MB used, to account for the fill slice. However, RSS reported keeps roughly >> increasing every cycle. >> >> I'm using Go 1.14.4 and on it, RSS jumps to 22GB after a few cycles. >> memtestc >> (C equivalent, compiled with gcc) does not show this behavior. The RSS goes >> down to 1GB-ish every cycle. >> >> Any clues why the RSS is much higher than expected in Go and keeps >> climbing in Go? >> >> — >> Manish >> Founder, https://dgraph.io >> > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/039ca670-90fd-488d-a5ad-7bf6d3303885n%40googlegroups.com.
