Are you certain your CGo code isn’t creating threads? > On Mar 26, 2022, at 1:10 PM, Shlomi Amit <shlo...@gmail.com> wrote: > > Yes. I already monitoring the runtime stat of number of go routines (Can be > seen in the screenshot as well) and it's not increasing. > > On Sat, Mar 26, 2022, 20:40 robert engels <reng...@ix.netcom.com > <mailto:reng...@ix.netcom.com>> wrote: > Are you certain the number of Go routines is not increasing - each Go routine > requires stack. See https://tpaschalis.github.io/goroutines-size/ > <https://tpaschalis.github.io/goroutines-size/> > >> On Mar 24, 2022, at 3:18 PM, Shlomi Amit <shlo...@gmail.com >> <mailto:shlo...@gmail.com>> wrote: >> >> Hi. >> >> I’m trying to find a memory leak in my application. >> I’ve added some runtime memory stats to the logs (Heap & Stack stats.. go >> routine and heapObject counts). >> I can see that from time to time m.StackInuse is increasing without any >> obvious reason from the application perspective (At least not one that I can >> find yet). >> Here some additional runtime mem stats (Taken at the same time): >> StackInUse 56MB >> HeapSys: 147MB. >> HeapAlloc: 97MB >> >> While StackInUse and HeapSys seems to increase from time to time, HeapAlloc >> stays about the same during 6 hours while the application is running. >> There seems to be a bit of correlation between StackInUse and HeapSys, but >> overtime StackInUse increase more than HeapSys. (Is HeapSize includes >> StackInUse?) >> >> In addition to adding runtime memory logs, I’m also creating periodic heap >> profile dumps. >> My problem is that when analyzing the heap with pprof, it gives me no clue >> why StackInUse is so high. >> The pprof inuse_space shows: >> “Showing nodes accounting for 55.31MB, 93.25% of 59.31MB total” >> What this total MB represent? It doesn’t seem to match HeapAlloc, HeapSys or >> StackInUse. >> Does pprof heap profile even include StackInUse? >> >> I really need to understand where the leak is coming from, but after looking >> in many places, the memory stats are still not clear to me, and neither what >> memory stat pprof heap profile really represent. >> Note that I’m also logging HeapObjects count and I don’t see any leak there… >> It’s just StackInUse increasing from time to time (As it seems, it's always >> double itself... ~8MB->16MB->32), and HeapSys. >> >> Note that I’m also using Cgo, but my understanding is Cgo memory allocations >> will not be reflected by the runtime memory stats. Is this correct and I >> assume if runtime memory stats are increasing this is defiantly because of >> go code and not C code? >> I hope I was clear, but added a screenshot for the different memory stats. >> Marked in red the point in times that stackInUse increase. (My current >> understanding which might be wrong, is that stackInUse is not included in >> HeapSys, this is why they are stacked in the graph). >> >> I know my write is a bit messed up and you might not really be sure what's >> being asked, so in if I'll try to summarize: >> >> >> What stackInUse represents? Is it part of HeapSys? >> What HeapSys represents? (Both it and StackInUse are way more high than >> heapAlloc) >> Why pprof inuse_space doesn't seem to have any notion of stuff which was >> allocated by StackInUse or HeapSys? It seems to pretty much corelate with >> HeapAlloc. >> Since I'm also using Cgo, I do understand C memory allocation will not be >> reflected in pprof, perhaps they are being reflected by StackInUse or >> HeapInUse? >> Considering StackInUse seems to double itself in size when it increase, it >> seems like some kind of memory pre-allocation to have enough headroom than >> needed, not something which I expect to happen by C code. >> Most importantly, with the below memory stats being increased, what's the >> approach to investigate? >> Currently I'm using docker, and sometimes get OOM. What's best memory stats >> which will the most close to describe container RSS memory in use? >> (Specifically, my docker container was limited to 300MiB) >> >> I know these are many question, but I did try and go over the runtime memory >> stats documentation, and I'm still very confused, especially not after the >> actual memory stats I'm seeing. >> BTW, I'm using go 1.17.5 >> >> <Untitled.jpg> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "golang-nuts" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to golang-nuts+unsubscr...@googlegroups.com >> <mailto:golang-nuts+unsubscr...@googlegroups.com>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/golang-nuts/e7bedd37-76a2-48e9-82bb-6309c60787bbn%40googlegroups.com >> >> <https://groups.google.com/d/msgid/golang-nuts/e7bedd37-76a2-48e9-82bb-6309c60787bbn%40googlegroups.com?utm_medium=email&utm_source=footer>. >> <Untitled.jpg> > > > -- > You received this message because you are subscribed to the Google Groups > "golang-nuts" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to golang-nuts+unsubscr...@googlegroups.com > <mailto:golang-nuts+unsubscr...@googlegroups.com>. > To view this discussion on the web visit > https://groups.google.com/d/msgid/golang-nuts/CAAbwE0McE6OULqOyrjHtw6tp493%3DLeDm624_tck71RDfkN%2BOEQ%40mail.gmail.com > > <https://groups.google.com/d/msgid/golang-nuts/CAAbwE0McE6OULqOyrjHtw6tp493%3DLeDm624_tck71RDfkN%2BOEQ%40mail.gmail.com?utm_medium=email&utm_source=footer>.
-- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/A65169CD-326B-431A-8EC7-5DFF9EA56795%40ix.netcom.com.
