On Friday, 5 October 2018 12:56:50 UTC+2, Thomas S wrote: > > > My service crash because "panic - out of memory", and I'm not sure how to > monitore it. > I just want to know, when it crashes, where are the main memory > allocations, to understand the problem. > > It is pretty sudden, I can hardly use pprof in real time. And has it > crashed, pprof can't produce the final file for after-run analysis. >
For some of our systems at work, we use a fairly simple approach. In our main function we spin up a goroutine that with a certain frequency reads runtime.ReadMemStats. When it detects a rise in the value above a set threshold it will do a pprof.WriteHeapProfile() to a file with time+date+alloc size in the file name. You can add things like a startup quiet period a minimum threshold, etc. If your peaks are not too sudden you should be able to catch dump with whatever problematic code you have. Dumps are not too big and we have a script that cleans out dumps that are more than a month old. I cannot share the code, but it should be fairly straightforward. This has helped us on a few occasions where we had some similar problems. /Klaus -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
