I also have done an image processing server in the past and used two different image library cgo bindings. Each one has its own specific way it manages the C memory and you can leak of you don't observe them. I am not familiar with the OpenCV binging though.
Are you leaking any goroutines? How extensively are you using the OpenCV features, and can you swap to another library to confirm whether it is OpenCV or not? I used to use ImageMagick bindings (which I still maintain) and now I use OpenImageIO bindings (which I authored). I haven't had memory leak issues with these libs in current production use. Justin On Sat, May 6, 2017, 12:27 AM <miha.di...@visionect.com> wrote: > Hi > > > > <https://lh3.googleusercontent.com/-rQzo0h1nFmc/WQxWfRtgdaI/AAAAAAAAAEw/fyQ4N86p2csv0ktKCb5xaIB00hNSBrgagCLcB/s1600/rss.png> > > 1. System > > > Ubuntu 16.04.1 LTS \n \l > Linux G-00017 4.4.0-57-generic #78-Ubuntu SMP Fri Dec 9 23:50:32 UTC 2016 > x86_64 x86_64 x86_64 GNU/Linux > > <https://lh3.googleusercontent.com/-rQzo0h1nFmc/WQxWfRtgdaI/AAAAAAAAAEw/fyQ4N86p2csv0ktKCb5xaIB00hNSBrgagCLcB/s1600/rss.png> > > 16GB of RAM > > App runs inside docker: Docker version 17.03.1-ce, build c6d412e > > > > <https://lh3.googleusercontent.com/-rQzo0h1nFmc/WQxWfRtgdaI/AAAAAAAAAEw/fyQ4N86p2csv0ktKCb5xaIB00hNSBrgagCLcB/s1600/rss.png> > > 2. Memory usage problem > > I am having memory usage problems in production, where things do fail > because some Go processes use much more memory than expected. > > I have prepared a test, where I am running my my server application, and > I am measuring, how memory is consumed for one process. > > I fount out that more and more system memory is consumed with time. On the > graph below, you can see how RSS has risen from ~0.5GB up to ~6Gb in a few > days. > > > > <https://lh3.googleusercontent.com/-rQzo0h1nFmc/WQxWfRtgdaI/AAAAAAAAAEw/fyQ4N86p2csv0ktKCb5xaIB00hNSBrgagCLcB/s1600/rss.png> > > > > <https://lh3.googleusercontent.com/-rQzo0h1nFmc/WQxWfRtgdaI/AAAAAAAAAEw/fyQ4N86p2csv0ktKCb5xaIB00hNSBrgagCLcB/s1600/rss.png> > > > <https://lh3.googleusercontent.com/-rQzo0h1nFmc/WQxWfRtgdaI/AAAAAAAAAEw/fyQ4N86p2csv0ktKCb5xaIB00hNSBrgagCLcB/s1600/rss.png> > > > > > > > > The app does a lot of image processing for the images from remote source > (in the range of 100 images each second). All this processing, means a lot > of allocations, so naturally I tough, this must be memory a memory leak. > > So I run pprof (see attachment: pprof.log), which reports heap usage below > 300Mb. This heap usage, doesn't seam to change much over the time. > > Then, there are some CGO bindings with OpenCV, so I thought there must be > some leak in C code. I run the thing with Valgrind, which didn't find > anything except some minor leak, which I had already fixed by now. > > > > <https://lh3.googleusercontent.com/-rQzo0h1nFmc/WQxWfRtgdaI/AAAAAAAAAEw/fyQ4N86p2csv0ktKCb5xaIB00hNSBrgagCLcB/s1600/rss.png> > > I also run the app with GOGC=50, but it didn't help. > > > I am looking for the explanation what is happening, and how this can be > solved? > > > > Thanks! > > -- > You received this message because you are subscribed to the Google Groups > "golang-nuts" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to golang-nuts+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
