Hi
<https://lh3.googleusercontent.com/-rQzo0h1nFmc/WQxWfRtgdaI/AAAAAAAAAEw/fyQ4N86p2csv0ktKCb5xaIB00hNSBrgagCLcB/s1600/rss.png> 1. System Ubuntu 16.04.1 LTS \n \l Linux G-00017 4.4.0-57-generic #78-Ubuntu SMP Fri Dec 9 23:50:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux <https://lh3.googleusercontent.com/-rQzo0h1nFmc/WQxWfRtgdaI/AAAAAAAAAEw/fyQ4N86p2csv0ktKCb5xaIB00hNSBrgagCLcB/s1600/rss.png> 16GB of RAM App runs inside docker: Docker version 17.03.1-ce, build c6d412e <https://lh3.googleusercontent.com/-rQzo0h1nFmc/WQxWfRtgdaI/AAAAAAAAAEw/fyQ4N86p2csv0ktKCb5xaIB00hNSBrgagCLcB/s1600/rss.png> 2. Memory usage problem I am having memory usage problems in production, where things do fail because some Go processes use much more memory than expected. I have prepared a test, where I am running my my server application, and I am measuring, how memory is consumed for one process. I fount out that more and more system memory is consumed with time. On the graph below, you can see how RSS has risen from ~0.5GB up to ~6Gb in a few days. <https://lh3.googleusercontent.com/-rQzo0h1nFmc/WQxWfRtgdaI/AAAAAAAAAEw/fyQ4N86p2csv0ktKCb5xaIB00hNSBrgagCLcB/s1600/rss.png> <https://lh3.googleusercontent.com/-rQzo0h1nFmc/WQxWfRtgdaI/AAAAAAAAAEw/fyQ4N86p2csv0ktKCb5xaIB00hNSBrgagCLcB/s1600/rss.png> <https://lh3.googleusercontent.com/-rQzo0h1nFmc/WQxWfRtgdaI/AAAAAAAAAEw/fyQ4N86p2csv0ktKCb5xaIB00hNSBrgagCLcB/s1600/rss.png> The app does a lot of image processing for the images from remote source (in the range of 100 images each second). All this processing, means a lot of allocations, so naturally I tough, this must be memory a memory leak. So I run pprof (see attachment: pprof.log), which reports heap usage below 300Mb. This heap usage, doesn't seam to change much over the time. Then, there are some CGO bindings with OpenCV, so I thought there must be some leak in C code. I run the thing with Valgrind, which didn't find anything except some minor leak, which I had already fixed by now. <https://lh3.googleusercontent.com/-rQzo0h1nFmc/WQxWfRtgdaI/AAAAAAAAAEw/fyQ4N86p2csv0ktKCb5xaIB00hNSBrgagCLcB/s1600/rss.png> I also run the app with GOGC=50, but it didn't help. I am looking for the explanation what is happening, and how this can be solved? Thanks! -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
$ ls -la total 30208 drwxrwxr-x 2 miha miha 4096 apr 19 09:45 . drwxr-xr-x 42 miha miha 12288 apr 18 14:10 .. -rwxr-xr-x 1 miha miha 12505040 mar 22 10:04 engine -rw-rw-r-- 1 miha miha 2274390 apr 18 14:10 engine1.heap -rw-rw-r-- 1 miha miha 2291245 apr 18 14:39 engine2.heap -rw-rw-r-- 1 miha miha 2293808 apr 18 15:22 engine3.heap -rw-rw-r-- 1 miha miha 2293874 apr 18 15:34 engine4.heap -rw-rw-r-- 1 miha miha 2293838 apr 18 15:41 engine5.heap -rw-rw-r-- 1 miha miha 2294298 apr 18 15:51 engine6.heap -rw-rw-r-- 1 miha miha 2296224 apr 18 16:05 engine7.heap -rw-rw-r-- 1 miha miha 2344997 apr 19 09:45 engine8.heap -rw-rw-r-- 1 miha miha 119 apr 18 14:39 pprof_cmd $ go tool pprof -inuse_space -top engine engine1.heap269.50MB of 271.78MB total (99.16%) Dropped 424 nodes (cum <= 1.36MB) flat flat% sum% cum cum% 108.29MB 39.84% 39.84% 108.29MB 39.84% gopkg.in/vmihailenco/msgpack%2ev2.(*Decoder).DecodeBytes 103.34MB 38.02% 77.87% 103.34MB 38.02% render.(*DisplaysEncoder).AddRectsBackend 32.81MB 12.07% 89.94% 32.81MB 12.07% render.(*DisplaysEncoder).UpdateDisplaysDimensions 16.80MB 6.18% 96.12% 16.80MB 6.18% driver/imgproc.CEightToXBit 2.97MB 1.09% 97.22% 2.97MB 1.09% driver/imgproc.CvRotateOrtoghonal 2.24MB 0.83% 98.04% 2.24MB 0.83% driver/imgproc.CutRectangle 1.55MB 0.57% 98.61% 1.55MB 0.57% io.copyBuffer 1.50MB 0.55% 99.16% 1.50MB 0.55% time.NewTimer 0 0% 99.16% 121.86MB 44.84% backend.(*ClientGroup).ReceiverLoop 0 0% 99.16% 121.86MB 44.84% backend.(*ClientGroup).Response 0 0% 99.16% 121.86MB 44.84% backend.(*ClientGroup).ResponseReceiver 0 0% 99.16% 2.24MB 0.83% backend/webkit.InitIPCAlt.func1 0 0% 99.16% 108.29MB 39.84% backend/webkit.InitIPCAlt.func3 0 0% 99.16% 16.80MB 6.18% cache.CheckCache 0 0% 99.16% 16.80MB 6.18% driver.(*DisplayGeneric).Filter 0 0% 99.16% 16.80MB 6.18% driver.Generic.Filter 0 0% 99.16% 108.29MB 39.84% gopkg.in/vmihailenco/msgpack%2ev2.(*Decoder).Decode 0 0% 99.16% 108.29MB 39.84% gopkg.in/vmihailenco/msgpack%2ev2.(*Decoder).DecodeValue 0 0% 99.16% 108.29MB 39.84% gopkg.in/vmihailenco/msgpack%2ev2.(*Decoder).bytesValue 0 0% 99.16% 108.29MB 39.84% gopkg.in/vmihailenco/msgpack%2ev2.(*Decoder).decode 0 0% 99.16% 108.29MB 39.84% gopkg.in/vmihailenco/msgpack%2ev2.(*Decoder).structValue 0 0% 99.16% 108.29MB 39.84% gopkg.in/vmihailenco/msgpack%2ev2.(*field).DecodeValue 0 0% 99.16% 108.29MB 39.84% gopkg.in/vmihailenco/msgpack%2ev2.Unmarshal 0 0% 99.16% 108.29MB 39.84% gopkg.in/vmihailenco/msgpack%2ev2.decodeBytesValue 0 0% 99.16% 108.29MB 39.84% gopkg.in/vmihailenco/msgpack%2ev2.decodePtrValue 0 0% 99.16% 108.29MB 39.84% gopkg.in/vmihailenco/msgpack%2ev2.decodeStructValue 0 0% 99.16% 1.55MB 0.57% io.Copy 0 0% 99.16% 108.29MB 39.84% kipc.(*Bus).Listen 0 0% 99.16% 108.29MB 39.84% kipc.(*Bus).handleIncoming 0 0% 99.16% 1.50MB 0.55% kipc.(*Gateway).SendImage 0 0% 99.16% 108.29MB 39.84% kipc.(*MsgPackBackend).Unmarshal 0 0% 99.16% 1.50MB 0.55% kipc.(*Service).CallTimeout 0 0% 99.16% 33.32MB 12.26% kipc.(*Service).handleRequest.func1 0 0% 99.16% 33.32MB 12.26% kipc.(*Service).pickupRequest 0 0% 99.16% 108.29MB 39.84% kipc.createObject 0 0% 99.16% 121.86MB 44.84% main.(*Client).Response 0 0% 99.16% 32.81MB 12.07% main.(*Client).setRenderHelperUnlocked 0 0% 99.16% 2.27MB 0.83% main.(*sender).Send 0 0% 99.16% 3.52MB 1.29% main.(*sender).start.func1 0 0% 99.16% 32.81MB 12.07% main.NewClient 0 0% 99.16% 33.32MB 12.26% main.announce 0 0% 99.16% 18.52MB 6.81% render.(*DisplaysEncoder).GenerateImagePacket 0 0% 99.16% 19.77MB 7.28% render.(*DisplaysEncoder).generateImagePacketUnlocked 0 0% 99.16% 32.81MB 12.07% render.(*DisplaysEncoder).initDisplayDimensions 0 0% 99.16% 19.77MB 7.28% render.(*DisplaysEncoder).syncDisplaysRectsUnlocked 0 0% 99.16% 19.77MB 7.28% render.(*displayRects).sync 0 0% 99.16% 19.77MB 7.28% render.(*displayRects).syncUnlocked 0 0% 99.16% 32.81MB 12.07% render.NewDisplaysEncoder 0 0% 99.16% 271.28MB 99.82% runtime.goexit 0 0% 99.16% 1.50MB 0.55% time.After miha@G-00017:~/engine_prof$ go tool pprof -inuse_space --base engine1.heap engine engine2.heap Entering interactive mode (type "help" for commands) (pprof) top 24.10MB of 23.10MB total ( 100%) Dropped 417 nodes (cum <= 0.12MB) Showing top 10 nodes out of 59 (cum >= 8.44MB) flat flat% sum% cum cum% 5.95MB 25.77% 25.77% 5.95MB 25.77% github.com/pebbe/zmq4.(*Socket).RecvBytes 4.49MB 19.46% 45.23% 4.49MB 19.46% driver/imgproc.CutRectangle 3.49MB 15.13% 60.36% 3.49MB 15.13% bytes.makeSlice 2.97MB 12.85% 73.21% 2.97MB 12.85% driver/imgproc.CvRotateOrtoghonal 2.97MB 12.85% 86.06% 2.97MB 12.85% gopkg.in/vmihailenco/msgpack%2ev2.(*Decoder).DecodeBytes 2.30MB 9.96% 96.02% 2.30MB 9.96% driver/imgproc.CEightToXBit 1.92MB 8.31% 100% 1.92MB 8.31% driver/imgproc.CvToGrayImage 0 0% 100% 8.44MB 36.54% backend.(*ClientGroup).ReceiverLoop 0 0% 100% 8.44MB 36.54% backend.(*ClientGroup).Response 0 0% 100% 8.44MB 36.54% backend.(*ClientGroup).ResponseReceiver (pprof) ^C miha@G-00017:~/engine_prof$ go tool pprof -inuse_space --base engine1.heap engine engine3.heap Entering interactive mode (type "help" for commands) (pprof) top 1153.39kB of -5053.97kB total ( -23%) flat flat% sum% cum cum% 641.34kB -13% -13% 641.34kB -13% bytes.makeSlice 512.05kB -10% -23% 512.05kB -10% time.NewTimer 0 -0% -23% 0 -0% backend.(*ClientGroup).BackendNotify 0 -0% -23% 0 -0% backend.(*ClientGroup).Request 0 -0% -23% 0 -0% backend.(*ClientGroup).Update 0 -0% -23% 0 -0% backend.(*ClientGroup).notifyAboutClient 0 -0% -23% 0 -0% backend.(*EventGroup).Event 0 -0% -23% 0 -0% backend.(*EventGroup).EventReceiver.func1 0 -0% -23% 0 -0% backend.NewDataEvent 0 -0% -23% 0 -0% backend.init (pprof) ^C miha@G-00017:~/engine_prof$ go tool pprof -inuse_space --base engine1.heap -top engine engine4.heap 22316.61kB of 21532.85kB total ( 100%) Dropped 424 nodes (cum <= 107.66kB) flat flat% sum% cum cum% 5008.01kB 23.26% 23.26% 5008.01kB 23.26% driver/imgproc.CEightToXBit 4065.95kB 18.88% 42.14% 4065.95kB 18.88% driver/imgproc.CutRectangle 4022.88kB 18.68% 60.82% 4022.88kB 18.68% driver/imgproc.CvRotateOrtoghonal 3775.82kB 17.54% 78.36% 3775.82kB 17.54% github.com/pebbe/zmq4.(*Socket).RecvBytes 1965.46kB 9.13% 87.49% 1965.46kB 9.13% driver/imgproc.CvToGrayImage 1455.56kB 6.76% 94.25% 1455.56kB 6.76% bytes.makeSlice 982.73kB 4.56% 98.81% 982.73kB 4.56% driver/imgproc.CToEightBit 528.17kB 2.45% 100% 528.17kB 2.45% io.copyBuffer 512.05kB 2.38% 100% 512.05kB 2.38% time.NewTimer 0 0% 100% 12533.98kB 58.21% backend.(*ClientGroup).ReceiverLoop 0 0% 100% 12533.98kB 58.21% backend.(*ClientGroup).Response 0 0% 100% 12533.98kB 58.21% backend.(*ClientGroup).ResponseReceiver 0 0% 100% 4065.95kB 18.88% backend/webkit.InitIPCAlt.func1 0 0% 100% 1455.56kB 6.76% bytes.(*Buffer).Write 0 0% 100% 1455.56kB 6.76% bytes.(*Buffer).grow 0 0% 100% 6973.47kB 32.39% cache.CheckCache 0 0% 100% 7956.20kB 36.95% driver.(*DispGeneric).Filter 0 0% 100% 7956.20kB 36.95% driver.DispGeneric.Filter 0 0% 100% 727.78kB 3.38% github.com/pebbe/zmq4.(*Reactor).Run 0 0% 100% 3775.82kB 17.54% github.com/pebbe/zmq4.(*Socket).RecvMessageBytes 0 0% 100% 1455.56kB 6.76% gopkg.in/vmihailenco/msgpack%2ev2.(*Encoder).Encode 0 0% 100% 1455.56kB 6.76% gopkg.in/vmihailenco/msgpack%2ev2.(*Encoder).EncodeBytes 0 0% 100% 1455.56kB 6.76% gopkg.in/vmihailenco/msgpack%2ev2.(*Encoder).EncodeValue 0 0% 100% 1455.56kB 6.76% gopkg.in/vmihailenco/msgpack%2ev2.(*Encoder).encode 0 0% 100% 1455.56kB 6.76% gopkg.in/vmihailenco/msgpack%2ev2.(*Encoder).encodeArray 0 0% 100% 1455.56kB 6.76% gopkg.in/vmihailenco/msgpack%2ev2.(*Encoder).encodeSlice 0 0% 100% 1455.56kB 6.76% gopkg.in/vmihailenco/msgpack%2ev2.(*Encoder).encodeStruct 0 0% 100% 1455.56kB 6.76% gopkg.in/vmihailenco/msgpack%2ev2.(*Encoder).write 0 0% 100% 1455.56kB 6.76% gopkg.in/vmihailenco/msgpack%2ev2.(*field).EncodeValue 0 0% 100% 1455.56kB 6.76% gopkg.in/vmihailenco/msgpack%2ev2.Marshal 0 0% 100% 1455.56kB 6.76% gopkg.in/vmihailenco/msgpack%2ev2.encodeBytesValue 0 0% 100% 1455.56kB 6.76% gopkg.in/vmihailenco/msgpack%2ev2.encodePtrValue 0 0% 100% 1455.56kB 6.76% gopkg.in/vmihailenco/msgpack%2ev2.encodeSliceValue 0 0% 100% 1455.56kB 6.76% gopkg.in/vmihailenco/msgpack%2ev2.encodeStructValue 0 0% 100% 528.17kB 2.45% io.Copy 0 0% 100% 3048.04kB 14.16% kipc.(*Bus).Worker 0 0% 100% 512.05kB 2.38% kipc.(*Gateway).SendImage 0 0% 100% 512.05kB 2.38% kipc.(*Service).CallTimeout 0 0% 100% 727.78kB 3.38% kipc.(*Service).Requester.func1 0 0% 100% 727.78kB 3.38% kipc.(*Service).dealerReceiver 0 0% 100% 727.78kB 3.38% kipc.(*Service).dealerReceiver.func1 0 0% 100% 12533.98kB 58.21% main.(*Client).Response 0 0% 100% 1967.60kB 9.14% main.(*sender).Send 0 0% 100% 1412.70kB 6.56% main.(*sender).start.func1 0 0% 100% 12533.98kB 58.21% render.(*DisplaysEncoder).GenerateImagePacket 0 0% 100% 1965.46kB 9.13% render.(*DisplaysEncoder).applyRectsToImageUnlocked 0 0% 100% 11979.07kB 55.63% render.(*DisplaysEncoder).generateImagePacketUnlocked 0 0% 100% 11979.07kB 55.63% render.(*DisplaysEncoder).syncDisplaysRectsUnlocked 0 0% 100% 10013.62kB 46.50% render.(*displayRects).sync 0 0% 100% 10013.62kB 46.50% render.(*displayRects).syncUnlocked 0 0% 100% 21532.85kB 100% runtime.goexit 0 0% 100% 512.05kB 2.38% time.After miha@G-00017:~/engine_prof$ go tool pprof -inuse_space --base engine1.heap -top engine engine5.heap 2105.19kB of 1179.02kB total ( 100%) Dropped 451 nodes (cum <= 5.89kB) flat flat% sum% cum cum% 2096.90kB 100% 100% 2096.90kB 100% driver/imgproc.CEightToXBit 8.29kB 0.7% 100% 8.29kB 0.7% driver/imgproc.CutRectangle 0 0% 100% 1455.56kB 100% backend.(*ClientGroup).ReceiverLoop 0 0% 100% 1455.56kB 100% backend.(*ClientGroup).Response 0 0% 100% 1455.56kB 100% backend.(*ClientGroup).ResponseReceiver 0 0% 100% 8.29kB 0.7% backend/webkit.InitIPCAlt.func1 0 0% 100% 641.34kB 54.40% bytes.(*Buffer).Write 0 0% 100% 2096.90kB 100% cache.CheckCache 0 0% 100% 2096.90kB 100% driver.(*DispGeneric).Filter 0 0% 100% 2096.90kB 100% driver.DispGeneric.Filter 0 0% 100% 641.34kB 54.40% gopkg.in/vmihailenco/msgpack%2ev2.(*Encoder).EncodeBytes 0 0% 100% 641.34kB 54.40% gopkg.in/vmihailenco/msgpack%2ev2.(*Encoder).write 0 0% 100% 641.34kB 54.40% gopkg.in/vmihailenco/msgpack%2ev2.encodeBytesValue 0 0% 100% 1455.56kB 100% main.(*Client).Response 0 0% 100% 641.34kB 54.40% main.(*Client).refresh 0 0% 100% 641.34kB 54.40% main.(*Client).refreshAsynchronous 0 0% 100% 498.92kB 42.32% main.(*sender).start.func1 0 0% 100% 641.34kB 54.40% main.NewClient.func2 0 0% 100% 641.34kB 54.40% render.(*DisplaysEncoder).DirtyImagePacket 0 0% 100% 1455.56kB 100% render.(*DisplaysEncoder).GenerateImagePacket 0 0% 100% 2096.90kB 100% render.(*DisplaysEncoder).generateImagePacketUnlocked 0 0% 100% 2096.90kB 100% render.(*DisplaysEncoder).syncDisplaysRectsUnlocked 0 0% 100% 2096.90kB 100% render.(*displayRects).sync 0 0% 100% 2096.90kB 100% render.(*displayRects).syncUnlocked 0 0% 100% 1179.02kB 100% runtime.goexit miha@G-00017:~/engine_prof$ go tool pprof -inuse_space --base engine1.heap -top engine engine6.heap 28110.37kB of 25518.79kB total ( 100%) Dropped 436 nodes (cum <= 127.59kB) flat flat% sum% cum cum% 6096.08kB 23.89% 23.89% 6096.08kB 23.89% github.com/pebbe/zmq4.(*Socket).RecvBytes 6080.30kB 23.83% 47.72% 6080.30kB 23.83% gopkg.in/vmihailenco/msgpack%2ev2.(*Decoder).DecodeBytes 5822.22kB 22.82% 70.53% 5822.22kB 22.82% driver/imgproc.CEightToXBit 4065.95kB 15.93% 86.46% 4065.95kB 15.93% driver/imgproc.CutRectangle 4022.88kB 15.76% 100% 4022.88kB 15.76% driver/imgproc.CvRotateOrtoghonal 982.73kB 3.85% 100% 982.73kB 3.85% driver/imgproc.CToEightBit 528.17kB 2.07% 100% 528.17kB 2.07% io.copyBuffer 512.04kB 2.01% 100% 512.04kB 2.01% backend/webkit.(*WebkitBackend).RecvResponse 0 0% 100% 12622.55kB 49.46% backend.(*ClientGroup).ReceiverLoop 0 0% 100% 12110.51kB 47.46% backend.(*ClientGroup).Response 0 0% 100% 12622.55kB 49.46% backend.(*ClientGroup).ResponseReceiver 0 0% 100% 4065.95kB 15.93% backend/webkit.InitIPCAlt.func1 0 0% 100% 6080.30kB 23.83% backend/webkit.InitIPCAlt.func3 0 0% 100% 5822.22kB 22.82% cache.CheckCache 0 0% 100% 6804.95kB 26.67% driver.(*DispGeneric).Filter 0 0% 100% 6804.95kB 26.67% driver.DispGeneric.Filter 0 0% 100% 6096.08kB 23.89% github.com/pebbe/zmq4.(*Socket).RecvMessageBytes 0 0% 100% 6080.30kB 23.83% gopkg.in/vmihailenco/msgpack%2ev2.(*Decoder).Decode 0 0% 100% 6080.30kB 23.83% gopkg.in/vmihailenco/msgpack%2ev2.(*Decoder).DecodeValue 0 0% 100% 6080.30kB 23.83% gopkg.in/vmihailenco/msgpack%2ev2.(*Decoder).bytesValue 0 0% 100% 6080.30kB 23.83% gopkg.in/vmihailenco/msgpack%2ev2.(*Decoder).decode 0 0% 100% 6080.30kB 23.83% gopkg.in/vmihailenco/msgpack%2ev2.(*Decoder).structValue 0 0% 100% 6080.30kB 23.83% gopkg.in/vmihailenco/msgpack%2ev2.(*field).DecodeValue 0 0% 100% 6080.30kB 23.83% gopkg.in/vmihailenco/msgpack%2ev2.Unmarshal 0 0% 100% 6080.30kB 23.83% gopkg.in/vmihailenco/msgpack%2ev2.decodeBytesValue 0 0% 100% 6080.30kB 23.83% gopkg.in/vmihailenco/msgpack%2ev2.decodePtrValue 0 0% 100% 6080.30kB 23.83% gopkg.in/vmihailenco/msgpack%2ev2.decodeStructValue 0 0% 100% 528.17kB 2.07% io.Copy 0 0% 100% 6080.30kB 23.83% kipc.(*Bus).Listen 0 0% 100% 6096.08kB 23.89% kipc.(*Bus).Worker 0 0% 100% 6080.30kB 23.83% kipc.(*Bus).handleIncoming 0 0% 100% 6080.30kB 23.83% kipc.(*MsgPackBackend).Unmarshal 0 0% 100% 6080.30kB 23.83% kipc.createObject 0 0% 100% 12110.51kB 47.46% main.(*Client).Response 0 0% 100% 12110.51kB 47.46% render.(*DisplaysEncoder).GenerateImagePacket 0 0% 100% 1965.46kB 7.70% render.(*DisplaysEncoder).applyRectsToImageUnlocked 0 0% 100% 10827.83kB 42.43% render.(*DisplaysEncoder).generateImagePacketUnlocked 0 0% 100% 10827.83kB 42.43% render.(*DisplaysEncoder).syncDisplaysRectsUnlocked 0 0% 100% 8862.37kB 34.73% render.(*displayRects).sync 0 0% 100% 8862.37kB 34.73% render.(*displayRects).syncUnlocked 0 0% 100% 25518.79kB 100% runtime.goexit miha@G-00017:~/engine_prof$ go tool pprof -inuse_space --base engine1.heap -top engine engine7.heap 6626.80kB of -6.16kB total (-1.1e+05%) flat flat% sum% cum cum% 2183.33kB -3.5e+04% -3.5e+04% 2183.33kB -3.5e+04% driver/imgproc.CEightToXBit 1929.05kB -3.1e+04% -6.7e+04% 1929.05kB -3.1e+04% gopkg.in/vmihailenco/msgpack%2ev2.(*Decoder).DecodeBytes 1929.05kB -3.1e+04% -9.8e+04% 1929.05kB -3.1e+04% github.com/pebbe/zmq4.(*Socket).RecvBytes 585.36kB -9.5e+03% -1.1e+05% 585.36kB -9.5e+03% bytes.makeSlice 0 -0% -1.1e+05% 0 -0% backend.(*ClientGroup).BackendNotify 0 -0% -1.1e+05% 0 -0% backend.(*ClientGroup).Request 0 -0% -1.1e+05% 0 -0% backend.(*ClientGroup).Update 0 -0% -1.1e+05% 0 -0% backend.(*ClientGroup).notifyAboutClient 0 -0% -1.1e+05% 0 -0% backend.(*EventGroup).Event 0 -0% -1.1e+05% 0 -0% backend.(*EventGroup).EventReceiver.func1 0 -0% -1.1e+05% 0 -0% backend.NewDataEvent 0 -0% -1.1e+05% 0 -0% backend.init 0 -0% -1.1e+05% 0 -0% backend/webkit.(*WebkitBackend).Notify 0 -0% -1.1e+05% 0 -0% backend/webkit.(*WebkitBackend).RecvResponse 0 -0% -1.1e+05% 0 -0% backend/webkit.(*WebkitBackend).SendRequest 0 -0% -1.1e+05% 0 -0% backend/webkit.(*WebkitBackend).clientsBordersUpdate 0 -0% -1.1e+05% 0 -0% backend/webkit.(*WkProcess).CheckRSSLimit 0 -0% -1.1e+05% 0 -0% backend/webkit.(*WkProcess).KillWebkitUnlocked 0 -0% -1.1e+05% 0 -0% backend/webkit.(*WkProcess).StartWebkit 0 -0% -1.1e+05% 0 -0% backend/webkit.(*WkProcess).WaitWebkit 0 -0% -1.1e+05% 0 -0% backend/webkit.(*X11Instance).Exec 0 -0% -1.1e+05% 0 -0% backend/webkit.(*X11Instance).GetFree 0 -0% -1.1e+05% 0 -0% backend/webkit.(*X11Instance).IsRunning 0 -0% -1.1e+05% 0 -0% backend/webkit.(*X11Instance).Run 0 -0% -1.1e+05% 0 -0% backend/webkit.(*X11Instance).getFree 0 -0% -1.1e+05% 0 -0% backend/webkit.(*X11Instance).getFreeHelper 0 -0% -1.1e+05% 0 -0% backend/webkit.(*X11Instance).isRunning 0 -0% -1.1e+05% 0 -0% backend/webkit.(*X11Instance).removePredictionisIfExited 0 -0% -1.1e+05% 0 -0% backend/webkit.(*cookiefsState).isMounted 0 -0% -1.1e+05% 0 -0% backend/webkit.GetDescription 0 -0% -1.1e+05% 0 -0% backend/webkit.InitIPCAlt.func2 0 -0% -1.1e+05% 1929.05kB -3.1e+04% backend/webkit.InitIPCAlt.func3 0 -0% -1.1e+05% 0 -0% backend/webkit.New 0 -0% -1.1e+05% 0 -0% backend/webkit.NewFromFields 0 -0% -1.1e+05% 0 -0% backend/webkit.NewWkProcess.func1 0 -0% -1.1e+05% 0 -0% backend/webkit.RSSWatchdog 0 -0% -1.1e+05% 0 -0% backend/webkit.SendWebKitAddDevices 0 -0% -1.1e+05% 0 -0% backend/webkit.SendWebkitReloadTimeout 0 -0% -1.1e+05% 0 -0% backend/webkit.SendWebkitUrl 0 -0% -1.1e+05% 0 -0% backend/webkit.SendWebkitViewChange 0 -0% -1.1e+05% 0 -0% backend/webkit.WebKitWebProcessWatchdog 0 -0% -1.1e+05% 0 -0% backend/webkit.WebkitPublish 0 -0% -1.1e+05% 0 -0% backend/webkit.X11Exec 0 -0% -1.1e+05% 0 -0% backend/webkit.X11GetDisplayUnlocked 0 -0% -1.1e+05% 0 -0% backend/webkit.exeSimpleCmd 0 -0% -1.1e+05% 0 -0% backend/webkit.getProcessNoThreads 0 -0% -1.1e+05% 0 -0% backend/webkit.getResidentMemory 0 -0% -1.1e+05% 0 -0% bitbucket.org/StephaneBunel/xxhash-go.Checksum32 0 -0% -1.1e+05% 0 -0% bufio.(*Reader).ReadBytes 0 -0% -1.1e+05% 0 -0% bufio.(*Reader).ReadString 0 -0% -1.1e+05% 0 -0% bufio.(*Writer).Write 0 -0% -1.1e+05% 0 -0% bufio.(*Writer).flush 0 -0% -1.1e+05% 0 -0% bytes.(*Buffer).ReadFrom 0 -0% -1.1e+05% 1369.12kB -2.2e+04% bytes.(*Buffer).Write 0 -0% -1.1e+05% 0 -0% bytes.(*Buffer).WriteByte 0 -0% -1.1e+05% 585.36kB -9.5e+03% bytes.(*Buffer).grow 0 -0% -1.1e+05% 0 -0% cache.(*CacheKey).String 0 -0% -1.1e+05% 0 -0% cache.CacheKey.String 0 -0% -1.1e+05% 2183.33kB -3.5e+04% cache.CheckCache 0 -0% -1.1e+05% 0 -0% cache.addToCache 0 -0% -1.1e+05% 0 -0% cache.cacheAge 0 -0% -1.1e+05% 0 -0% cache.cacheMakeSpace 0 -0% -1.1e+05% 0 -0% cache.glob..func1 0 -0% -1.1e+05% 0 -0% cache.removeFromCache 0 -0% -1.1e+05% 0 -0% common.Display.RectangleTranslateToReal 0 -0% -1.1e+05% 0 -0% common.Display.RectangleTranslateToSession 0 -0% -1.1e+05% 0 -0% common.Displays.Borders 0 -0% -1.1e+05% 0 -0% common.Grep 0 -0% -1.1e+05% 0 -0% container/list.(*List).PushBack 0 -0% -1.1e+05% 0 -0% context.WithDeadline 0 -0% -1.1e+05% 0 -0% context.WithTimeout 0 -0% -1.1e+05% 0 -0% crypto/rand.init 0 -0% -1.1e+05% 2183.33kB -3.5e+04% driver.(*DispGeneric).Filter 0 -0% -1.1e+05% 0 -0% driver.(*DispGeneric).Settings 0 -0% -1.1e+05% 2183.33kB -3.5e+04% driver.DispGeneric.Filter 0 -0% -1.1e+05% 0 -0% driver.SizeRectFor16BitQuantEncoding 0 -0% -1.1e+05% 0 -0% driver.driverSizeRectFor16BitQuantEncodingBothDirections 0 -0% -1.1e+05% 0 -0% driver/imgproc.CToEightBit 0 -0% -1.1e+05% 0 -0% driver/imgproc.CvDrawOn 0 -0% -1.1e+05% 0 -0% driver/imgproc.CvDrawOn.func1 miha@G-00017:~/engine_prof$ go tool pprof -inuse_space --base engine1.heap -top engine engine8.heap 12790.25kB of 10606.92kB total ( 100%) Dropped 421 nodes (cum <= 53.03kB) flat flat% sum% cum cum% 3048.04kB 28.74% 28.74% 3048.04kB 28.74% github.com/pebbe/zmq4.(*Socket).RecvBytes 3040.15kB 28.66% 57.40% 3040.15kB 28.66% driver/imgproc.CvRotateOrtoghonal 3040.15kB 28.66% 86.06% 3040.15kB 28.66% gopkg.in/vmihailenco/msgpack%2ev2.(*Decoder).DecodeBytes 1224.27kB 11.54% 97.60% 1224.27kB 11.54% driver/imgproc.CutRectangle 982.73kB 9.26% 100% 982.73kB 9.26% driver/imgproc.CvToGrayImage 926.75kB 8.74% 100% 926.75kB 8.74% bytes.makeSlice 528.17kB 4.98% 100% 528.17kB 4.98% io.copyBuffer 0 0% 100% 3122.23kB 29.44% backend.(*ClientGroup).ReceiverLoop 0 0% 100% 3122.23kB 29.44% backend.(*ClientGroup).Response 0 0% 100% 3122.23kB 29.44% backend.(*ClientGroup).ResponseReceiver 0 0% 100% 1224.27kB 11.54% backend/webkit.InitIPCAlt.func1 0 0% 100% 3040.15kB 28.66% backend/webkit.InitIPCAlt.func3 0 0% 100% 727.78kB 6.86% bytes.(*Buffer).Write 0 0% 100% 198.97kB 1.88% bytes.(*Buffer).WriteString 0 0% 100% 926.75kB 8.74% bytes.(*Buffer).grow 0 0% 100% 3048.04kB 28.74% github.com/pebbe/zmq4.(*Socket).RecvMessageBytes 0 0% 100% 3040.15kB 28.66% gopkg.in/vmihailenco/msgpack%2ev2.(*Decoder).Decode 0 0% 100% 3040.15kB 28.66% gopkg.in/vmihailenco/msgpack%2ev2.(*Decoder).DecodeValue 0 0% 100% 3040.15kB 28.66% gopkg.in/vmihailenco/msgpack%2ev2.(*Decoder).bytesValue 0 0% 100% 3040.15kB 28.66% gopkg.in/vmihailenco/msgpack%2ev2.(*Decoder).decode 0 0% 100% 3040.15kB 28.66% gopkg.in/vmihailenco/msgpack%2ev2.(*Decoder).structValue 0 0% 100% 926.75kB 8.74% gopkg.in/vmihailenco/msgpack%2ev2.(*Encoder).Encode 0 0% 100% 727.78kB 6.86% gopkg.in/vmihailenco/msgpack%2ev2.(*Encoder).EncodeBytes 0 0% 100% 198.97kB 1.88% gopkg.in/vmihailenco/msgpack%2ev2.(*Encoder).EncodeString 0 0% 100% 926.75kB 8.74% gopkg.in/vmihailenco/msgpack%2ev2.(*Encoder).EncodeValue 0 0% 100% 926.75kB 8.74% gopkg.in/vmihailenco/msgpack%2ev2.(*Encoder).encode 0 0% 100% 926.75kB 8.74% gopkg.in/vmihailenco/msgpack%2ev2.(*Encoder).encodeArray 0 0% 100% 926.75kB 8.74% gopkg.in/vmihailenco/msgpack%2ev2.(*Encoder).encodeSlice 0 0% 100% 926.75kB 8.74% gopkg.in/vmihailenco/msgpack%2ev2.(*Encoder).encodeStruct 0 0% 100% 727.78kB 6.86% gopkg.in/vmihailenco/msgpack%2ev2.(*Encoder).write 0 0% 100% 198.97kB 1.88% gopkg.in/vmihailenco/msgpack%2ev2.(*Encoder).writeString 0 0% 100% 3040.15kB 28.66% gopkg.in/vmihailenco/msgpack%2ev2.(*field).DecodeValue 0 0% 100% 926.75kB 8.74% gopkg.in/vmihailenco/msgpack%2ev2.(*field).EncodeValue 0 0% 100% 926.75kB 8.74% gopkg.in/vmihailenco/msgpack%2ev2.Marshal 0 0% 100% 3040.15kB 28.66% gopkg.in/vmihailenco/msgpack%2ev2.Unmarshal 0 0% 100% 3040.15kB 28.66% gopkg.in/vmihailenco/msgpack%2ev2.decodeBytesValue 0 0% 100% 3040.15kB 28.66% gopkg.in/vmihailenco/msgpack%2ev2.decodePtrValue 0 0% 100% 3040.15kB 28.66% gopkg.in/vmihailenco/msgpack%2ev2.decodeStructValue 0 0% 100% 727.78kB 6.86% gopkg.in/vmihailenco/msgpack%2ev2.encodeBytesValue 0 0% 100% 926.75kB 8.74% gopkg.in/vmihailenco/msgpack%2ev2.encodePtrValue 0 0% 100% 926.75kB 8.74% gopkg.in/vmihailenco/msgpack%2ev2.encodeSliceValue 0 0% 100% 982.73kB 9.26% gopkg.in/vmihailenco/msgpack%2ev2.encodeStringValue 0 0% 100% 926.75kB 8.74% gopkg.in/vmihailenco/msgpack%2ev2.encodeStructValue 0 0% 100% 528.17kB 4.98% io.Copy 0 0% 100% 3040.15kB 28.66% kipc.(*Bus).Listen 0 0% 100% 3048.04kB 28.74% kipc.(*Bus).Worker 0 0% 100% 3040.15kB 28.66% kipc.(*Bus).handleIncoming 0 0% 100% 3040.15kB 28.66% kipc.(*MsgPackBackend).Unmarshal 0 0% 100% 3040.15kB 28.66% kipc.createObject 0 0% 100% 3122.23kB 29.44% main.(*Client).Response 0 0% 100% 926.75kB 8.74% main.(*sender).Send 0 0% 100% 3122.23kB 29.44% render.(*DisplaysEncoder).GenerateImagePacket 0 0% 100% 1839.54kB 17.34% render.(*DisplaysEncoder).generateImagePacketUnlocked 0 0% 100% 1839.54kB 17.34% render.(*DisplaysEncoder).syncDisplaysRectsUnlocked 0 0% 100% 1839.54kB 17.34% render.(*displayRects).sync 0 0% 100% 1839.54kB 17.34% render.(*displayRects).syncUnlocked 0 0% 100% 10606.92kB 100% runtime.goexit miha@G-00017:~/engine_prof$