You have 4601 goroutines blocked on reading from the websocket goroutine profile: total 4583 1526 @ 0x42d77a 0x42806b 0x4277c9 0x67ffb8 0x680024 0x681861 0x6924e0 0x5b2e5c 0x5b34bc 0x7bc06d 0x7bc1b6 0x7bd237 0x7a2d94 0x45da11 # 0x4277c8 net.runtime_pollWait+0x58 /usr/lib/go-1.7/src/runtime/netpoll.go:160 # 0x67ffb7 net.(*pollDesc).wait+0x37 /usr/lib/go-1.7/src/net/fd_poll_runtime.go:73 # 0x680023 net.(*pollDesc).waitRead+0x33 /usr/lib/go-1.7/src/net/fd_poll_runtime.go:78 # 0x681860 net.(*netFD).Read+0x1a0 /usr/lib/go-1.7/src/net/fd_unix.go:243 # 0x6924df net.(*conn).Read+0x6f /usr/lib/go-1.7/src/net/net.go:173 # 0x5b2e5b bufio.(*Reader).fill+0x10b /usr/lib/go-1.7/src/bufio/bufio.go:97 # 0x5b34bb bufio.(*Reader).Read+0x1bb /usr/lib/go-1.7/src/bufio/bufio.go:209 # 0x7bc06c bitbucket.org/sjtushi/comet/vendor/github.com/gorilla/websocket.(*Conn).readFull+0x8c
If the websocket never disconnects, then the goroutine will pin all memory it is using. You probably need to add some timeouts to your application. On Tuesday, 6 September 2016 19:14:29 UTC+10, aide...@gmail.com wrote: > > I can make sure that the goroutine is exited, because the goroutine count > is stable but the heap count is always increase. You can see more detail > pprof from https://comet.shiyanlou.com/debug/pprof/. > > > 在 2016年9月6日星期二 UTC+8下午5:05:05,Dave Cheney写道: >> >> Are you sure that goroutines are exiting? You can check this in the >> /debug/pprof handler. It will tell you how many goroutines are currently >> running. >> >> On Tuesday, 6 September 2016 18:49:29 UTC+10, aide...@gmail.com wrote: >>> >>> I think >>> https://gist.github.com/aiden0z/b8cf00953e81f778bd584fa2ff7eaae7#file-server-go-L268 >>> error >>> is not the core problem. The error is ignored because the go-socket.io >>> no error returned, see >>> https://github.com/googollee/go-socket.io/blob/master/server.go#L91. >>> >>> go 1.6 and 1.7 I have tried, the problem still stands. >>> >>> >>> 在 2016年9月6日星期二 UTC+8下午4:25:40,Dave Cheney写道: >>>> >>>> This error handling looks wrong, >>>> >>>> >>>> https://gist.github.com/aiden0z/b8cf00953e81f778bd584fa2ff7eaae7#file-server-go-L268 >>>> >>>> Any error from socketio is ignored, and the method always succeeds >>>> unconditionally. >>>> >>>> Also, which version of Go are you using ? >>>> >>>> On Tuesday, 6 September 2016 18:07:42 UTC+10, aide...@gmail.com wrote: >>>>> >>>>> I have restart the the websocket service, and collect new callgraph. >>>>> >>>>> 1. in-use heap callgraph >>>>> >>>>> >>>>> <https://lh3.googleusercontent.com/-mOinFHs1gmM/V854u84GSMI/AAAAAAAAAFI/Dz7zJN6hobsrhRDBTfE__iR-AXp-rpxkACLcB/s1600/comet_inuse_heap.png> >>>>> >>>>> 2. heap alloc callgraph >>>>> >>>>> >>>>> <https://lh3.googleusercontent.com/-zriittHlglE/V8542kygEqI/AAAAAAAAAFM/BojzIQ73zy0XbkrNeU4LX7dUAjC6T0ipQCLcB/s1600/comet_alloc_heap.png> >>>>> >>>>> >>>>> >>>>> 3. source code, pls see gist >>>>> https://gist.github.com/aiden0z/b8cf00953e81f778bd584fa2ff7eaae7 >>>>> >>>>> >>>>> >>>>> 在 2016年9月6日星期二 UTC+8下午3:42:06,Dave Cheney写道: >>>>>> >>>>>> Are you sure that goroutines are exiting? You can check this in the >>>>>> /debug/pprof handler. It will tell you how many goroutines are currently >>>>>> running. >>>>> >>>>> -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.