Re: [go-nuts] Deadlocking on channels

2017-01-21 Thread lee
Hi Alex, Thanks for that. I'll check it out and integrate it in. Will give it a longer soak test too. Yes packets do get received out of order. They are to a nanosecond accuracy from the source and so with transmission times etc they do come ever so slightly out of order. As I am also compar

Re: [go-nuts] Deadlocking on channels

2017-01-21 Thread Alex Bucataru
In https://play.golang.org/p/fqPXXpNufO you need to protect all channelMap access with a mutex, in https://play.golang.org/p/88zT7hBLeD you don't. If you make it a package-level var, `make` it in `init` or check if it is nil before you `make` in jobDispatcher, as you should assume you have no c

Re: [go-nuts] Deadlocking on channels

2017-01-21 Thread lee
I just uploaded a working example to https://play.golang.org/p/88zT7hBLeD It is a long running process so will need running locally on a machine as the playground kills it. Hopefully this will help get to the bottom of it! Lee On Saturday, January 21, 2017 at 8:24:04 AM UTC, l...@pinkfroot.com

Re: [go-nuts] Deadlocking on channels

2017-01-21 Thread lee
Thanks Alex, I did what I would call a halfway house so to speak... https://play.golang.org/p/fqPXXpNufO It ran for much longer before locking on the same line! Trouble is, now I can't actually see the lock whereas I could before! I'm wondering if I should keep a global channelMap and lock t

Re: [go-nuts] Deadlocking on channels

2017-01-20 Thread Alex Bucataru
Actually, a buffered channel alone is not enough... A more robust solution is to move the shutdown branch in jobDispatcher into a function; then call this function in a goroutine of its own right after you create shutdown channel. That will prevent the message sending and channel closing from d

Re: [go-nuts] Deadlocking on channels

2017-01-20 Thread Alex Bucataru
Your deadlock happens because the message channel is not buffered. The jobDispatcher goroutine is blocked waiting for channel to accept msg, while the processPackets packets just sent the shutdown signal and exited (on the time.Ticker branch of the select). A buffer of any size, even 1, would a

Re: [go-nuts] Deadlocking on channels

2017-01-20 Thread lee
Yeah I did try with it enabled and it sees nothing. I use it all the time, it's great!! Thing is I can see the race condition in the code, just trying to work out how to avoid it! On Friday, January 20, 2017 at 10:09:05 PM UTC, Val wrote: > > Did you try with the race detector enabled? What wa

Re: [go-nuts] Deadlocking on channels

2017-01-20 Thread Val
Did you try with the race detector enabled? What was the output? If not, you really should, and it's super easy to do. Any complain from the race detector is a bug that must be fixed, not a mere warning. Cheers Val On Friday, January 20, 2017 at 8:10:42 PM UTC+1, l...@pinkfroot.com wrote: > I

Re: [go-nuts] Deadlocking on channels

2017-01-20 Thread lee
Perfect thanks! That is really useful! Well it confirms that it is blocked on the `channel <- msg` line. It is kind of a race condition but one that I don't think will be detected by the detector! On Friday, January 20, 2017 at 6:28:33 PM UTC, Shawn Milochik wrote: > > Use pprof: https://golan

Re: [go-nuts] Deadlocking on channels

2017-01-20 Thread Shawn Milochik
Use pprof: https://golang.org/pkg/net/http/pprof/ Then, when it hangs, you can pull a traceback directly from your app and see which goroutine is blocked, and exactly which line of code is the cause. You can easily get stats with wget or curl: http://localhost:6060/debug/pprof/goroutine?debug=2

[go-nuts] Deadlocking on channels

2017-01-20 Thread lee
Hi all, I just posted this to SO and would appreciate some advice!! http://stackoverflow.com/questions/41765198/deadlock-with-buffered-channel My first attempt (coming from other languages) was that all the messages from addMessages went into some huge global map with locks. This ended in tea