If you have one connection established to that server which is functioning (perhaps with -v on the client ssh) can you get the problem to occur with a second connection to that server?
That's a good question, I'm not actually sure if I've ever opened two connections to it at once. For better or worse today is a "good" day so I'll have to wait to test this.
If so, can you take a look at whether you are getting any fresh processes from your second connection attempts when they stall? (The question is: how far does a stalled attempt reach before it runs into this problem?)
Not sure what you mean here about "fresh processes", do you want me to look at the output of ps or something else?