Re: 99% of requests fast, 1% hang forever (80%/20% in the afternoon)

Additional basic info: The jenkins master is running on a 32 core box, we have 38 slaves (of which 10 are currently down), everything in the build queue (6) is waiting on a specially tagged slave. I can run api or UI requests. System load average on the master is 0.06, jenkins is using about

99% of requests fast, 1% hang forever (80%/20% in the afternoon)

Jenkins 1.634 Java(TM) SE Runtime Environment (build 1.7.0_55-b13) Lots more info at https://gist.github.com/wwalker/fc85ea65033ac9b0f714 In short, if I make 1000 simple requests (https://jenkins/hudson/job/$i/config.xml) where $i is iterating over a list of job names, these return in a very co