|
||||||||
|
This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira |
||||||||
You received this message because you are subscribed to the Google Groups "Jenkins Issues" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Hi Daniel, hi Oleg.
We are currently using Jenkins 1.509.4 on our servers. The servers have up to 50 build hosts connected to them, with roughly 1000 executors spawned in total.
During peaks in test execution load, a significant queue can accumulate. During these times, we found that even a handful of builds in the queue could lock up the servers so badly, that it did not not start new builds in a timely manner, causing the queue to become increasingly longer, thus worsening the problem.
In short: (High number of executors = Expensive Queue Maintenance) + Queued builds = Constant synchronization on Queue.
This will also cause the entire GUI to become unresponsive, as it waits for the Queue to be free, before rendering the "nodes" side-panel.
We are not currently experiencing this issue, because we are running a modified version of the Jenkins servers, where the majority of the "synchronized" calls have been replaced with read/write locks, that allow simultaneous access of several methods into the queue.
You can find the patches against 1.509.4 and 1.554.1 here:
https://github.com/HedAurabesh/jenkins/tree/queue-1.509.4
https://github.com/HedAurabesh/jenkins/tree/queue-1.554.1
We intend to release a formal pull request to Jenkins soon.
The only side-effect of this change is, that the "scheduled builds" side-panel on the Job overview sometimes renders a build to be BOTH scheduled AND in progress. But that is a little price to pay for the server not getting stuck. Additionally, it is nothing a quick "F5" can't fix.![]()
Best regards,
Martin