[JIRA] [remoting] (JENKINS-5413) SCM polling getting hung

Mon, 12 Jan 2015 23:29:20 -0800

 

At Atmel we're now managing this issue by having the following system groovy script run every couple of minutes to monitor the processor load:

import java.lang.management.*;

def threadBean = ManagementFactory.getThreadMXBean();
def osBean     = ManagementFactory.getOperatingSystemMXBean();

println "\n\n\n[Checking state of (master)]";

println "Current CPU Time used by Jenkins: " + threadBean.getCurrentThreadCpuTime() + "ns";

double processLoad = (osBean.getProcessCpuLoad() * 100).round(2);
double cpuLoad = (osBean.getSystemCpuLoad() * 100).round(2);
println "Process CPU Load: " + processLoad + "%";
println "CPU Load: " + cpuLoad + "%";

if (processLoad < 90) {
  println "\n\n\n === Load is less than 90%, nothing to do ===\n\n\n";
  println "\n\n\n[Done checking: CPU Load: " + cpuLoad + "%]\n\n\n";
  return;
} else {
  println "\n\n\n === Load is more than 90%, checking for stuck threads! ===\n\n\n";
}


println "\n\n\n[Checking all threads]\n\n\n";
def threadNum = 0;
def killThreadNum = 0;

def stacktraces = Thread.getAllStackTraces();
stacktraces.each { thread, stack ->
  if (thread.getName().contains("trigger/TimerTrigger/check") ) {
    println "=== Interrupting thread " + thread.getName()+ " ===";
    thread.interrupt();
    killThreadNum++;
  }
  threadNum++;
}

println "\n\n\n[Done checking: " + threadNum + " threads, killed " + killThreadNum + "]\n\n\n";

return; // Suppress groovy state dump

Note that we had to check for TimerTrigger, not SCM Polling as the original code did. This is currently running on 1.580.2.
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators.
For more information on JIRA, see: http://www.atlassian.com/software/jira

--
You received this message because you are subscribed to the Google Groups "Jenkins Issues" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Reply via email to