[JIRA] [ssh-slaves] (JENKINS-22320) High CPU consumption because of SSH communication

Mon, 24 Mar 2014 01:42:31 -0700

Issue Type: Bug Bug
Assignee: Kohsuke Kawaguchi
Attachments: Selection_145.png, Selection_146.png
Components: ssh-slaves
Created: 24/Mar/14 8:41 AM
Description:

Our Jenkins instance runs 1.553 (since two weeks, was an three year old version before) on Linux with 10 slaves: 6 Linux connected through SSH and 4 windows with WebStart agent started as services.

When no job is running, CPU is almost 0. Correct.

When only two or three jobs are running, Jenkins CPU raises between 70% to 160%. atop declares 96% of CPU time spent in IRQ (even with "no" disk access), most CPU consumption is considered as system time. On average since boot, the master node consumes 100% of one CPU. Even if it has 4 CPUs, job execution time is between 2x and 3x compared to older version.

I configured JMX and did a quick CPU profiling. Top consumer threads are unnamed and are all related to SSH communication

"Thread-13" - Thread t@90
   java.lang.Thread.State: RUNNABLE
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at com.trilead.ssh2.crypto.cipher.CipherInputStream.fill_buffer(CipherInputStream.java:41)
	at com.trilead.ssh2.crypto.cipher.CipherInputStream.internal_read(CipherInputStream.java:52)
	at com.trilead.ssh2.crypto.cipher.CipherInputStream.getBlock(CipherInputStream.java:79)
	at com.trilead.ssh2.crypto.cipher.CipherInputStream.read(CipherInputStream.java:108)
	at com.trilead.ssh2.transport.TransportConnection.receiveMessage(TransportConnection.java:232)
	at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:682)
	at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:480)
	at java.lang.Thread.run(Thread.java:744)



So there is chance the "SSH agent plugin" is concerned.



I am ready to do deeper analysis on my system if required, and of course to test patches.
Project: Jenkins
Labels: performance
Priority: Critical Critical
Reporter: Yves Martin
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators.
For more information on JIRA, see: http://www.atlassian.com/software/jira

--
You received this message because you are subscribed to the Google Groups "Jenkins Issues" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Reply via email to