Teppo Kurki created JENKINS-13154: ------------------------------------- Summary: Heavy thread congestion with FingerPrint.save Key: JENKINS-13154 URL: https://issues.jenkins-ci.org/browse/JENKINS-13154 Project: Jenkins Issue Type: Bug Components: core Affects Versions: current Environment: System Properties Name ↓ Value awt.toolkit sun.awt.X11.XToolkit executable-war /foobar/jenkins/jenkins.war file.encoding UTF-8 file.encoding.pkg sun.io file.separator / guice.disable.misplaced.annotation.check true hudson.diyChunking true hudson.upstreamCulprits true java.awt.graphicsenv sun.awt.X11GraphicsEnvironment java.awt.headless true java.awt.printerjob sun.print.PSPrinterJob java.class.path /foobar/jenkins/jenkins.war java.class.version 51.0 java.endorsed.dirs /foobar/dist/jdk/jdk1.7.0_02/jre/lib/endorsed java.ext.dirs /foobar/dist/jdk/jdk1.7.0_02/jre/lib/ext:/usr/java/packages/lib/ext java.home /foobar/dist/jdk/jdk1.7.0_02/jre java.io.tmpdir /foobar/scratch java.library.path /usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib java.runtime.name Java(TM) SE Runtime Environment java.runtime.version 1.7.0_02-b13 java.specification.name Java Platform API Specification java.specification.vendor Oracle Corporation java.specification.version 1.7 java.vendor Oracle Corporationa java.vendor.url http://java.oracle.com/ java.vendor.url.bug http://bugreport.sun.com/bugreport/ java.version 1.7.0_02 java.vm.info mixed mode java.vm.name Java HotSpot(TM) 64-Bit Server VM java.vm.specification.name Java Virtual Machine Specification java.vm.specification.vendor Oracle Corporation java.vm.specification.version 1.7 java.vm.vendor Oracle Corporation java.vm.version 22.0-b10 jna.platform.library.path /usr/lib64:/lib64:/usr/lib:/lib line.separator mail.smtp.sendpartial true mail.smtps.sendpartial true os.arch amd64 os.name Linux os.version 2.6.18-164.15.1.el5 path.separator : securerandom.source file:/dev/./urandom sun.arch.data.model 64 sun.boot.class.path /foobar/dist/jdk/jdk1.7.0_02/jre/lib/resources.jar:/foobar/dist/jdk/jdk1.7.0_02/jre/lib/rt.jar:/foobar/dist/jdk/jdk1.7.0_02/jre/lib/sunrsasign.jar:/foobar/dist/jdk/jdk1.7.0_02/jre/lib/jsse.jar:/foobar/dist/jdk/jdk1.7.0_02/jre/lib/jce.jar:/foobar/dist/jdk/jdk1.7.0_02/jre/lib/charsets.jar:/foobar/dist/jdk/jdk1.7.0_02/jre/classes sun.boot.library.path /foobar/dist/jdk/jdk1.7.0_02/jre/lib/amd64 sun.cpu.endian little sun.cpu.isalist sun.io.unicode.encoding UnicodeLittle sun.java.command /foobar/jenkins/jenkins.war --httpPort=8088 sun.java.launcher SUN_STANDARD sun.jnu.encoding UTF-8 sun.management.compiler HotSpot 64-Bit Tiered Compilers sun.os.patch.level unknown svnkit.http.methods Digest,Basic,NTLM,Negotiate svnkit.ssh2.persistent false user.country FI user.dir /foobar/jenkins user.home /foobar/baz user.language fi user.name baz user.timezone Europe/Helsinki Environment Variables Name ↓ Value G_BROKEN_FILENAMES 1 HISTSIZE 1000 HOME /foobar/baz HUDSON_HOME /foobar/jenkins INPUTRC /etc/inputrc JAVA_HOME /foobar/dist/jdk/current JENKINS_HOME /foobar/jenkins JETTYENV_HOME /foobar/baz/env-jetty LANG en_US.UTF-8 LC_ADDRESS fi_FI.UTF-8 LC_ALL fi_FI.utf8 LC_CTYPE fi_FI.UTF-8 LC_IDENTIFICATION fi_FI.UTF-8 LC_MEASUREMENT fi_FI.UTF-8 LC_MONETARY fi_FI.UTF-8 LC_NAME fi_FI.UTF-8 LC_PAPER fi_FI.UTF-8 LC_TELEPHONE fi_FI.UTF-8 LESSOPEN |/usr/bin/lesspipe.sh %s LOGNAME baz LS_COLORS M2_HOME /foobar/dist/maven/current MAIL /var/spool/mail/baz MAVEN_OPTS -XX:+UseCompressedOops -Djava.security.egd=file:///dev/urandom -Xmx512m MVN /foobar/dist/maven/current/bin/mvn NLSPATH /usr/dt/lib/nls/msg/%L/%N.cat PAGER less PATH /opt/CollabNet_Subversion/bin:/opt/CollabNet_Subversion/bin:/foobar/dist/jdk/current/bin:/foobar/dist/maven/current/bin:/foobar/baz/env-jetty/bin:/foobar/baz/env-jetty/dist/jdk/current/bin:/foobar/baz/env-fab/bin:/foobar/baz/env-apache/bin:/foobar/baz/bin:/foobar/baz/python/bin:/foobar/dist/jdk/current/bin:/foobar/dist/maven/current/bin:/opt/CollabNet_Subversion/bin/:/foobar/scala-2.9.1.final/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/usr/symcli/bin PWD /foobar/jenkins READLINK readlink -f SHELL /bin/bash SHLVL 1 SSH_TTY /dev/pts/2 TERM xterm-256color TMP /foobar/baz/scratch TMPDIR /foobar/baz/scratch USER baz USE_SSL on XFILESEARCHPATH /usr/dt/app-defaults/%L/Dt Plugins Name ↓ Version Enabled Pinned ant 1.1 true false javadoc 1.0 true false timestamper 1.2.2 true false greenballs 1.11 true false maven-plugin 1.455 true false instant-messaging 1.21 true false ircbot 2.18 true false join 1.13 true false rsync 1.0-SNAPSHOT (private-02/04/2011 11:58-mpe) true false PrioritySorter 1.3 false false claim 1.7 true false radiatorviewplugin 1.13 true false build-pipeline-plugin 1.2.3 true false BlameSubversion 1.25 true false downstream-buildview 1.6 true false audit-trail 1.7 true false cvs 1.6 true false port-allocator 1.5 true false disk-usage 0.15 true false m2-extra-steps 1.1.7 true false subversion 1.34 true false ssh-slaves 0.21 true false next-build-number 1.0 true false email-ext 2.18 true false git 1.1.16 true false translation 1.8 Reporter: Teppo Kurki
We are seeing repeatable very heavy lock congestion with FingerPrint.save. Jenkins apparently slides down to a state where a lot of threads have locked / are competing for a lock on FingerPrint instance and are competing for a lock on AnnotationMapper (a singleton). Everything grings to a halt. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jenkins-ci.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira