[
https://issues.jenkins-ci.org/browse/JENKINS-13154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikko Peltonen reopened JENKINS-13154:
--------------------------------------
Jenkins 1.463 did not fix this problem for us. We're still seeing similar
congestion, and many of our builds are stuck for tens of minutes in "Waiting
for Jenkins to finish collecting data" at the end of the build.
I'll attach another thread dump taken with 1.463.
> Heavy thread congestion with FingerPrint.save
> ---------------------------------------------
>
> Key: JENKINS-13154
> URL: https://issues.jenkins-ci.org/browse/JENKINS-13154
> Project: Jenkins
> Issue Type: Bug
> Components: core
> Affects Versions: current
> Environment: System Properties
> Name ↓ Value
> awt.toolkit sun.awt.X11.XToolkit
> executable-war /foobar/jenkins/jenkins.war
> file.encoding UTF-8
> file.encoding.pkg sun.io
> file.separator /
> guice.disable.misplaced.annotation.check true
> hudson.diyChunking true
> hudson.upstreamCulprits true
> java.awt.graphicsenv sun.awt.X11GraphicsEnvironment
> java.awt.headless true
> java.awt.printerjob sun.print.PSPrinterJob
> java.class.path /foobar/jenkins/jenkins.war
> java.class.version 51.0
> java.endorsed.dirs /foobar/dist/jdk/jdk1.7.0_02/jre/lib/endorsed
> java.ext.dirs
> /foobar/dist/jdk/jdk1.7.0_02/jre/lib/ext:/usr/java/packages/lib/ext
> java.home /foobar/dist/jdk/jdk1.7.0_02/jre
> java.io.tmpdir /foobar/scratch
> java.library.path
> /usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> java.runtime.name Java(TM) SE Runtime Environment
> java.runtime.version 1.7.0_02-b13
> java.specification.name Java Platform API Specification
> java.specification.vendor Oracle Corporation
> java.specification.version 1.7
> java.vendor Oracle Corporationa
> java.vendor.url http://java.oracle.com/
> java.vendor.url.bug http://bugreport.sun.com/bugreport/
> java.version 1.7.0_02
> java.vm.info mixed mode
> java.vm.name Java HotSpot(TM) 64-Bit Server VM
> java.vm.specification.name Java Virtual Machine Specification
> java.vm.specification.vendor Oracle Corporation
> java.vm.specification.version 1.7
> java.vm.vendor Oracle Corporation
> java.vm.version 22.0-b10
> jna.platform.library.path /usr/lib64:/lib64:/usr/lib:/lib
> line.separator
> mail.smtp.sendpartial true
> mail.smtps.sendpartial true
> os.arch amd64
> os.name Linux
> os.version 2.6.18-164.15.1.el5
> path.separator :
> securerandom.source file:/dev/./urandom
> sun.arch.data.model 64
> sun.boot.class.path
> /foobar/dist/jdk/jdk1.7.0_02/jre/lib/resources.jar:/foobar/dist/jdk/jdk1.7.0_02/jre/lib/rt.jar:/foobar/dist/jdk/jdk1.7.0_02/jre/lib/sunrsasign.jar:/foobar/dist/jdk/jdk1.7.0_02/jre/lib/jsse.jar:/foobar/dist/jdk/jdk1.7.0_02/jre/lib/jce.jar:/foobar/dist/jdk/jdk1.7.0_02/jre/lib/charsets.jar:/foobar/dist/jdk/jdk1.7.0_02/jre/classes
> sun.boot.library.path /foobar/dist/jdk/jdk1.7.0_02/jre/lib/amd64
> sun.cpu.endian little
> sun.cpu.isalist
> sun.io.unicode.encoding UnicodeLittle
> sun.java.command /foobar/jenkins/jenkins.war --httpPort=8088
> sun.java.launcher SUN_STANDARD
> sun.jnu.encoding UTF-8
> sun.management.compiler HotSpot 64-Bit Tiered Compilers
> sun.os.patch.level unknown
> svnkit.http.methods Digest,Basic,NTLM,Negotiate
> svnkit.ssh2.persistent false
> user.country FI
> user.dir /foobar/jenkins
> user.home /foobar/baz
> user.language fi
> user.name baz
> user.timezone Europe/Helsinki
> Environment Variables
> Name ↓ Value
> G_BROKEN_FILENAMES 1
> HISTSIZE 1000
> HOME /foobar/baz
> HUDSON_HOME /foobar/jenkins
> INPUTRC /etc/inputrc
> JAVA_HOME /foobar/dist/jdk/current
> JENKINS_HOME /foobar/jenkins
> JETTYENV_HOME /foobar/baz/env-jetty
> LANG en_US.UTF-8
> LC_ADDRESS fi_FI.UTF-8
> LC_ALL fi_FI.utf8
> LC_CTYPE fi_FI.UTF-8
> LC_IDENTIFICATION fi_FI.UTF-8
> LC_MEASUREMENT fi_FI.UTF-8
> LC_MONETARY fi_FI.UTF-8
> LC_NAME fi_FI.UTF-8
> LC_PAPER fi_FI.UTF-8
> LC_TELEPHONE fi_FI.UTF-8
> LESSOPEN |/usr/bin/lesspipe.sh %s
> LOGNAME baz
> LS_COLORS
> M2_HOME /foobar/dist/maven/current
> MAIL /var/spool/mail/baz
> MAVEN_OPTS -XX:+UseCompressedOops -Djava.security.egd=file:///dev/urandom
> -Xmx512m
> MVN /foobar/dist/maven/current/bin/mvn
> NLSPATH /usr/dt/lib/nls/msg/%L/%N.cat
> PAGER less
> PATH
> /opt/CollabNet_Subversion/bin:/opt/CollabNet_Subversion/bin:/foobar/dist/jdk/current/bin:/foobar/dist/maven/current/bin:/foobar/baz/env-jetty/bin:/foobar/baz/env-jetty/dist/jdk/current/bin:/foobar/baz/env-fab/bin:/foobar/baz/env-apache/bin:/foobar/baz/bin:/foobar/baz/python/bin:/foobar/dist/jdk/current/bin:/foobar/dist/maven/current/bin:/opt/CollabNet_Subversion/bin/:/foobar/scala-2.9.1.final/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/usr/symcli/bin
> PWD /foobar/jenkins
> READLINK readlink -f
> SHELL /bin/bash
> SHLVL 1
> SSH_TTY /dev/pts/2
> TERM xterm-256color
> TMP /foobar/baz/scratch
> TMPDIR /foobar/baz/scratch
> USER baz
> USE_SSL on
> XFILESEARCHPATH /usr/dt/app-defaults/%L/Dt
> Plugins
> Name ↓ Version Enabled Pinned
> ant 1.1 true false
> javadoc 1.0 true false
> timestamper 1.2.2 true false
> greenballs 1.11 true false
> maven-plugin 1.455 true false
> instant-messaging 1.21 true false
> ircbot 2.18 true false
> join 1.13 true false
> rsync 1.0-SNAPSHOT (private-02/04/2011 11:58-mpe) true false
> PrioritySorter 1.3 false false
> claim 1.7 true false
> radiatorviewplugin 1.13 true false
> build-pipeline-plugin 1.2.3 true false
> BlameSubversion 1.25 true false
> downstream-buildview 1.6 true false
> audit-trail 1.7 true false
> cvs 1.6 true false
> port-allocator 1.5 true false
> disk-usage 0.15 true false
> m2-extra-steps 1.1.7 true false
> subversion 1.34 true false
> ssh-slaves 0.21 true false
> next-build-number 1.0 true false
> email-ext 2.18 true false
> git 1.1.16 true false
> translation 1.8
> Reporter: Teppo Kurki
> Attachments: Thread Dump Analyzer screenshot.jpg,
> threaddump_samples.txt, threads_sample_jenkins_1.463.txt
>
>
> We are seeing repeatable very heavy lock congestion with FingerPrint.save.
> Jenkins apparently slides down to a state where a lot of threads have locked
> / are competing for a lock on FingerPrint instance and are competing for a
> lock on AnnotationMapper (a singleton). Everything grings to a halt.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jenkins-ci.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
