Hello,
I get the following stack trace once a day:
Exception:
FATAL: unable to create new native thread
java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:640)
at hudson.Proc$LocalProc.<init>(Proc.java:264)
at hudson.Proc$LocalProc.<init>(Proc.java:216)
at hudson.Launcher$LocalLauncher.launch(Launcher.java:763)
at hudson.Launcher$ProcStarter.start(Launcher.java:353)
at
hudson.plugins.perforce.HudsonP4DefaultExecutor.exec(HudsonP4DefaultExecutor.java:79)
at
com.tek42.perforce.parse.AbstractPerforceTemplate.getPerforceResponse(AbstractPerforceTemplate.java:321)
at
com.tek42.perforce.parse.AbstractPerforceTemplate.getPerforceResponse(AbstractPerforceTemplate.java:292)
at com.tek42.perforce.parse.Workspaces.getWorkspace(Workspaces.java:61)
at
hudson.plugins.perforce.PerforceSCM.getPerforceWorkspace(PerforceSCM.java:1467)
at
hudson.plugins.perforce.PerforceSCM.getPerforceWorkspace(PerforceSCM.java:1428)
at hudson.plugins.perforce.PerforceSCM.checkout(PerforceSCM.java:792)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1325)
at
hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:676)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:88)
at
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:581)
at hudson.model.Run.execute(Run.java:1516)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
at hudson.model.ResourceController.execute(ResourceController.java:88)
at hudson.model.Executor.run(Executor.java:236)
Environment:
* Windows 7 Professional with SP1
* Jenkins 1.498
* Java SDK 1.6.0_26-b03 (32-Bit)
* Active Directory plugin 1.30
The URL http:/jenkinsserver/threadDump shows that there are hundreds of
ComThreads:
ComThread for RequestHandlerThread[#301]
"ComThread for RequestHandlerThread[#301]" Id=50942 Group=main RUNNABLE
(in native)
at com4j.Win32Lock.suspend0(Native Method)
at com4j.Win32Lock.suspend(Win32Lock.java:37)
at com4j.ComThread.run0(ComThread.java:146)
at com4j.ComThread.run(ComThread.java:134)
The Monitoring plugin
(https://wiki.jenkins-ci.org/display/JENKINS/Monitoring) shows that
about 40 threads are added every hour. I tried to kill the ComThreads
with the plugin but this did not work.
As far as I understand the threads belong to the Active Directory plugin
but I do not understand why they are created / never finish.
I have activated logging for the Active Directory plugin by enabling the
following loggers (all with log level all):
* hudson.plugins.active_directory
* org.acegisecurity
* com4j
The log contains a lot of entries like this:
Jan 16, 2013 12:20:27 PM
org.acegisecurity.context.HttpSessionContextIntegrationFilter doFilter
FINE: SecurityContextHolder now cleared, as request processing completed
Jan 16, 2013 12:20:27 PM
org.acegisecurity.providers.anonymous.AnonymousProcessingFilter doFilter
FINE: Populated SecurityContextHolder with anonymous token:
'org.acegisecurity.providers.anonymous.AnonymousAuthenticationToken@7ceb964f:
Username: anonymous; Password: [PROTECTED]; Authenticated: true;
Details: org.acegisecurity.ui.WebAuthenticationDetails@1de6:
RemoteIpAddress: ##.##.##.##; SessionId: null; Granted Authorities: '
Jan 16, 2013 12:20:27 PM
org.acegisecurity.context.HttpSessionContextIntegrationFilter
storeSecurityContextInSession
FINE: The HttpSession is currently null, and the
HttpSessionContextIntegrationFilter is prohibited from creating an
HttpSession (because the allowSessionCreation property is false) -
SecurityContext thus not stored for next request
Jan 16, 2013 12:20:27 PM
org.acegisecurity.ui.basicauth.BasicProcessingFilter doFilter
FINE: Authorization header: null
Jan 16, 2013 12:20:27 PM org.acegisecurity.ui.ExceptionTranslationFilter
doFilter
FINE: Chain processed normally
Jan 16, 2013 12:20:27 PM
org.acegisecurity.context.HttpSessionContextIntegrationFilter doFilter
FINE: New SecurityContext instance will be associated with
SecurityContextHolder
Jan 16, 2013 12:20:27 PM
org.acegisecurity.context.HttpSessionContextIntegrationFilter
readSecurityContextFromSession
FINE: No HttpSession currently exists
Jan 16, 2013 12:20:27 PM
org.acegisecurity.context.HttpSessionContextIntegrationFilter doFilter
FINE: SecurityContextHolder now cleared, as request processing completed
Jan 16, 2013 12:20:27 PM
org.acegisecurity.context.HttpSessionContextIntegrationFilter
storeSecurityContextInSession
FINE: The HttpSession is currently null, and the
HttpSessionContextIntegrationFilter is prohibited from creating an
HttpSession (because the allowSessionCreation property is false) -
SecurityContext thus not stored for next request
Jan 16, 2013 12:20:27 PM org.acegisecurity.ui.ExceptionTranslationFilter
doFilter
FINE: Chain processed normally
A lot of traffic (about four requests per second) is caused by the
desktop application Hudson Tray Tracker
(https://code.google.com/p/hudson-tray-tracker/). This application does
not log in (it uses anonymous access).
I have also
* enabled access logging, but found no unusual URL requests
* connected with VisualVM (http://visualvm.java.net/) to the Jenkins
JVM, but got no more informations
* created a script that logs the number of threads and the virtual size
of the Java process, but it only shows that both numbers are constantly
increasing
At the moment I do not know where I could still searching for the cause
of the creation of so many threads. Any other ideas?
Regards
Bernhard Berbuir
