Thanks Mathus. We were able to resolve the issue by restarting the master. Here’s what happened. A network outage caused the master to loose connectivity to the slaves. After some time, the master issues a termination request to the slaves. When the network issue was resolved, the slaves reconnected. However, the termination request was never cancelled by the master.
From: Baptiste Mathus <m...@batmat.net<mailto:m...@batmat.net>> Reply-To: "jenkinsci-users@googlegroups.com<mailto:jenkinsci-users@googlegroups.com>" <jenkinsci-users@googlegroups.com<mailto:jenkinsci-users@googlegroups.com>> Date: Friday, January 29, 2016 at 1:45 AM To: "jenkinsci-users@googlegroups.com<mailto:jenkinsci-users@googlegroups.com>" <jenkinsci-users@googlegroups.com<mailto:jenkinsci-users@googlegroups.com>> Subject: Re: slave termination error As you can see in the stack trace, this error seems to come from the ResponseTimeMonitor. Are per chance your master node and the build agents not on the same network? Could you try pinging nodes from the master and paste the result here so that a can judge if it appears normal or not? My 2 cents Le 28 janv. 2016 4:00 PM, "Ashish Yadav" <ashish.ya...@firemon.com<mailto:ashish.ya...@firemon.com>> a écrit : Any guidance is greatly appreciated… From: Ashish Yadav <ashish.ya...@firemon.com<mailto:ashish.ya...@firemon.com>> Date: Thursday, January 28, 2016 at 8:09 AM To: "jenkinsci-users@googlegroups.com<mailto:jenkinsci-users@googlegroups.com>" <jenkinsci-users@googlegroups.com<mailto:jenkinsci-users@googlegroups.com>> Subject: Re: slave termination error I restarted the slave node and am now getting this error even though the job finishes successfully. Jan 28, 2016 8:04:30 AM WARNING hudson.model.Executor finish2 Executor #0 for NodeABC : executing dummy-job #49 termination trace hudson.model.Computer$TerminationRequest: Termination requested at Wed Jan 27 21:46:03 CST 2016 by Thread[Monitoring thread for Response Time started on Wed Jan 27 21:46:03 CST 2016,5,main] [id=278206] at hudson.model.Computer.recordTermination(Computer.java:216) at hudson.model.Computer.disconnect(Computer.java:466) at hudson.slaves.SlaveComputer.disconnect(SlaveComputer.java:601) at hudson.node_monitors.ResponseTimeMonitor$1.monitor(ResponseTimeMonitor.java:72) at hudson.node_monitors.AbstractNodeMonitorDescriptor$Record.run(AbstractNodeMonitorDescriptor.java:302) Jan 28, 2016 8:04:30 AM WARNING hudson.model.Executor finish2 Executor #0 for NodeABC : executing dummy-job #49 termination trace hudson.model.Computer$TerminationRequest: Termination requested at Thu Jan 28 07:10:51 CST 2016 by Thread[Handling POST /computer/NodeABC/doDisconnect from xxx.xxx.xxx.xxx : RequestHandlerThread[#47],5,main] [id=273178] from HTTP request for http://xxx/computer/NodeABC/doDisconnect at hudson.model.Computer.recordTermination(Computer.java:207) at hudson.model.Computer.disconnect(Computer.java:466) at hudson.slaves.SlaveComputer.disconnect(SlaveComputer.java:601) at hudson.slaves.SlaveComputer.doDoDisconnect(SlaveComputer.java:594) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:298) at org.kohsuke.stapler.interceptor.RequirePOST$Processor.invoke(RequirePOST.java:46) at org.kohsuke.stapler.Function$InterceptedFunction.invoke(Function.java:399) at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:161) at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:96) at org.kohsuke.stapler.MetaClass$1.doDispatch(MetaClass.java:121) at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876) at org.kohsuke.stapler.MetaClass$13.dispatch(MetaClass.java:411) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876) at org.kohsuke.stapler.MetaClass$4.doDispatch(MetaClass.java:211) at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:649) at org.kohsuke.stapler.Stapler.service(Stapler.java:238) at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:686) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1494) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:123) at hudson.plugins.audit_trail.AuditTrailFilter.doFilter(AuditTrailFilter.java:95) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:120) at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:114) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482) at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:48) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84) at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at jenkins.security.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:117) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:93) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249) at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:67) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76) at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:168) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482) at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482) at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482) at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1474) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:499) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:533) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:428) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) at org.eclipse.jetty.server.Server.handle(Server.java:370) at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489) at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:960) at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1021) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240) at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82) at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:668) at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52) at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Jan 28, 2016 8:04:30 AM WARNING hudson.model.Executor finish2 Executor #0 for NodeABC : executing dummy-job-to-test-success-email #49 termination trace hudson.model.Computer$TerminationRequest: Termination requested at Thu Jan 28 08:01:39 CST 2016 by Thread[Handling POST /computer/NodeABC/doDisconnect from xxx.xxx.xxx.xxx : RequestHandlerThread[#42],5,main] [id=272809] from HTTP request for http://xxx/computer/NodeABC/doDisconnect at hudson.model.Computer.recordTermination(Computer.java:207) at hudson.model.Computer.disconnect(Computer.java:466) at hudson.slaves.SlaveComputer.disconnect(SlaveComputer.java:601) at hudson.slaves.SlaveComputer.doDoDisconnect(SlaveComputer.java:594) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:298) at org.kohsuke.stapler.interceptor.RequirePOST$Processor.invoke(RequirePOST.java:46) at org.kohsuke.stapler.Function$InterceptedFunction.invoke(Function.java:399) at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:161) at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:96) at org.kohsuke.stapler.MetaClass$1.doDispatch(MetaClass.java:121) at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876) at org.kohsuke.stapler.MetaClass$13.dispatch(MetaClass.java:411) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876) at org.kohsuke.stapler.MetaClass$4.doDispatch(MetaClass.java:211) at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:649) at org.kohsuke.stapler.Stapler.service(Stapler.java:238) at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:686) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1494) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:123) at hudson.plugins.audit_trail.AuditTrailFilter.doFilter(AuditTrailFilter.java:95) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:120) at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:114) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482) at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:48) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84) at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at jenkins.security.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:117) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:93) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249) at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:67) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76) at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:168) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482) at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482) at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482) at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1474) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:499) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:533) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:428) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) at org.eclipse.jetty.server.Server.handle(Server.java:370) at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489) at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:960) at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1021) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240) at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82) at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:668) at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52) at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) From: Ashish Yadav <ashish.ya...@firemon.com<mailto:ashish.ya...@firemon.com>> Reply-To: "jenkinsci-users@googlegroups.com<mailto:jenkinsci-users@googlegroups.com>" <jenkinsci-users@googlegroups.com<mailto:jenkinsci-users@googlegroups.com>> Date: Thursday, January 28, 2016 at 7:03 AM To: "jenkinsci-users@googlegroups.com<mailto:jenkinsci-users@googlegroups.com>" <jenkinsci-users@googlegroups.com<mailto:jenkinsci-users@googlegroups.com>> Subject: slave termination error I am running Jenkins v1.609.3 The jobs on my slave nodes are executing correctly. However, now I am getting the following error for every one of my nodes. Executor #0 for NodeABC : executing job1 #104 termination trace hudson.model.Computer$TerminationRequest: Termination requested at Wed Jan 27 21:46:03 CST 2016 by Thread[Monitoring thread for Response Time started on Wed Jan 27 21:46:03 CST 2016,5,main] [id=278206] at hudson.model.Computer.recordTermination(Computer.java:216) at hudson.model.Computer.disconnect(Computer.java:466) at hudson.slaves.SlaveComputer.disconnect(SlaveComputer.java:601) at hudson.node_monitors.ResponseTimeMonitor$1.monitor(ResponseTimeMonitor.java:72) at hudson.node_monitors.AbstractNodeMonitorDescriptor$Record.run(AbstractNodeMonitorDescriptor.java:302) All my slaves are online and connected to master. Do I need to disconnect and re-connect them? -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com<mailto:jenkinsci-users+unsubscr...@googlegroups.com>. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/D2CF6DE0.151AD%25ashish.yadav%40firemon.com<https://groups.google.com/d/msgid/jenkinsci-users/D2CF6DE0.151AD%25ashish.yadav%40firemon.com?utm_medium=email&utm_source=footer>. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com<mailto:jenkinsci-users+unsubscr...@googlegroups.com>. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/D2CF88FE.151D5%25ashish.yadav%40firemon.com<https://groups.google.com/d/msgid/jenkinsci-users/D2CF88FE.151D5%25ashish.yadav%40firemon.com?utm_medium=email&utm_source=footer>. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com<mailto:jenkinsci-users+unsubscr...@googlegroups.com>. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CANWgJS7haH8WbcwXNDiSynsn6eWsme%2BY_LV8UqTcCHvMMFDKxA%40mail.gmail.com<https://groups.google.com/d/msgid/jenkinsci-users/CANWgJS7haH8WbcwXNDiSynsn6eWsme%2BY_LV8UqTcCHvMMFDKxA%40mail.gmail.com?utm_medium=email&utm_source=footer>. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/D2D0C143.1526D%25ashish.yadav%40firemon.com. For more options, visit https://groups.google.com/d/optout.
