On the [email protected] mailing list last night, on thread “jenkins emergency restart notice”, Chris Lambertus wrote:
> Jenkins will be restarted at midnight tonight UTC to address > problems related to ‘too many open files’ and to attempt to > correct a problem with some windows agents. I will be stopping > new builds so that the queue can empty over the next 2 hours. This morning, two people responded on that thread that builds for the Apache Kafka and Apache Brooklyn projects are still encountering this problem. I’ll keep an eye on this thread. Hopefully they’ll figure it out shortly. It’s possible that our issue is different from that of other projects; if the situation isn’t resolved in the next 24 hours I’ll ask on hipchat. -- Steve www.lucidworks.com > On Jul 18, 2017, at 11:28 AM, Cassandra Targett <[email protected]> wrote: > > This is the 2nd one of these I've seen in 2 days (different jobs) but > it seems to resolve itself on the next run. > > Steve Rowe, do you have any insight into what we can do to avoid it? > I'll be happy to ask INFRA if not. > > On Tue, Jul 18, 2017 at 7:32 AM, Apache Jenkins Server > <[email protected]> wrote: >> Build: https://builds.apache.org/job/Solr-reference-guide-7.x/156/ >> >> Log: >> Started by timer >> [EnvInject] - Loading node environment variables. >> Building remotely on H19 (git-websites) in workspace >> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x >>> git rev-parse --is-inside-work-tree # timeout=10 >> Fetching changes from the remote Git repository >>> git config remote.origin.url git://git.apache.org/lucene-solr.git # >>> timeout=10 >> Cleaning workspace >>> git rev-parse --verify HEAD # timeout=10 >> Resetting working tree >>> git reset --hard # timeout=10 >>> git clean -fdx # timeout=10 >> Fetching upstream changes from git://git.apache.org/lucene-solr.git >>> git --version # timeout=10 >>> git fetch --tags --progress git://git.apache.org/lucene-solr.git >>> +refs/heads/*:refs/remotes/origin/* >>> git rev-parse refs/remotes/origin/branch_7x^{commit} # timeout=10 >>> git rev-parse refs/remotes/origin/origin/branch_7x^{commit} # timeout=10 >> Checking out Revision e66ef6b992c248b66200ac97814951e5d312ab6c >> (refs/remotes/origin/branch_7x) >> Commit message: "SOLR-11105: Randomize PointFields in "cloud-minimal-jmx" >> and "cloud-minimal-inplace-updates" test configsets" >>> git config core.sparsecheckout # timeout=10 >>> git checkout -f e66ef6b992c248b66200ac97814951e5d312ab6c >>> git rev-list e66ef6b992c248b66200ac97814951e5d312ab6c # timeout=10 >> No emails were triggered. >> [Solr-reference-guide-7.x] $ /bin/bash -xe /tmp/jenkins8362459747270854181.sh >> + set -x >> + set -e >> + RVM_PATH=/home/jenkins/.rvm >> + RUBY_VERSION=ruby-2.3.3 >> + GEMSET=solr-refguide-gemset >> + curl -sSL https://get.rvm.io >> + bash -s -- --ignore-dotfiles stable >> Turning on ignore dotfiles mode. >> Downloading https://github.com/rvm/rvm/archive/1.29.2.tar.gz >> Downloading >> https://github.com/rvm/rvm/releases/download/1.29.2/1.29.2.tar.gz.asc >> Found PGP signature at: >> 'https://github.com/rvm/rvm/releases/download/1.29.2/1.29.2.tar.gz.asc', >> but no GPG software exists to validate it, skipping. >> >> Upgrading the RVM installation in /home/jenkins/.rvm/ >> Upgrade of RVM in /home/jenkins/.rvm/ is complete. >> >> # jenkins, >> # >> # Thank you for using RVM! >> # We sincerely hope that RVM helps to make your life easier and more >> enjoyable!!! >> # >> # ~Wayne, Michal & team. >> >> In case of problems: https://rvm.io/help and https://twitter.com/rvm_io >> >> Upgrade Notes: >> >> * No new notes to display. >> >> + set +x >> Running 'source /home/jenkins/.rvm/scripts/rvm' >> Running 'rvm autolibs disable' >> Running 'rvm install ruby-2.3.3' >> Already installed ruby-2.3.3. >> To reinstall use: >> >> rvm reinstall ruby-2.3.3 >> >> Running 'rvm gemset create solr-refguide-gemset' >> ruby-2.3.3 - #gemset created >> /home/jenkins/.rvm/gems/ruby-2.3.3@solr-refguide-gemset >> ruby-2.3.3 - #generating solr-refguide-gemset wrappers........ >> Running 'rvm ruby-2.3.3@solr-refguide-gemset' >> Using /home/jenkins/.rvm/gems/ruby-2.3.3 with gemset solr-refguide-gemset >> Running 'gem install --force --version 3.5.0 jekyll' >> Successfully installed jekyll-3.5.0 >> Parsing documentation for jekyll-3.5.0 >> Done installing documentation for jekyll after 2 seconds >> 1 gem installed >> Running 'gem install --force --version 2.1.0 jekyll-asciidoc' >> Successfully installed jekyll-asciidoc-2.1.0 >> Parsing documentation for jekyll-asciidoc-2.1.0 >> Done installing documentation for jekyll-asciidoc after 0 seconds >> 1 gem installed >> Running 'gem install --force --version 1.1.2 pygments.rb' >> Successfully installed pygments.rb-1.1.2 >> Parsing documentation for pygments.rb-1.1.2 >> Done installing documentation for pygments.rb after 0 seconds >> 1 gem installed >> + ant clean build-site build-pdf >> Buildfile: >> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/solr-ref-guide/build.xml >> >> clean: >> >> build-init: >> [mkdir] Created dir: >> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/content >> [echo] Copying all non template files from src ... >> [copy] Copying 343 files to >> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/content >> [echo] Copy (w/prop replacement) any template files from src... >> [copy] Copying 1 file to >> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/content >> >> ivy-availability-check: >> >> ivy-fail: >> >> ivy-configure: >> [ivy:configure] :: Apache Ivy 2.3.0 - 20130110142753 :: >> http://ant.apache.org/ivy/ :: >> [ivy:configure] :: loading settings :: file = >> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/lucene/top-level-ivy-settings.xml >> >> resolve: >> [ivy:retrieve] >> [ivy:retrieve] :: problems summary :: >> [ivy:retrieve] :::: ERRORS >> [ivy:retrieve] unknown resolver null >> [ivy:retrieve] unknown resolver null >> [ivy:retrieve] unknown resolver null >> [ivy:retrieve] unknown resolver null >> [ivy:retrieve] unknown resolver null >> [ivy:retrieve] unknown resolver null >> [ivy:retrieve] >> [ivy:retrieve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS >> >> build-tools-jar: >> [mkdir] Created dir: >> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/classes >> [javac] Compiling 3 source files to >> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/classes >> [jar] Building jar: >> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/solr-ref-guide-tools.jar >> >> build-nav-data-files: >> [java] Building up tree of all known pages >> [java] Creating >> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/content/_data/pdf-main-body.adoc >> [java] Creating >> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/content/_data/scrollnav.json >> [java] Creating >> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/content/_data/sidebar.json >> >> -build-site: >> [echo] Running Jekyll... >> [exec] Configuration file: >> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/content/_config.yml >> [exec] Source: >> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/content >> [exec] Destination: ../html-site >> [exec] Incremental build: disabled. Enable with --incremental >> [exec] Generating... >> [exec] [31m Deprecation: The 'gems' configuration option has been >> renamed to 'plugins'. Please update your config file accordingly. [0m >> [exec] done in 15.003 seconds. >> [exec] Auto-regeneration: disabled. Use --watch to enable. >> >> check-links-and-anchors: >> >> build-site: >> [echo] Ready to browse site: >> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/html-site/index.html >> >> build-init: >> [delete] Deleting directory >> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/content >> [mkdir] Created dir: >> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/content >> [echo] Copying all non template files from src ... >> [copy] Copying 343 files to >> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/content >> [echo] Copy (w/prop replacement) any template files from src... >> [copy] Copying 1 file to >> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/content >> >> ivy-availability-check: >> >> ivy-configure: >> [ivy:configure] :: loading settings :: file = >> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/lucene/top-level-ivy-settings.xml >> >> resolve: >> >> build-tools-jar: >> >> build-nav-data-files: >> [java] Building up tree of all known pages >> [java] Creating >> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/content/_data/pdf-main-body.adoc >> [java] Creating >> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/content/_data/scrollnav.json >> [java] Creating >> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/content/_data/sidebar.json >> >> -build-raw-pdf: >> [mkdir] Created dir: >> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/pdf-tmp >> [asciidoctor:convert] Using inherited external environment to resolve gems >> (/home/jenkins/.rvm/gems/ruby-2.3.3@solr-refguide-gemset), i.e. build is >> platform dependent! >> [asciidoctor:convert] Render SolrRefGuide-all.adoc from >> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/content/pdf >> to >> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/pdf-tmp >> with backend=pdf >> [asciidoctor:convert] Ignoring executable-hooks-1.3.2 because its extensions >> are not built. Try: gem pristine executable-hooks --version 1.3.2 >> [asciidoctor:convert] Ignoring ffi-1.9.18 because its extensions are not >> built. Try: gem pristine ffi --version 1.9.18 >> [asciidoctor:convert] Ignoring gem-wrappers-1.2.7 because its extensions are >> not built. Try: gem pristine gem-wrappers --version 1.2.7 >> [asciidoctor:convert] asciidoctor: ERROR: about-this-guide.adoc: line 1: >> invalid part, must have at least one section (e.g., chapter, appendix, etc.) >> [asciidoctor:convert] asciidoctor: ERROR: solr-glossary.adoc: line 1: >> invalid part, must have at least one section (e.g., chapter, appendix, etc.) >> [move] Moving 1 file to >> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/pdf-tmp >> >> -reduce-pdf-size: >> >> build-pdf: >> [echo] Finished Building >> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/apache-solr-ref-guide-7.1-DRAFT.pdf >> >> BUILD SUCCESSFUL >> Total time: 3 minutes 43 seconds >> Archiving artifacts >> ERROR: Failed to archive artifacts: solr/build/solr-ref-guide/*.pdf >> java.io.IOException: Too many open files >> at java.io.UnixFileSystem.createFileExclusively(Native Method) >> at java.io.File.createTempFile(File.java:2024) >> at hudson.util.AtomicFileWriter.<init>(AtomicFileWriter.java:68) >> Caused: java.io.IOException: Failed to create a temporary file in >> /x1/jenkins/jenkins-home/jobs/Solr-reference-guide-7.x/builds/156 >> at hudson.util.AtomicFileWriter.<init>(AtomicFileWriter.java:70) >> at hudson.util.AtomicFileWriter.<init>(AtomicFileWriter.java:57) >> at hudson.XmlFile.write(XmlFile.java:168) >> at hudson.model.Run.save(Run.java:1923) >> at hudson.model.Run.pickArtifactManager(Run.java:1065) >> at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:244) >> at >> hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81) >> at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) >> at >> hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:735) >> at >> hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:676) >> at hudson.model.Build$BuildExecution.post2(Build.java:186) >> at >> hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:621) >> at hudson.model.Run.execute(Run.java:1760) >> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) >> at hudson.model.ResourceController.execute(ResourceController.java:97) >> at hudson.model.Executor.run(Executor.java:405) >> Build step 'Archive the artifacts' changed build result to FAILURE >> Publishing Javadoc >> FATAL: Unable to copy Javadoc from >> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/html-site >> to /x1/jenkins/jenkins-home/jobs/Solr-reference-guide-7.x/builds/156/javadoc >> java.io.IOException: java.nio.file.FileSystemException: >> /x1/jenkins/jenkins-home/jobs/Solr-reference-guide-7.x/builds/156/javadoc/README.md: >> Too many open files >> at >> sun.nio.fs.UnixException.translateToIOException(UnixException.java:91) >> at >> sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) >> at >> sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) >> at >> sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214) >> at >> java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:434) >> at java.nio.file.Files.newOutputStream(Files.java:216) >> at hudson.util.IOUtils.copy(IOUtils.java:39) >> at hudson.FilePath.readFromTar(FilePath.java:2287) >> Caused: java.io.IOException: Failed to extract >> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/html-site/**/* >> at hudson.FilePath.readFromTar(FilePath.java:2297) >> at hudson.FilePath.copyRecursiveTo(FilePath.java:2207) >> at hudson.FilePath.copyRecursiveTo(FilePath.java:2115) >> at hudson.FilePath.copyRecursiveTo(FilePath.java:2098) >> at hudson.tasks.JavadocArchiver.perform(JavadocArchiver.java:108) >> at >> hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81) >> at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) >> at >> hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:735) >> at >> hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:676) >> at hudson.model.Build$BuildExecution.post2(Build.java:186) >> at >> hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:621) >> at hudson.model.Run.execute(Run.java:1760) >> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) >> at hudson.model.ResourceController.execute(ResourceController.java:97) >> at hudson.model.Executor.run(Executor.java:405) >> >> at hudson.FilePath.copyRecursiveTo(FilePath.java:2214) >> at hudson.FilePath.copyRecursiveTo(FilePath.java:2115) >> at hudson.FilePath.copyRecursiveTo(FilePath.java:2098) >> at hudson.tasks.JavadocArchiver.perform(JavadocArchiver.java:108) >> at >> hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81) >> at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) >> at >> hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:735) >> at >> hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:676) >> at hudson.model.Build$BuildExecution.post2(Build.java:186) >> at >> hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:621) >> at hudson.model.Run.execute(Run.java:1760) >> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) >> at hudson.model.ResourceController.execute(ResourceController.java:97) >> at hudson.model.Executor.run(Executor.java:405) >> Caused by: java.util.concurrent.ExecutionException: java.io.IOException: >> This archives contains unclosed entries. >> at hudson.remoting.Channel$2.adapt(Channel.java:862) >> at hudson.remoting.Channel$2.adapt(Channel.java:857) >> at hudson.remoting.FutureAdapter.get(FutureAdapter.java:59) >> at hudson.FilePath.copyRecursiveTo(FilePath.java:2210) >> ... 13 more >> Caused by: java.io.IOException: This archives contains unclosed entries. >> at >> org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.finish(TarArchiveOutputStream.java:225) >> at >> org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.close(TarArchiveOutputStream.java:241) >> at hudson.util.io.TarArchiver.close(TarArchiver.java:124) >> at hudson.FilePath.writeToTar(FilePath.java:2261) >> at hudson.FilePath.access$2100(FilePath.java:196) >> at hudson.FilePath$45.invoke(FilePath.java:2202) >> at hudson.FilePath$45.invoke(FilePath.java:2198) >> at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2739) >> at hudson.remoting.UserRequest.perform(UserRequest.java:153) >> at hudson.remoting.UserRequest.perform(UserRequest.java:50) >> at hudson.remoting.Request$2.run(Request.java:336) >> at >> hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) >> at java.util.concurrent.FutureTask.run(FutureTask.java:266) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >> at java.lang.Thread.run(Thread.java:748) >> at ......remote call to H19(Native Method) >> at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1545) >> at hudson.remoting.UserResponse.retrieve(UserRequest.java:253) >> at hudson.remoting.Channel$2.adapt(Channel.java:860) >> ... 16 more >> Email was triggered for: Failure - Any >> Sending email for trigger: Failure - Any >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
