An update from Chris Lambertus on [email protected] mailing list, about 2 hours ago:
> This appears to be a bug in one of the new versions of either jenkins or a > plugin. We are still looking into the issue. The limit was raised yesterday > by a factor of 4. -- Steve www.lucidworks.com > On Jul 18, 2017, at 11:49 AM, Steve Rowe <[email protected]> wrote: > > On the [email protected] mailing list last night, on thread “jenkins > emergency restart notice”, Chris Lambertus wrote: > >> Jenkins will be restarted at midnight tonight UTC to address >> problems related to ‘too many open files’ and to attempt to >> correct a problem with some windows agents. I will be stopping >> new builds so that the queue can empty over the next 2 hours. > > This morning, two people responded on that thread that builds for the Apache > Kafka and Apache Brooklyn projects are still encountering this problem. > > I’ll keep an eye on this thread. Hopefully they’ll figure it out shortly. > It’s possible that our issue is different from that of other projects; if the > situation isn’t resolved in the next 24 hours I’ll ask on hipchat. > > -- > Steve > www.lucidworks.com > >> On Jul 18, 2017, at 11:28 AM, Cassandra Targett <[email protected]> >> wrote: >> >> This is the 2nd one of these I've seen in 2 days (different jobs) but >> it seems to resolve itself on the next run. >> >> Steve Rowe, do you have any insight into what we can do to avoid it? >> I'll be happy to ask INFRA if not. >> >> On Tue, Jul 18, 2017 at 7:32 AM, Apache Jenkins Server >> <[email protected]> wrote: >>> Build: https://builds.apache.org/job/Solr-reference-guide-7.x/156/ >>> >>> Log: >>> Started by timer >>> [EnvInject] - Loading node environment variables. >>> Building remotely on H19 (git-websites) in workspace >>> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x >>>> git rev-parse --is-inside-work-tree # timeout=10 >>> Fetching changes from the remote Git repository >>>> git config remote.origin.url git://git.apache.org/lucene-solr.git # >>>> timeout=10 >>> Cleaning workspace >>>> git rev-parse --verify HEAD # timeout=10 >>> Resetting working tree >>>> git reset --hard # timeout=10 >>>> git clean -fdx # timeout=10 >>> Fetching upstream changes from git://git.apache.org/lucene-solr.git >>>> git --version # timeout=10 >>>> git fetch --tags --progress git://git.apache.org/lucene-solr.git >>>> +refs/heads/*:refs/remotes/origin/* >>>> git rev-parse refs/remotes/origin/branch_7x^{commit} # timeout=10 >>>> git rev-parse refs/remotes/origin/origin/branch_7x^{commit} # timeout=10 >>> Checking out Revision e66ef6b992c248b66200ac97814951e5d312ab6c >>> (refs/remotes/origin/branch_7x) >>> Commit message: "SOLR-11105: Randomize PointFields in "cloud-minimal-jmx" >>> and "cloud-minimal-inplace-updates" test configsets" >>>> git config core.sparsecheckout # timeout=10 >>>> git checkout -f e66ef6b992c248b66200ac97814951e5d312ab6c >>>> git rev-list e66ef6b992c248b66200ac97814951e5d312ab6c # timeout=10 >>> No emails were triggered. >>> [Solr-reference-guide-7.x] $ /bin/bash -xe >>> /tmp/jenkins8362459747270854181.sh >>> + set -x >>> + set -e >>> + RVM_PATH=/home/jenkins/.rvm >>> + RUBY_VERSION=ruby-2.3.3 >>> + GEMSET=solr-refguide-gemset >>> + curl -sSL https://get.rvm.io >>> + bash -s -- --ignore-dotfiles stable >>> Turning on ignore dotfiles mode. >>> Downloading https://github.com/rvm/rvm/archive/1.29.2.tar.gz >>> Downloading >>> https://github.com/rvm/rvm/releases/download/1.29.2/1.29.2.tar.gz.asc >>> Found PGP signature at: >>> 'https://github.com/rvm/rvm/releases/download/1.29.2/1.29.2.tar.gz.asc', >>> but no GPG software exists to validate it, skipping. >>> >>> Upgrading the RVM installation in /home/jenkins/.rvm/ >>> Upgrade of RVM in /home/jenkins/.rvm/ is complete. >>> >>> # jenkins, >>> # >>> # Thank you for using RVM! >>> # We sincerely hope that RVM helps to make your life easier and more >>> enjoyable!!! >>> # >>> # ~Wayne, Michal & team. >>> >>> In case of problems: https://rvm.io/help and https://twitter.com/rvm_io >>> >>> Upgrade Notes: >>> >>> * No new notes to display. >>> >>> + set +x >>> Running 'source /home/jenkins/.rvm/scripts/rvm' >>> Running 'rvm autolibs disable' >>> Running 'rvm install ruby-2.3.3' >>> Already installed ruby-2.3.3. >>> To reinstall use: >>> >>> rvm reinstall ruby-2.3.3 >>> >>> Running 'rvm gemset create solr-refguide-gemset' >>> ruby-2.3.3 - #gemset created >>> /home/jenkins/.rvm/gems/ruby-2.3.3@solr-refguide-gemset >>> ruby-2.3.3 - #generating solr-refguide-gemset wrappers........ >>> Running 'rvm ruby-2.3.3@solr-refguide-gemset' >>> Using /home/jenkins/.rvm/gems/ruby-2.3.3 with gemset solr-refguide-gemset >>> Running 'gem install --force --version 3.5.0 jekyll' >>> Successfully installed jekyll-3.5.0 >>> Parsing documentation for jekyll-3.5.0 >>> Done installing documentation for jekyll after 2 seconds >>> 1 gem installed >>> Running 'gem install --force --version 2.1.0 jekyll-asciidoc' >>> Successfully installed jekyll-asciidoc-2.1.0 >>> Parsing documentation for jekyll-asciidoc-2.1.0 >>> Done installing documentation for jekyll-asciidoc after 0 seconds >>> 1 gem installed >>> Running 'gem install --force --version 1.1.2 pygments.rb' >>> Successfully installed pygments.rb-1.1.2 >>> Parsing documentation for pygments.rb-1.1.2 >>> Done installing documentation for pygments.rb after 0 seconds >>> 1 gem installed >>> + ant clean build-site build-pdf >>> Buildfile: >>> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/solr-ref-guide/build.xml >>> >>> clean: >>> >>> build-init: >>> [mkdir] Created dir: >>> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/content >>> [echo] Copying all non template files from src ... >>> [copy] Copying 343 files to >>> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/content >>> [echo] Copy (w/prop replacement) any template files from src... >>> [copy] Copying 1 file to >>> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/content >>> >>> ivy-availability-check: >>> >>> ivy-fail: >>> >>> ivy-configure: >>> [ivy:configure] :: Apache Ivy 2.3.0 - 20130110142753 :: >>> http://ant.apache.org/ivy/ :: >>> [ivy:configure] :: loading settings :: file = >>> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/lucene/top-level-ivy-settings.xml >>> >>> resolve: >>> [ivy:retrieve] >>> [ivy:retrieve] :: problems summary :: >>> [ivy:retrieve] :::: ERRORS >>> [ivy:retrieve] unknown resolver null >>> [ivy:retrieve] unknown resolver null >>> [ivy:retrieve] unknown resolver null >>> [ivy:retrieve] unknown resolver null >>> [ivy:retrieve] unknown resolver null >>> [ivy:retrieve] unknown resolver null >>> [ivy:retrieve] >>> [ivy:retrieve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS >>> >>> build-tools-jar: >>> [mkdir] Created dir: >>> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/classes >>> [javac] Compiling 3 source files to >>> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/classes >>> [jar] Building jar: >>> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/solr-ref-guide-tools.jar >>> >>> build-nav-data-files: >>> [java] Building up tree of all known pages >>> [java] Creating >>> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/content/_data/pdf-main-body.adoc >>> [java] Creating >>> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/content/_data/scrollnav.json >>> [java] Creating >>> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/content/_data/sidebar.json >>> >>> -build-site: >>> [echo] Running Jekyll... >>> [exec] Configuration file: >>> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/content/_config.yml >>> [exec] Source: >>> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/content >>> [exec] Destination: ../html-site >>> [exec] Incremental build: disabled. Enable with --incremental >>> [exec] Generating... >>> [exec] [31m Deprecation: The 'gems' configuration option has been >>> renamed to 'plugins'. Please update your config file accordingly. [0m >>> [exec] done in 15.003 seconds. >>> [exec] Auto-regeneration: disabled. Use --watch to enable. >>> >>> check-links-and-anchors: >>> >>> build-site: >>> [echo] Ready to browse site: >>> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/html-site/index.html >>> >>> build-init: >>> [delete] Deleting directory >>> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/content >>> [mkdir] Created dir: >>> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/content >>> [echo] Copying all non template files from src ... >>> [copy] Copying 343 files to >>> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/content >>> [echo] Copy (w/prop replacement) any template files from src... >>> [copy] Copying 1 file to >>> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/content >>> >>> ivy-availability-check: >>> >>> ivy-configure: >>> [ivy:configure] :: loading settings :: file = >>> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/lucene/top-level-ivy-settings.xml >>> >>> resolve: >>> >>> build-tools-jar: >>> >>> build-nav-data-files: >>> [java] Building up tree of all known pages >>> [java] Creating >>> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/content/_data/pdf-main-body.adoc >>> [java] Creating >>> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/content/_data/scrollnav.json >>> [java] Creating >>> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/content/_data/sidebar.json >>> >>> -build-raw-pdf: >>> [mkdir] Created dir: >>> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/pdf-tmp >>> [asciidoctor:convert] Using inherited external environment to resolve gems >>> (/home/jenkins/.rvm/gems/ruby-2.3.3@solr-refguide-gemset), i.e. build is >>> platform dependent! >>> [asciidoctor:convert] Render SolrRefGuide-all.adoc from >>> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/content/pdf >>> to >>> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/pdf-tmp >>> with backend=pdf >>> [asciidoctor:convert] Ignoring executable-hooks-1.3.2 because its >>> extensions are not built. Try: gem pristine executable-hooks --version >>> 1.3.2 >>> [asciidoctor:convert] Ignoring ffi-1.9.18 because its extensions are not >>> built. Try: gem pristine ffi --version 1.9.18 >>> [asciidoctor:convert] Ignoring gem-wrappers-1.2.7 because its extensions >>> are not built. Try: gem pristine gem-wrappers --version 1.2.7 >>> [asciidoctor:convert] asciidoctor: ERROR: about-this-guide.adoc: line 1: >>> invalid part, must have at least one section (e.g., chapter, appendix, etc.) >>> [asciidoctor:convert] asciidoctor: ERROR: solr-glossary.adoc: line 1: >>> invalid part, must have at least one section (e.g., chapter, appendix, etc.) >>> [move] Moving 1 file to >>> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/pdf-tmp >>> >>> -reduce-pdf-size: >>> >>> build-pdf: >>> [echo] Finished Building >>> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/apache-solr-ref-guide-7.1-DRAFT.pdf >>> >>> BUILD SUCCESSFUL >>> Total time: 3 minutes 43 seconds >>> Archiving artifacts >>> ERROR: Failed to archive artifacts: solr/build/solr-ref-guide/*.pdf >>> java.io.IOException: Too many open files >>> at java.io.UnixFileSystem.createFileExclusively(Native Method) >>> at java.io.File.createTempFile(File.java:2024) >>> at hudson.util.AtomicFileWriter.<init>(AtomicFileWriter.java:68) >>> Caused: java.io.IOException: Failed to create a temporary file in >>> /x1/jenkins/jenkins-home/jobs/Solr-reference-guide-7.x/builds/156 >>> at hudson.util.AtomicFileWriter.<init>(AtomicFileWriter.java:70) >>> at hudson.util.AtomicFileWriter.<init>(AtomicFileWriter.java:57) >>> at hudson.XmlFile.write(XmlFile.java:168) >>> at hudson.model.Run.save(Run.java:1923) >>> at hudson.model.Run.pickArtifactManager(Run.java:1065) >>> at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:244) >>> at >>> hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81) >>> at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) >>> at >>> hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:735) >>> at >>> hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:676) >>> at hudson.model.Build$BuildExecution.post2(Build.java:186) >>> at >>> hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:621) >>> at hudson.model.Run.execute(Run.java:1760) >>> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) >>> at hudson.model.ResourceController.execute(ResourceController.java:97) >>> at hudson.model.Executor.run(Executor.java:405) >>> Build step 'Archive the artifacts' changed build result to FAILURE >>> Publishing Javadoc >>> FATAL: Unable to copy Javadoc from >>> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/html-site >>> to >>> /x1/jenkins/jenkins-home/jobs/Solr-reference-guide-7.x/builds/156/javadoc >>> java.io.IOException: java.nio.file.FileSystemException: >>> /x1/jenkins/jenkins-home/jobs/Solr-reference-guide-7.x/builds/156/javadoc/README.md: >>> Too many open files >>> at >>> sun.nio.fs.UnixException.translateToIOException(UnixException.java:91) >>> at >>> sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) >>> at >>> sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) >>> at >>> sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214) >>> at >>> java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:434) >>> at java.nio.file.Files.newOutputStream(Files.java:216) >>> at hudson.util.IOUtils.copy(IOUtils.java:39) >>> at hudson.FilePath.readFromTar(FilePath.java:2287) >>> Caused: java.io.IOException: Failed to extract >>> /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-7.x/solr/build/solr-ref-guide/html-site/**/* >>> at hudson.FilePath.readFromTar(FilePath.java:2297) >>> at hudson.FilePath.copyRecursiveTo(FilePath.java:2207) >>> at hudson.FilePath.copyRecursiveTo(FilePath.java:2115) >>> at hudson.FilePath.copyRecursiveTo(FilePath.java:2098) >>> at hudson.tasks.JavadocArchiver.perform(JavadocArchiver.java:108) >>> at >>> hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81) >>> at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) >>> at >>> hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:735) >>> at >>> hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:676) >>> at hudson.model.Build$BuildExecution.post2(Build.java:186) >>> at >>> hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:621) >>> at hudson.model.Run.execute(Run.java:1760) >>> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) >>> at hudson.model.ResourceController.execute(ResourceController.java:97) >>> at hudson.model.Executor.run(Executor.java:405) >>> >>> at hudson.FilePath.copyRecursiveTo(FilePath.java:2214) >>> at hudson.FilePath.copyRecursiveTo(FilePath.java:2115) >>> at hudson.FilePath.copyRecursiveTo(FilePath.java:2098) >>> at hudson.tasks.JavadocArchiver.perform(JavadocArchiver.java:108) >>> at >>> hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81) >>> at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) >>> at >>> hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:735) >>> at >>> hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:676) >>> at hudson.model.Build$BuildExecution.post2(Build.java:186) >>> at >>> hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:621) >>> at hudson.model.Run.execute(Run.java:1760) >>> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) >>> at hudson.model.ResourceController.execute(ResourceController.java:97) >>> at hudson.model.Executor.run(Executor.java:405) >>> Caused by: java.util.concurrent.ExecutionException: java.io.IOException: >>> This archives contains unclosed entries. >>> at hudson.remoting.Channel$2.adapt(Channel.java:862) >>> at hudson.remoting.Channel$2.adapt(Channel.java:857) >>> at hudson.remoting.FutureAdapter.get(FutureAdapter.java:59) >>> at hudson.FilePath.copyRecursiveTo(FilePath.java:2210) >>> ... 13 more >>> Caused by: java.io.IOException: This archives contains unclosed entries. >>> at >>> org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.finish(TarArchiveOutputStream.java:225) >>> at >>> org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.close(TarArchiveOutputStream.java:241) >>> at hudson.util.io.TarArchiver.close(TarArchiver.java:124) >>> at hudson.FilePath.writeToTar(FilePath.java:2261) >>> at hudson.FilePath.access$2100(FilePath.java:196) >>> at hudson.FilePath$45.invoke(FilePath.java:2202) >>> at hudson.FilePath$45.invoke(FilePath.java:2198) >>> at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2739) >>> at hudson.remoting.UserRequest.perform(UserRequest.java:153) >>> at hudson.remoting.UserRequest.perform(UserRequest.java:50) >>> at hudson.remoting.Request$2.run(Request.java:336) >>> at >>> hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) >>> at java.util.concurrent.FutureTask.run(FutureTask.java:266) >>> at >>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >>> at java.lang.Thread.run(Thread.java:748) >>> at ......remote call to H19(Native Method) >>> at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1545) >>> at hudson.remoting.UserResponse.retrieve(UserRequest.java:253) >>> at hudson.remoting.Channel$2.adapt(Channel.java:860) >>> ... 16 more >>> Email was triggered for: Failure - Any >>> Sending email for trigger: Failure - Any >>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: [email protected] >>> For additional commands, e-mail: [email protected] >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
