Got to the bottom of this via stackoverflow <http://stackoverflow.com/questions/33027437/insufficient-memory-on-jenkins-server-startup>: it was an issue with the glibc distributed with Red Hat Enterprise Linux 6 as discussed here <https://issues.apache.org/jira/browse/HADOOP-7154>. The issue can be worked around via explicit setting of the environment variable MALLOC_ARENA_MAX, in my case export MALLOC_ARENA_MAX=2. Without explicit setting of this variable, the JVM will apparently attempt to spawn (8 x cpu core count) threads, each consuming 64M. My 40 core case would have required northward of 10 gigs of virtual ram, exceeding (by itself) the ulimit on my machine of 8 gigs. Setting this to 2 reduces VM consumption to around 128 megs, though I'll probably increase it to something like 16 when I start using Jenkins seriously.
On Tuesday, October 6, 2015 at 10:00:36 AM UTC-4, Trevor Williams wrote: > > Ugh, correcting spelling of the subject. > > Me fail English? That's unpossible! > > On Monday, October 5, 2015 at 7:27:01 PM UTC-4, Trevor Williams wrote: >> >> Hi all, >> >> First time user of Jenkins here, and having a bit of trouble getting it >> started. From the Linux shell I run a command like: >> >> java -Xms512m -Xmx512m -jar jenkins.war >> >> and consistently get an error like: >> >> # There is insufficient memory for the Java Runtime Environment to >> continue. >> # pthread_getattr_np >> # An error report file with more information is saved as: >> # /home/twilliams/.jenkins/hs_err_pid36290.log >> >> First, the basics: >> >> - Jenkins 1.631 >> - Running via the jetty embedded in the war file >> - OpenJDK 1.7.0_51 >> - Oracle Linux (3.8.13-55.1.5.el6uek.x86_64) >> - 386 GB ram >> - 40 cores >> >> I get the same problem with a number of other configurations as well: >> using Java Hotspot 1.8.0_60, running through Apache Tomcat, and using all >> sorts of different values for -Xms/-Xmx/-Xss and similar options. >> >> I've done a fair bit of research and *think* I know what the problem is, >> but am at a loss as how to solve it. I suspect that I'm running into the >> virtual memory overcommit issue mentioned here >> <http://serverfault.com/questions/317115/jenkins-ci-cannot-allocate-memory>; >> the relevant bits from ulimit: >> >> --($:)-- ulimit -a >> ... >> max locked memory (kbytes, -l) 64 >> max memory size (kbytes, -m) 8388608 >> stack size (kbytes, -s) 8192 >> virtual memory (kbytes, -v) 8388608 >> ... >> >> If I double the virtual memory limit as root, I can start Jenkins, but >> I'd rather not run Jenkins as the root user. >> >> Another workaround: a soon-to-be-decommissioned machine with 48 GB ram >> and 24 cores can start Jenkins without issue, though (I suspect) just >> barely: according to htop, its virtual memory footprint is just over 8 GB. >> I suspect, as a result, that the memory overcommit issue is scaling with >> the number of processors on the machine, presumably the result of Jenkins >> starting a number of threads proportional to the number of processors >> present on the host machine. I roughly captured the thread count via ps >> -eLf | grep jenkins | wc -l and found that the thread count spikes at >> around 114 on the 40 core machine, and 84 on the 24 core machine. >> >> Does this explanation seem sound? Provided it does... >> >> 1. Is there any way to configure Jenkins to reduce the number of >> threads it spawns at startup? >> 2. Are there any VMs available that don't suffer from the overcommit >> issue, or some configuration option to address it? >> >> The sanest option at this point may be to just run Jenkins in a >> virtualized environment to limit the resources at its disposal to something >> reasonable, but at this point I'm interested in this problem on an >> intellectual level and want to know how to get this recalcitrant >> configuration to behave. >> >> Thanks, >> Trevor >> > -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/aba1b6a4-6461-457f-a826-d3aa4b0bb0fd%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
