Re: Memory issues with Jenkins

We haven't had this OutOfMemoryError now for 3 weeks running Jenkins. We did four things. 1) Reduced master executors from 15 to 4 2) Reduced some job steps running on "master" and instead use a build agent for these steps. We still have one stage/step that needs to run on master. 3 Configured ma

Re: Memory issues with Jenkins

I created a Pipeline job to run jstack every 10 minutes (though running on Jenkins master since that is where the Jenkins is running). onsdag 14. august 2019 16.07.02 UTC+2 skrev Félix Belzunce Arcos følgende: > > Hi Sverre Moe, > > I am the person who talked to you this morning :-) > > Long term

Re: Memory issues with Jenkins

I only have the option to downgrade to 1.18 of Server Sent Events (SSE) Gateway Plugin I would have to download the 1.17 and manually downgrade it. >From the discussions it seems I also need to downgrade BlueOcean to 1.17 > Downgrading to BlueOcean 1.17 (which in turn uses sse-gateway 1.17) appe

Re: Memory issues with Jenkins

Hi Sverre Moe, I am the person who talked to you this morning :-) Long term solution is to avoid building on the master to avoid performance issue and the need to increase the number of processes and open files in the machine where the jenkins master is located. Building on the master is also

Re: Memory issues with Jenkins

I have not read the whole thread in detail, but the “Unable to create new native thread” OutOfMemoryErrors from your original thread where one of the stack traces involves org.jenkinsci.plugins.ssegateway.sse.EventDispatcher.scheduleRetryQueueProcessing looks like it could be related to https:

Re: Memory issues with Jenkins

We got an 30 minute free CloudBees support. It was too short to dig deeper to find the problem, but the person I was talking to (after examining our logs) mentioned what he thought was the problem and gave a suggestion. We should not use Jenkins master at all for builds (allocated with the node

Re: Memory issues with Jenkins

Sadly I was mistaken. We do not use NFS for JENKINS_HOME. We do however use NFS for the location where builds copy the RPM build artifacts. mandag 5. august 2019 22.17.46 UTC+2 skrev Ivan Fernandez Calvo følgende: > > Hi, > > Severe has another email thread open, I think it is the same Jenkins

Re: Memory issues with Jenkins

Hi, Severe has another email thread open, I think it is the same Jenkins instance https://groups.google.com/d/msgid/jenkinsci-users/cc2d0bdb-b15f-4bec-a0a3-0562ea8c7df7%40googlegroups.com?utm_medium=email&utm_source=footer. I dunno what happens on your instance but probably it isn’t better that

Re: Memory issues with Jenkins

We don't have NFS for JENKINS_HOME. Master and all clients are using local file system. Anyway, I can see the stack count going extremely high before going into OutOfMemory and then disconnecting all agents. But we do have a network cifs folder where all agents are reading and writing to lot's

Re: Memory issues with Jenkins

if it is the same instance with the agent disconnection errors, your problem is the NFS performance, it blocks tons of threads in IO operations that make agents cannot disconnect or connects blocking thousands of threads, the Jenkins instance works until is collapsed by the number of threads bl

Re: Memory issues with Jenkins

Have not yet resolved the issue. Not found a solution, nor what the cause actually is. lørdag 3. august 2019 13.04.10 UTC+2 skrev guerkan demirci følgende: > > Hi Sverre Moe, > > could you resolve the issue? > I have the same problem with Jenkins. > > Best > > > Am Montag, 29. Juli 2019 11:30:58

Re: Memory issues with Jenkins

Hi Sverre Moe, could you resolve the issue? I have the same problem with Jenkins. Best Am Montag, 29. Juli 2019 11:30:58 UTC+2 schrieb Sverre Moe: > > I have analyzed the GC log > > > https://www.gceasy.io/my-gc-report.jsp?p=c2hhcmVkLzIwMTkvMDcvMjgvLS1nYy0yMDE5LTA3LTI2XzE3LTMxLTIyLmxvZy4wLmN1cn

Re: Memory issues with Jenkins

I have analyzed the GC log https://www.gceasy.io/my-gc-report.jsp?p=c2hhcmVkLzIwMTkvMDcvMjgvLS1nYy0yMDE5LTA3LTI2XzE3LTMxLTIyLmxvZy4wLmN1cnJlbnQtLTE1LTI0LTQy&channel=WEB This may be related to another issue I am having when build agents are getting disconnected. https://groups.google.com/forum/#!

Re: Memory issues with Jenkins

I checked out that link Tried the ulimit settings https://support.cloudbees.com/hc/en-us/articles/222446987-Prepare-Jenkins-for-Support#bulimitsettingsjustforlinuxos Will try the additional JVM flags suggested in https://support.cloudbees.com/hc/en-us/articles/222446987-Prepare-Jenkins-for-Suppor

Re: Memory issues with Jenkins

jenkins@meoscorebs12:~> ulimit -a core file size (blocks, -c) unlimited data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 80229 max locked memory (kbytes, -l)

Re: Memory issues with Jenkins

Hi, When the issue happens, Did you check the number of threads that Jenkins has open? How many file descriptors can your process open (run ulimit -a with the user jenkins)? here you have a good KB about memory and user limit on Jenkins Prepare Jenkins for Support

Re: Memory issues with Jenkins

Could it be issue with the virtual memory in the jenkins server? Because Jenkins does consume a lot of virtual memory. PID USER PR NIVIRTRESSHR S %CPU %MEM TIME+ COMMAND 13565 jenkins 20 0 12.641g 0.011t 13552 S 0.000 56.62 6

Re: Memory issues with Jenkins

Since we don't use 32bit, the reason must be - the virtual memory of the OS has been fully depleted How can I check for this, and remedy it? fredag 5. juli 2019 03.17.38 UTC+2 skrev Jan Monterrubio følgende: > Correct me if I’m wrong but I don’t think increasing heap size will > actually affec

Re: Memory issues with Jenkins

I will try turning on GC logging. torsdag 4. juli 2019 23.04.55 UTC+2 skrev Baptiste Mathus følgende: > > Did you enable GC logging to have a better understanding of the profile of > your memory consumption? If not, I would recommend you do it first and > analyze them. > https://jenkins.io/blog/

Re: Memory issues with Jenkins

Correct me if I’m wrong but I don’t think increasing heap size will actually affect your ability to create more native threads. See this for a possible explanation: https://plumbr.io/outofmemoryerror/unable-to-create-new-native-thread On Thu, Jul 4, 2019 at 16:03 Baptiste Mathus wrote: > Did yo

Re: Memory issues with Jenkins

Did you enable GC logging to have a better understanding of the profile of your memory consumption? If not, I would recommend you do it first and analyze them. https://jenkins.io/blog/2016/11/21/gc-tuning/ explained this part (and much more) quite well. Then, once you understand better when it cra

Re: Memory issues with Jenkins

Today it has been chaotic. Several build agents disconnected Unexpected termination of the channel Many builds failed because of Memory error. I have tried restarting Jenkins several times today. Anyone have any suggestions? tirsdag 2. juli 2019 14.34.25 UTC+2 skrev Sverre Moe følgende: > > We

Re: Memory issues with Jenkins

On Nov 9, 2012, at 7:08 AM, David Weintraub wrote: > There's not a lot of jobs running on this. We're just getting Jenkins setup, > and maybe get one or two builds per day. The problem seems to stem from the > Analytic plugins: Findbugs, PMD, Checkstyle, Warnings, and CPD. The build > complete

Re: Memory issues with Jenkins

ble in the chart. Mark Waite > > From: Dirk Weigenand >To: jenkinsci-users@googlegroups.com >Sent: Saturday, November 10, 2012 12:48 PM >Subject: Re: Memory issues with Jenkins > >Hi, > >the static analysis plugins hold their state

Re: Memory issues with Jenkins

Hi, the static analysis plugins hold their state in Maps. At least this has been the state of affairs last tie i looked. Kohsuke supplied patches that reduce this amount but when you have the count of warnings you mentioned it is simply a problem of allocating memory to your application server ins

Re: Memory issues with Jenkins

Check your build where you are mentioning ant task..take advanced option adjacent to it...there give jvm memory... as argument.. On Fri, Nov 9, 2012 at 8:38 PM, David Weintraub wrote: > There's not a lot of jobs running on this. We're just getting Jenkins > setup, and maybe get one or two builds

Re: Memory issues with Jenkins

There's not a lot of jobs running on this. We're just getting Jenkins setup, and maybe get one or two builds per day. The problem seems to stem from the Analytic plugins: Findbugs, PMD, Checkstyle, Warnings, and CPD. The build completes fine, but when it is calculating the issues from the previo

RE: Memory issues with Jenkins

nkins Wiki? --Rob From: jenkinsci-users@googlegroups.com [mailto:jenkinsci-users@googlegroups.com] On Behalf Of Varghese Renny Sent: Friday, November 09, 2012 12:15 AM To: jenkinsci-users@googlegroups.com Subject: Re: Memory issues with Jenkins Hi David, You have to set JVM in two places, on

Re: Memory issues with Jenkins

Hi David, You have to set JVM in two places, one for system, one for the particular job you are running..check it out..I think 2GB memory is more than enough for one job..You can analze it through monitoring plugin.. Options are you can dump your heap memory to some location in your system an