Ok, I did analyse the heap dump with Eclipse's Memory Analyzer Tool and here's what I found:
One instance of "java.util.concurrent.ScheduledThreadPoolExecutor" loaded by "<system class loader>" occupies 6,801,517,856 (89.16%) bytes. The instance is referenced by org.tmatesoft.svn.core.wc.DefaultSVNRepositoryPool @ 0x561b36858 , loaded by "hudson.ClassicPluginStrategy$AntClassLoader2 @ 0x56057c428". The memory is accumulated in one instance of "java.util.concurrent.RunnableScheduledFuture[]" loaded by "<system class loader>". Does it match JENKINS-34213? On Tuesday, May 17, 2016 at 9:29:16 AM UTC-4, Stephen Connolly wrote: > > Jenkins 2.5 has the fix IIUC > > On 17 May 2016 at 14:28, Stephen Connolly <[email protected] > <javascript:>> wrote: > >> It is fixed, but in the latest version of remoting... which is not >> something you can upgrade without either building a custom build of Jenkins >> or upgrading Jenkins. >> >> You could crack open your jenkins.war and replace the remoting.jar with >> the fixed version and seal it back up again and see if that fixes your >> issue... probably the quickest way to confirm if my theory as to the source >> of your leak is correct, but you would need to know what you are doing >> >> On 17 May 2016 at 14:14, Ugo Bellavance <[email protected] <javascript:>> >> wrote: >> >>> Thanks, but I can see that this issue is fixed but I don't know how to >>> update my Jenkins install so that I have the fix. I'm using 1.656. >>> >>> Ugo >>> >>> On Tuesday, May 17, 2016 at 8:56:34 AM UTC-4, Stephen Connolly wrote: >>>> >>>> I will repeat: >>>> >>>> > I am suspecting JENKINS-34213 may be your issue. >>>> >>>> On 17 May 2016 at 13:12, Ugo Bellavance <[email protected]> wrote: >>>> >>>>> >>>>> >>>>> On Friday, May 13, 2016 at 8:50:56 AM UTC-4, Ugo Bellavance wrote: >>>>>> >>>>>> >>>>>> >>>>>> On Thursday, May 12, 2016 at 2:01:41 PM UTC-4, Raymond Accary wrote: >>>>>>> >>>>>>> Hi, >>>>>>> If it helps, you might avoid the crash by installing the monitoring >>>>>>> plugin, and triggering garbage collection once the memory is >>>>>>> approaching >>>>>>> the maximum allocated heap size. This is a workaround until someone is >>>>>>> able >>>>>>> to diagnose the root cause. I have an open issue : >>>>>>> https://issues.jenkins-ci.org/browse/JENKINS-34573 but thought I'd >>>>>>> run the suggestion. >>>>>>> >>>>>>>> >>>>>>>> >>>>>> Have anyone monitored the JVM with VisualVM? I have found that it >>>>>> looks like a specific memory pool may be filling up: Old Gen. I'm >>>>>> trying >>>>>> now with: >>>>>> >>>>>> -Djava.awt.headless=true -Dcom.sun.management.jmxremote -Xms10752m >>>>>> -Xmx10752m -XX:NewRatio=10 >>>>>> >>>>>> We'll see in a few days. In the last few days, the problem have >>>>>> occured each morning at around 4:45. I'm not sure if the same jobs run >>>>>> on >>>>>> the week-ends as well. >>>>>> >>>>> >>>>> From what I can see, it will always fill the memory. Probably a leak, >>>>> but I don't know how to be 100% sure about it. When I try to load a heap >>>>> dump in VisualVM, but it shows nothing except "Not supported for this >>>>> JVM. >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Jenkins Users" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/jenkinsci-users/1b43cd1d-531b-4638-8e53-15d5f5670f91%40googlegroups.com >>>>> >>>>> <https://groups.google.com/d/msgid/jenkinsci-users/1b43cd1d-531b-4638-8e53-15d5f5670f91%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Jenkins Users" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected] <javascript:>. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/jenkinsci-users/145bf27e-4bcc-4444-b46c-9d8fe4efc1a2%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/jenkinsci-users/145bf27e-4bcc-4444-b46c-9d8fe4efc1a2%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> > -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/a7867041-c143-4d08-a7ee-22e4b78a4418%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
