https://issues.jenkins-ci.org/browse/JENKINS-34919
On Wednesday, May 18, 2016 at 10:45:44 AM UTC-4, Ugo Bellavance wrote: > > We only have 1 Jenkins server so I'll create a JIRA, > > Thanks a lot for your help :)! > > On Wednesday, May 18, 2016 at 10:40:21 AM UTC-4, Stephen Connolly wrote: >> >> If you have no slaves and only run builds on the master then we can >> definitively state that JENKINS-34573 is not the root cause. If you have >> slaves (even if idle or unused) then we cannot. >> >> If you have a definitive answer to the above question then you should >> probably create a JIRA >> >> On 18 May 2016 at 14:29, Ugo Bellavance <[email protected]> wrote: >> >>> Hi Stephen, >>> >>> We don't use slaves. We only have one master. >>> >>> Should I create an issue in JIRA for my specific issue? >>> >>> (is there a reason why you got the conservation out of the group?) >>> >>> Thanks, >>> >>> On Wed, May 18, 2016 at 8:56 AM, Stephen Connolly < >>> [email protected]> wrote: >>> >>>> Hard to say for certain as that could still be references maintained by >>>> the unexported objects that are pending unexport. If you can get your >>>> instance into that state and then have it sit idle for a couple of >>>> hours... >>>> if it is JENKINS-34573 then you should see the retained memory reduce as >>>> one reference is cleared every minute. Alternatively disconnect and >>>> reconnect all your slaves and see if that frees the memory up - >>>> JENKINS-34573 is only an issue for long running busy slaves >>>> >>>> On 17 May 2016 at 19:01, Ugo Bellavance <[email protected]> wrote: >>>> >>>>> Ok, I did analyse the heap dump with Eclipse's Memory Analyzer Tool >>>>> and here's what I found: >>>>> >>>>> One instance of "java.util.concurrent.ScheduledThreadPoolExecutor" >>>>> loaded by "<system class loader>" occupies 6,801,517,856 (89.16%) bytes. >>>>> The instance is referenced by >>>>> org.tmatesoft.svn.core.wc.DefaultSVNRepositoryPool @ 0x561b36858 , loaded >>>>> by "hudson.ClassicPluginStrategy$AntClassLoader2 @ 0x56057c428". The >>>>> memory >>>>> is accumulated in one instance of >>>>> "java.util.concurrent.RunnableScheduledFuture[]" loaded by "<system class >>>>> loader>". >>>>> >>>>> Does it match JENKINS-34213? >>>>> >>>>> On Tuesday, May 17, 2016 at 9:29:16 AM UTC-4, Stephen Connolly wrote: >>>>>> >>>>>> Jenkins 2.5 has the fix IIUC >>>>>> >>>>>> On 17 May 2016 at 14:28, Stephen Connolly <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> It is fixed, but in the latest version of remoting... which is not >>>>>>> something you can upgrade without either building a custom build of >>>>>>> Jenkins >>>>>>> or upgrading Jenkins. >>>>>>> >>>>>>> You could crack open your jenkins.war and replace the remoting.jar >>>>>>> with the fixed version and seal it back up again and see if that fixes >>>>>>> your >>>>>>> issue... probably the quickest way to confirm if my theory as to the >>>>>>> source >>>>>>> of your leak is correct, but you would need to know what you are doing >>>>>>> >>>>>>> On 17 May 2016 at 14:14, Ugo Bellavance <[email protected]> wrote: >>>>>>> >>>>>>>> Thanks, but I can see that this issue is fixed but I don't know how >>>>>>>> to update my Jenkins install so that I have the fix. I'm using 1.656. >>>>>>>> >>>>>>>> Ugo >>>>>>>> >>>>>>>> On Tuesday, May 17, 2016 at 8:56:34 AM UTC-4, Stephen Connolly >>>>>>>> wrote: >>>>>>>>> >>>>>>>>> I will repeat: >>>>>>>>> >>>>>>>>> > I am suspecting JENKINS-34213 may be your issue. >>>>>>>>> >>>>>>>>> On 17 May 2016 at 13:12, Ugo Bellavance <[email protected]> wrote: >>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Friday, May 13, 2016 at 8:50:56 AM UTC-4, Ugo Bellavance wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Thursday, May 12, 2016 at 2:01:41 PM UTC-4, Raymond Accary >>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>> Hi, >>>>>>>>>>>> If it helps, you might avoid the crash by installing the >>>>>>>>>>>> monitoring plugin, and triggering garbage collection once the >>>>>>>>>>>> memory is >>>>>>>>>>>> approaching the maximum allocated heap size. This is a workaround >>>>>>>>>>>> until >>>>>>>>>>>> someone is able to diagnose the root cause. I have an open issue : >>>>>>>>>>>> https://issues.jenkins-ci.org/browse/JENKINS-34573 but thought >>>>>>>>>>>> I'd run the suggestion. >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>> Have anyone monitored the JVM with VisualVM? I have found that >>>>>>>>>>> it looks like a specific memory pool may be filling up: Old Gen. >>>>>>>>>>> I'm >>>>>>>>>>> trying now with: >>>>>>>>>>> >>>>>>>>>>> -Djava.awt.headless=true -Dcom.sun.management.jmxremote >>>>>>>>>>> -Xms10752m -Xmx10752m -XX:NewRatio=10 >>>>>>>>>>> >>>>>>>>>>> We'll see in a few days. In the last few days, the problem have >>>>>>>>>>> occured each morning at around 4:45. I'm not sure if the same jobs >>>>>>>>>>> run on >>>>>>>>>>> the week-ends as well. >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> From what I can see, it will always fill the memory. Probably a >>>>>>>>>> leak, but I don't know how to be 100% sure about it. When I try to >>>>>>>>>> load a >>>>>>>>>> heap dump in VisualVM, but it shows nothing except "Not supported >>>>>>>>>> for this >>>>>>>>>> JVM. >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>> Google Groups "Jenkins Users" group. >>>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>>> send an email to [email protected]. >>>>>>>>>> To view this discussion on the web visit >>>>>>>>>> https://groups.google.com/d/msgid/jenkinsci-users/1b43cd1d-531b-4638-8e53-15d5f5670f91%40googlegroups.com >>>>>>>>>> >>>>>>>>>> <https://groups.google.com/d/msgid/jenkinsci-users/1b43cd1d-531b-4638-8e53-15d5f5670f91%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>>>> . >>>>>>>>>> >>>>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "Jenkins Users" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to [email protected]. >>>>>>>> To view this discussion on the web visit >>>>>>>> https://groups.google.com/d/msgid/jenkinsci-users/145bf27e-4bcc-4444-b46c-9d8fe4efc1a2%40googlegroups.com >>>>>>>> >>>>>>>> <https://groups.google.com/d/msgid/jenkinsci-users/145bf27e-4bcc-4444-b46c-9d8fe4efc1a2%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>> . >>>>>>>> >>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>> >>>>>>> >>>>>>> >>>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Jenkins Users" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/jenkinsci-users/a7867041-c143-4d08-a7ee-22e4b78a4418%40googlegroups.com >>>>> >>>>> <https://groups.google.com/d/msgid/jenkinsci-users/a7867041-c143-4d08-a7ee-22e4b78a4418%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> -- >>>> You received this message because you are subscribed to a topic in the >>>> Google Groups "Jenkins Users" group. >>>> To unsubscribe from this topic, visit >>>> https://groups.google.com/d/topic/jenkinsci-users/qLwBFyQ84Z4/unsubscribe >>>> . >>>> To unsubscribe from this group and all its topics, send an email to >>>> [email protected]. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/jenkinsci-users/CA%2BnPnMyz55b4ppFhQ3STgH5fK7cwmanns8VR_yFmmDLZ5BVKeA%40mail.gmail.com >>>> >>>> <https://groups.google.com/d/msgid/jenkinsci-users/CA%2BnPnMyz55b4ppFhQ3STgH5fK7cwmanns8VR_yFmmDLZ5BVKeA%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> >>> >>> -- >>> Ugo Bellavance ([email protected]) >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Jenkins Users" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/jenkinsci-users/CACSSk%2BUNBm15yaMbhm85tcd%3D50aN2E8eKbf0BMvv0TEdoWS3Og%40mail.gmail.com >>> >>> <https://groups.google.com/d/msgid/jenkinsci-users/CACSSk%2BUNBm15yaMbhm85tcd%3D50aN2E8eKbf0BMvv0TEdoWS3Og%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/3c5e05e1-196a-4def-8447-10d1135a5c04%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
