We do have one NFS, for copying build artifacts to RPM repository. tirsdag 6. august 2019 09.08.48 UTC+2 skrev Sverre Moe følgende: > > I was mistaken. We did not use NFS. > The disk for JENKINS_HOME (Jenkins running on VM), is a LVM disk. > > mandag 29. juli 2019 18.15.20 UTC+2 skrev Ivan Fernandez Calvo følgende: >> >> check the Cloudbees links, I've helped to write those KB when I was on >> CloudBees :), I'm pretty sure that the NFS is your pain and the root cause >> of all your problems if you can rid of it better. >> >> El lunes, 29 de julio de 2019, 18:03:59 (UTC+2), slide escribió: >>> >>> CloudBees (not my employer) has some resources on using NFS (generally >>> the recommendation is to NOT use NFS for JENKINS_HOME). >>> >>> >>> https://support.cloudbees.com/hc/en-us/articles/115000486312-CJP-Performance-Best-Practices-for-Linux#nfs >>> >>> and >>> https://support.cloudbees.com/hc/en-us/articles/217479948-NFS-Guide >>> >>> On Mon, Jul 29, 2019 at 8:51 AM Sverre Moe <sver...@gmail.com> wrote: >>> >>>> Yes, we are using NFS for JENKINS_HOME. >>>> >>>> mandag 29. juli 2019 15.41.00 UTC+2 skrev Ivan Fernandez Calvo følgende: >>>>> >>>>> you have 83 threads in state:IN_NATIVE, probably stuck in IO >>>>> operations, those 83 threads are blocking the other 382 threads, if you >>>>> use >>>>> an NFS or similar device for you Jenkins HOME this is probably your >>>>> bottleneck, if not check the IO stats on the OS to see where you have the >>>>> bottleneck. >>>>> >>>>> El lunes, 29 de julio de 2019, 11:20:50 (UTC+2), Sverre Moe escribió: >>>>>> >>>>>> I was unable to determine something from the stack output >>>>>> Here is the result: >>>>>> https://fastthread.io/my-thread-report.jsp?p=c2hhcmVkLzIwMTkvMDcvMjkvLS1qc3RhY2sudHh0LS05LTE2LTI3 >>>>>> >>>>>> torsdag 18. juli 2019 11.28.06 UTC+2 skrev Sverre Moe følgende: >>>>>>> >>>>>>> There is no such reference in my jstack output. >>>>>>> The output says no deadlock detected. >>>>>>> I will try that site for analyzing the jstack. >>>>>>> >>>>>>> Even a normal running Jenkins has many BLOCKED threads. If that is >>>>>>> normal I don't know. >>>>>>> >>>>>>> We have a test Jenkins instance running on Java 11. That one does >>>>>>> not have any BLOCKED threads. >>>>>>> Our production Jenkins is running Java 8u181. >>>>>>> >>>>>>> torsdag 18. juli 2019 11.04.16 UTC+2 skrev Ivan Fernandez Calvo >>>>>>> følgende: >>>>>>>> >>>>>>>> In that dump I can not see which thread is blocking the others, the >>>>>>>> jstack output has a reference on each thread that said what thread is >>>>>>>> the >>>>>>>> blocker on each thread (- locked <0x00000000> a java.lang.Object), >>>>>>>> you can try to analyze those thread dump with >>>>>>>> https://fastthread.io/index.jsp or other online tools to see if >>>>>>>> you see something relevant, it looks like there is a deadlock. >>>>>>>> >>>>>>>> https://dzone.com/articles/how-to-read-a-thread-dump >>>>>>>> >>>>>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Jenkins Users" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to jenkins...@googlegroups.com. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/jenkinsci-users/cc2d0bdb-b15f-4bec-a0a3-0562ea8c7df7%40googlegroups.com >>>> >>>> <https://groups.google.com/d/msgid/jenkinsci-users/cc2d0bdb-b15f-4bec-a0a3-0562ea8c7df7%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> >>> >>> -- >>> Website: http://earl-of-code.com >>> >>
-- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/d6699e75-4a3c-4125-8b5d-4df0fc12fb47%40googlegroups.com.
