I’m guessing it was the second option as we don’t do automated builds of code, but use Jenkins like a CRON service for our environment (scripts that clean up logs, perform maintenance on the database, back office work, etc…). The permissions we took away was basically yanking the Authorized_keys file from the remote systems so that Jenkins couldn’t login to the system (send commands via ssh). I’m guessing the clean shutdown cleared out all the bad cruft from the metadata and that’s why Jenkins came up properly the second time. After that second boot up Jenkins behaved normally (other than not being able to successfully send the commands over ssh to the remote systems).
From: jenkinsci-users@googlegroups.com [mailto:jenkinsci-users@googlegroups.com] On Behalf Of Jennifer Hofmeister Sent: Friday, November 06, 2015 5:55 AM To: jenkinsci-users@googlegroups.com Subject: RE: Jenkins freaked out Hi, I can imagine two scenarios, that are Jenkins-unspecific: - You had a number of CMS commits after the outage (if this is possible on your systems), and once Jenkins was back up again, it polled all of them at once and thus ended up with a bulk of new builds, including several different changes in the same job - Your power outage caused some metadata loss (maybe Jenkins was writing at the time the outage occurred), and Jenkins “forgot” its build history, thus trying to fire up every one of those jobs to what it thought was an initial first build. I had a scenario similar to the last one when I wanted to update Jenkins to 1.6xx. The newer version failed to import the old build history into the new format, and I experienced the effect you described. So it could be some hiccup caused by corrupted data. Did you experience anything else after you restarted Jenkins? Do you think it has to do with the permissions (which permissions?) you took away? Jennifer From: jenkinsci-users@googlegroups.com<mailto:jenkinsci-users@googlegroups.com> [mailto:jenkinsci-users@googlegroups.com] On Behalf Of Kevin Brown Sent: Donnerstag, 5. November 2015 17:34 To: jenkinsci-users@googlegroups.com<mailto:jenkinsci-users@googlegroups.com> Subject: Jenkins freaked out I’m not exactly sure what happened with our Jenkins, but we had a power outage that took the machines down without any chance of graceful shutdown. When we got the systems back up, we started Jenkins up and instead of starting up and just bringing up the few jobs that were scheduled for around the time the machine started up, we had all the Build Executors full (30) and over 120 jobs in the Build Queue, quite a few of which were the same job, so it was firing off jobs as fast as the Build Executors could come free from another job. We were able to then shutdown Jenkins via its interface and after taking away the permissions on the remote systems it runs jobs on, we brought it back up and the Build Queue was empty and the Build Executors were quiet. Any thoughts on why this might have happened? Jenkins v 1.596 -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com<mailto:jenkinsci-users+unsubscr...@googlegroups.com>. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/FCBFCDE72C66B34C8F7AD8D62D262D4E36432244%40ilsaw05.corp.incentivelogic.com<https://groups.google.com/d/msgid/jenkinsci-users/FCBFCDE72C66B34C8F7AD8D62D262D4E36432244%40ilsaw05.corp.incentivelogic.com?utm_medium=email&utm_source=footer>. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com<mailto:jenkinsci-users+unsubscr...@googlegroups.com>. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAA587D532C45541B62417404F1D22030E83D378%40you-exch1.younicos.local<https://groups.google.com/d/msgid/jenkinsci-users/CAA587D532C45541B62417404F1D22030E83D378%40you-exch1.younicos.local?utm_medium=email&utm_source=footer>. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/FCBFCDE72C66B34C8F7AD8D62D262D4E3643345E%40ilsaw05.corp.incentivelogic.com. For more options, visit https://groups.google.com/d/optout.
