ok, we're back up w/all new jenkins workers. i'll be keeping an eye on these pretty closely today for any build failures caused by the new systems, and if things look bleak, i'll switch back to the original five.
thanks for your patience! On Fri, Dec 12, 2014 at 8:47 AM, shane knapp <skn...@berkeley.edu> wrote: > downtime is extended to 10am PST so that i can finish testing the numpy > upgrade... besides that, everything looks good and the system updates and > reboots went off w/o a hitch. > > shane > > On Fri, Dec 12, 2014 at 7:26 AM, shane knapp <skn...@berkeley.edu> wrote: > >> reminder: jenkins is going down NOW. >> >> On Thu, Dec 11, 2014 at 3:08 PM, shane knapp <skn...@berkeley.edu> wrote: >> >>> here's the plan... reboots, of course, come last. :) >>> >>> pause build queue at 7am, kill off (and eventually retrigger) any >>> stragglers at 8am. then begin maintenance: >>> >>> all systems: >>> * yum update all servers (amp-jekins-master, amp-jenkins-slave-{01..05}, >>> amp-jenkins-worker-{01..08}) >>> * reboots >>> >>> jenkins slaves: >>> * install python2.7 (along side 2.6, which would remain the default) >>> * install numpy 1.9.1 (currently on 1.4, breaking some spark branch >>> builds) >>> * add new slaves to the master, remove old ones (keep them around just >>> in case) >>> >>> there will be no jenkins system or plugin upgrades at this time. things >>> there seems to be working just fine! >>> >>> i'm expecting to be up and building by 9am at the latest. i'll update >>> this thread w/any new time estimates. >>> >>> word. >>> >>> shane, your rained-in devops guy :) >>> >>> On Wed, Dec 10, 2014 at 11:28 AM, shane knapp <skn...@berkeley.edu> >>> wrote: >>> >>>> reminder -- this is happening friday morning @ 730am! >>>> >>>> On Mon, Dec 1, 2014 at 5:10 PM, shane knapp <skn...@berkeley.edu> >>>> wrote: >>>> >>>>> i'll send out a reminder next week, but i wanted to give a heads up: >>>>> i'll be bringing down the entire jenkins infrastructure for reboots and >>>>> system updates. >>>>> >>>>> please let me know if there are any conflicts with this, thanks! >>>>> >>>>> shane >>>>> >>>> >>>> >>> >> >