> On Aug 30, 2018, at 9:46 AM, Chris Lambertus <c...@apache.org> wrote: > > > >> On Aug 29, 2018, at 2:35 PM, Chris Lambertus <c...@apache.org >> <mailto:c...@apache.org>> wrote: >> >> >> >>> On Aug 29, 2018, at 7:36 AM, Chris Lambertus <c...@apache.org >>> <mailto:c...@apache.org>> wrote: >>> >>> >>> >>>> On Aug 28, 2018, at 8:39 PM, Chris Lambertus <c...@apache.org >>>> <mailto:c...@apache.org>> wrote: >>>> >>>> If you have not been following status.a.o or the infrabot twitter, our >>>> Jenkins master has suffered a critical disk controller failure. We are >>>> working with our datacenter colocation provider to identify and repair the >>>> problem, but we do not currently have an estimated time of repair. Please >>>> keep an eye on status.apache.org <http://status.apache.org/> for further >>>> updates. >>> >>> >>> Jenkins remains down. Our hosting provider eventually identified and >>> replaced a failed backplane. Sadly, the nature of the backplane failure >>> corrupted both OS drives (which were mirrored.) Out of an abundance of >>> caution, we have decided to migrate the jenkins data to a different server, >>> one with a higher degree of fault tolerance and ability for us to monitor >>> the underlying hardware for predictive failure (as well as slightly higher >>> storage capacity.) Unfortunately, this operation will be time consuming, as >>> the jenkins data is over three terabytes. We will provide an additional >>> update in an hour or two as to the status of the copy and the updated ETR. >>> >> >> Apologies for the delay in updating. >> >> We are currently waiting for our colocation provider to provision the new >> server. We expect that to be done in the EU morning (approx 8-10 hours from >> now.) Once the new box is provisioned we will initiate the data copy to the >> new system. We will provide another update once the new box has been >> provisioned. >> > > > The new box has been provisioned and the data synchronization is underway. > Our current estimate is 10-12 hours to complete the copy and restore service. >
Jenkins has been returned to service. Thanks for your patience in bearing with us through this outage. -Chris
signature.asc
Description: Message signed with OpenPGP
