Just saw the post on gentoo-dev. It'll be interesting to see the postmortem as not being able to update the infra status page during an outage is a bit of a problem ;)
Thanks for the reply Hayley On Mon, 2021-06-28 at 23:37 -0500, Dale wrote: > Hayley wrote: > > Hey all, > > Since yesterday I've getting 502 Bad Gateway when trying to access > > gitweb.gentoo.org and was wondering if anyone else is having the > > same > > issue? > > The gentoo infra status page says that it should be up so I'm not > > sure > > what could be causing the problem but it's a 502 so I'm guessing it > > must be a server side issue? > > > > Thanks, > > Hayley > > > > > > > > > I just read a message on -project I think that was about some upgrade > problems and some parts of infra not working correctly. That could > be > the problem you were seeing. I think fixing the problem is still in > progress so may be a few days before everything is back up to 100%. > This is the message: > > > Alec Warner wrote: > > > Hi, > > > > > > Many of you reported that various infra-owned services were not > > > functioning properly these past two days. We had some package > > > upgrades roll out that were bad, and a subset of services were > > > non-functional while we worked through the incident. > > > > > > (a) The upgrades did not appear to cause any data loss, but > > > service > > > availability was reduced (500s / 502s for some HTTP services.) > > > (b) Some hosts may have had problems running some commands. E.g. > > > I > > > know that 'grep' didn't work for a while. If you observed errors > > > related to missing 'GLIBC_2.33' symbols then this was likely > > > related. > > > (c) Many replicated services (including infra-status) rely on > > > gitweb > > > to take updates from git; and gitweb was not up, so they were > > > unable > > > to receive updates. > > > (d) More impacted notes will follow in a postmortem that will > > > come > > > later this week for this incident. > > > > > > Thanks to all of you who reported problems and apologies for the > > > service disruptions. > > > > > > -A > > > Hope that helps. Should keep you from looking for the problem on > your > end at least. > > Dale > > :-) :-) >
