On Jan 19, 2021, at 14:22, Christopher Nielsen wrote: > This isn’t urgent - I was simply interested in checking the build backlog - > but can’t get a response from build.macports.org. > > Earlier I was seeing a “Bad Gateway” message via web browser, now it’s simply > timing out with a blank page.
The buildmaster's sqlite database has become corrupted. I have stopped the buildmaster process so you will now see the "Bad Gateway" message if you visit the page. I will investigate if we can salvage the database. We had this problem before in February 2020 and in May 2020. https://lists.macports.org/pipermail/macports-dev/2020-May/041882.html At those times, we deleted the database and allowed the buildmaster to create a new one. This does not affect the history of completed builds, which are not stored in the database, but we would lose the list of scheduled builds and would have to reschedule them. At this time there was a huge backlog of scheduled builds due to the recent jpeg-to-libjpeg-turbo switchover plus an additional backlog before that on the 10.14 and 11 x86_64 builders, so it will be a bit of work to reschedule everything. I also noticed the mprsyncup job regenerating all portindexes, which is unusual. Some of the PortIndex.quick and PortIndex.json files that got generated have 0 size, which is wrong. We had this problem before in November 2020. https://trac.macports.org/ticket/61507 Previously when we've had these problems I thought a high server uptime contributed to the problem. But the uptime now is only 13 days. Still, I'll begin by restarting the server and running disk first aid.
