On May 19, 2021, at 07:56, Bjarne D Mathiesen wrote: > Ryan Schmidt wrote: >> I personally am not comfortable adding other build machines to our buildbot >> system that I do not control. When I control the machines, I know what is >> installed on them and that they are set up correctly. Having build machines >> located outside of my local network also poses additional challenges, as >> I've learned by having our Apple Silicon build machine outside of my >> network, challenges which I would prefer to minimize, not increase. >> >> We currently use one build machine per OS version / arch, and have the >> hardware needed to do that. Adding more hardware such that we have more than >> one build machine per OS version / arch is not something our buildbot system >> was ever designed to accommodate, and would introduce problems. > > So we are more-or-less single-point-of-failure regarding the buildbots > !?
The buildbot system has always been a single point of failure, as has almost every other aspect of MacPorts infrastructure, with the exception of our mirrors. We have one mailing list server. One main repository. One main web server. Buildbot is not critical infrastructure, in that it is nice to have but the world does not end if it is down for a short period of time. People can still get any packages that were built before from our distributed network of mirror servers, and if a binary is not available for whatever the user wants to install, then it can be built from source on their system. > Personally, I'ld deem it advisable to have at least 3 complete > systems geographically dispersed. That seems completely infeasible to me. As far as the buildmaster goes, Buildbot is designed to have just one. There is a more complicated optional multiple-master mode available in the latest Buildbot, but AFAIK it's intended for each master to handle separate tasks. For example, one master handles the web interface while a second master handles scheduling. I intend to use this mode in the new buildbot setup. As for the workers, our system is designed with the assumption that there is one worker for each OS version / arch. We already have this with our existing hardware. Adding more hardware running additional copies of the workers we already have will introduce complications. If we want to do that, we would have to modify mpbb and the buildbot configuration to accommodate those complications. We would have to make the system much more robust against failures than it currently is. Right now, it works because one person (me) is able to intervene to fix problems that arise. There are also security considerations in allowing additional individuals to run the build machines that produce our officially signed binaries. >> >> Using Linux and commodity hardware is not applicable because it the macOS >> EULA only permits running macOS on Apple hardware, as we currently do. > > So it's not the proposed software stack that's the problem, but the > hardware stack. Then how about an old 2010/12 MacPro with all the bells > and whistles ? I've been able to boot a 2010 MacPro w/ 256GB RAM under > Ubuntu without any problems. We already have four 2009 Xserves. Their CPU and memory could be upgraded if needed. One of these is offline at the moment due to hardware failure. It can be repaired. If necessary, I'll repair it.
