Hello, I am the maintainer of the GCC ports and thus I reply about this particular topic.
Disabling LTO_BOOTSTRAP option by default has already been done for the devel ports on the i386, amd64 and aarch64 architectures (so for all tier 1 platforms): https://cgit.freebsd.org/ports/commit/?id=daf12c76bd531335f58e7602afd3bc8da8fc5c33 The same is about to be done for the production ports too as part of a larger update: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=273397 I have stated multiple times that enabling that option by default was not my choice, but since it was decided to do so I defended the choice as much as I could. I have finally found a good reason to disable it when even the official package builders were not able to build GCC any more. If there is a need to get LTO_BOOTSTRAP disabled by default quickly, I can commit that particular patch sooner than I initially planned. However, please note that LTO_BOOTSTRAP is an option: if the build times are too long for you, you can disable the option. The option has been enabled by default to distribute optimized packages, not because it is recommend to build the port with that option enabled. Actually, I even wrote a warning about it in pkg-help. The STANDARD_BOOTSTRAP option is being enabled by default instead of LTO_BOOTSTRAP: that bootstrap is much shorter in time and should not be a problem for any recent machine. I think that for most people disabling any bootstrap is perfectly fine, still enabling it by default should produce more stable official packages (I have not checked: GCC ports are very complex to maintain and I have lots of higher priorities issues to solve). Again, users are free to disable the option if they want faster builds and do not need any bootstrap. Cheers, Lorenzo Salvadore Sent from Proton Mail mobile -------- Messaggio originale -------- Il 24 Ott 2023, 21:12, Robert Clausecker ha scritto: > The build times have gone up to the point where they are unsustainable. > Frequent updates to key ports (like llvm*, rust, gcc*) make it so that > basically every time I prepare a new batch of commits, I have to rebuild a > variety of toolchain ports across 8 jails (amd64/i386/arm64/armv7 each for > FreeBSD 12.4 and 13.2). This takes multiple days. And I'm working with > hardware that's quite recent (for x86, an 8 thread Skylake box, for arm, an 8 > thread Windows 2023 dev kit). By the time the builds are done, some random > update has usually caused the ports to be out of date again, so if I were to > rebase, I would have to do all of this again. And again. And again. > Particularly bad offenders are gcc and rust. Ccache is ineffective for these > as gcc has LTO turned on, which seems to more than triple the regular build > time to more than 24 hours even on a fast Skylake box. This is single > threaded as I build multiple ports at once; if I were to build > multi-threaded, the same amount of total CPU hours would have been spent, so > that would not fix my problem. Ccache is also ineffective for rust of course. > There's another issue in that ccache doesn't scale to large cache sizes (my > experiments show that anything larger than 20 GB seems to cause problems as > ccache repeatedly tries to scan the whole thing for evictions), and the sizes > that work are just not enough to be effective. What would help is being able > to have one cache for each combination of ports tree and jail, but Poudriere > has no support for that. Another bad offender is texlive. For some reason, > texlive-texmf needs to be rebuilt frequently, despite mostly comprising data > that is just unpacked and repacked. This takes forever and pegs the disk at > 100% for more than an hour as the texlive source tarball is repeatedly > extracted and then compressed into packages. I don't get why the texlive > stuff is not split in such a way that the stuff that is just repacked lives > in its own port with no dependencies so it only needs to be rebuilt on rare > texlive updates. And it seems I'm slowly killing my build SSD like that. > After just about 9 months, it is already at 100 TB of writes just from port > builds. Building with workdirs in memory is no longer an option as that > frequently kills my build server by filling all its RAM with build files > until no processes can be started anymore. Poudriere does not have an > effective mechanism to prevent this (tmpfs limits don't work as the ports in > question require very large workdirs, tend to take very long to build and > tend to be built all at the same time for multiple jails). Using prebuilt > packages is not an option as they lag behind by several days/weeks and lead > to an inconsistent testing environment. It is also not a good solution to > chose non-default build options for these ports as it is not clear if that > would affect the validity of the testbuilds. How can we fix this problem and > make ports development sustainable again? Some ideas: - disable LTO and other > options by default that increase build times by such a ridiculous degree. > This would really make a huge impact with very little work. I don't think LTO > on toolchain ports improves build times enough in comparison to the extra > time it takes to build these. - for gcc, switch to single or no bootstrap by > default. We have known good toolchains we use to build gcc. There's really no > reason to build it multiple times just out of paranoia. The maintainer is > supposed to check that gcc is built correctly without bootstrapping so > consumers don't need to build it multiple times. - untangle some of the > dependencies so that less ports may trigger rebuilds of critical ports. For > example, llvm docs could be moved to separate ports so that updates in the > documentation toolchain do not trigger an LLVM rebuild. - reduce USES to > chose lighter dependencies by default. E.g. USES=llvm could depend on the > light flavour by default. I'm sure only very few ports need all of LLVM and > the light flavour is faster to build. - rework Poudriere's rebuild detection > to not rebuild every port for every random bullshit thing. For example, I > don't see why ports need to be rebuilt for transitive changes in build > dependencies. E.g. if port A has build depends on port B which build depends > on port C, and C is updated, then A has to be rebuilt despite its direct > dependencies being unchanged. This does not appear to be reasonable. - > unbundle libraries more thoroughly. We currently have dozens of copies of > LLVM, skia, webkit, and others in tree as ports just bundle them instead of > even making an attempt at unbundling. This means that every time they need to > be patched, it's a whackamole at finding all copies. Plus build times suffer > a lot. I know it's hard, but perhaps something can be done. For example, I > have given up on trying to make electron work on armv7 as with every major > version update, my patches are randomly being dropped and I have to do it all > again. Like all chromium ports, electron takes over two days to build on my > arm box and my time is insufficient for that. - stop bulk bumping RUN_DEPENDS > consumers when dependencies are updated, or at least think carefully before > doing so. RUN_DEPENDS are only installed after the build and should not > affect the build. For example, sysutils/cdrtools uses the command line opus > encoder and thus depends on audio/opus. There is absolutely no reason to bump > it when audio/opus is updated. It just causes everybody to needlessly rebuild > and reinstall ports. Sure there's the odd case where that needs to be done, > but it seems like some maintainers just always do that, even when it's not > needed. - maybe add a system where ports can declare the oldest version of > themselves they are compatible to, in the sense that consumers only need to > be rebuilt if they were built against a version older than that. For example, > if a shared library is updated with a bug fix that does not change the ABI, > there's no need to rebuild all consumers. With great frustration, Robert > Clausecker -- () ascii ribbon campaign - for an 8-bit clean world /\ - > against html email - against proprietary attachments
