Re: Simon McVittie > I'm calling for votes on the following resolution as formal advice from > the Technical Committee (Constitution ยง6.1.5). There are no non-accepted > amendments, so the options to vote on are "yes" or "further discussion".
I vote yes > further discussion. > ~~~~ begin text to be voted on ~~~~ > > Summary > ======= > > There are currently Debian 11 installations with both the merged-/usr and > non-merged-/usr filesystem layouts. All of these installations should > successfully upgrade via normal upgrade paths to a merged-/usr Debian 12. > Only after the release of Debian 12 can packages assume that all > installations will be merged-/usr. > > Main points: > > - We have recommended merged-/usr for Debian 12. > - Moving individual files is not merged-/usr. > - "Symlink farms" are not merged-/usr. > - Upgrading a non-merged-/usr system to Debian 12 needs to work. > - Upgrading a merged-/usr system to Debian 12 needs to work. > - Packages cannot assume all systems are merged-/usr until the Debian 13 > development cycle begins. > - Upgrading via apt in the usual way should work. > - Testing and QA systems should be able to avoid this transition, but if > they do, they cannot be upgraded beyond Debian 12. > - A package building incorrectly on a non-merged-/usr system is a bug. > - A package building incorrectly on a merged-/usr system is a bug. > - Please stop moving individual packages' files from the root filesystem > into /usr, at least for now. > > Definitions and current status > ============================== > > libQUAL refers to the directories described in FHS 3.0 section 3.10 [1], > such as lib64 on the amd64 architecture. > > Merged /usr is the filesystem layout in which /bin, /sbin, /lib and each > /libQUAL that exists are symbolic links to the corresponding directories > below /usr. This results in aliasing between /bin and /usr/bin, and > so on. > > In the merged-/usr layout, files whose canonical logical location is > in one of the affected directories on the root filesystem, such as > /bin/bash, /sbin/fsck and /lib/ld-linux.so.2, are physically located at > the corresponding path below /usr, such as /usr/bin/bash. Each file in > one of the affected directories is accessible via two paths: its canonical > logical location (such as /bin/bash or /usr/bin/env), and the other path > implied by the aliasing (such as /usr/bin/bash or /bin/env). > > There are two supported categories of Debian 11 installation, which are > currently considered equally-supported: > > - Merged-/usr installations. These were installed with debian-installer > from Debian 10 or later, or installed with debootstrap --merged-usr, > or converted from the non-merged-/usr layout by installing the usrmerge > package, or installed or converted by any similar procedure. They have > the merged-/usr layout. > > - Non-merged-/usr installations. These were installed with debian-installer > from Debian 9 or earlier and subsequently upgraded without converting > to merged-/usr, or installed with debootstrap --no-merged-usr, or > converted from the merged-/usr layout with dpkg's "dpkg-fsys-usrunmess" > utility or any similar procedure. They have the traditional, > non-merged-/usr layout in which /bin/bash and /usr/bin/env have exactly > those physical paths, and /usr/bin/bash and /bin/env do not exist. > > Merged-/usr is not the only filesystem layout that has been proposed for > unifying the root filesystem with /usr. For avoidance of doubt, we do not > consider other filesystem layouts to be implementations of merged-/usr. > In particular, we do not consider these to be implementations of merged-/usr: > > - all affected files physically located in /bin, /sbin, /lib and /libQUAL, > with /usr/bin as a symlink to /bin, etc. (this is the reverse of > merged-/usr, and was historically used in the hurd-i386 port) > > - a "symlink farm" in each of /bin, /sbin, /lib, /libQUAL with individual > symbolic links such as /bin/bash -> /usr/bin/bash for only those files that > historically had their canonical logical location on the root filesystem > > - a "symlink farm" in each of /bin, /sbin, /lib, /libQUAL with individual > symbolic links such as /bin/env -> /usr/bin/env for all files in the > affected directories, regardless of whether they historically had > their canonical logical location on the root filesystem > > [1]: > https://www.debian.org/doc/packaging-manuals/fhs/fhs-3.0.html#libltqualgtAlternateFormatEssential > > Upgrade path from Debian 11 to Debian 12 > ======================================== > > The technical committee resolved in #978636 [2] that Debian 12 'bookworm' > should only support the merged-/usr layout. This resolution describes > the implications of that decision in more detail. > > Debian installations have traditionally had a straightforward upgrade > path between consecutive releases, and the technical committee expects > maintainers to continue this. In the case of #978636, the upgrades we > are interested in are: > > - Upgrading from Debian 11 (stable release) to Debian 12 (stable release) > > - Upgrading from Debian 11 (stable release) to testing/unstable during the > development cycle that will lead to Debian 12, and subsequently upgrading > from testing/unstable to Debian 12 (stable release) > > What we understand this to imply is as follows: > > - Because Debian 11 installations with the merged-/usr layout already > exist, and because Debian 12 should only support the merged-/usr layout, > all packages in Debian 12 should be installable onto a merged-/usr system > along with their dependencies, and work correctly on the resulting system. > > - Because Debian 11 installations with the non-merged-/usr layout already > exist, all packages in Debian 12 should be installable onto a > non-merged-/usr > system along with their dependencies, and work correctly on the resulting > system. > > + The key reason for this is that apt is not required to perform the > upgrade between stable releases in any particular order, so long > as package dependency relationships are respected; therefore the > upgrade can happen in whatever order apt chooses, which can vary > between machines. Debian has not traditionally required use of a > special upgrade tool similar to Ubuntu's do-release-upgrade(8) and > we believe the upgrade to Debian 12 should be no different (see > below for more details on this topic). > > + Another reason for this is that during the development of Debian 12, > testing/unstable systems undergo a series of partial upgrades, which > similarly will happen in an undefined order. > > + We do not require that the resulting system remains non-merged-/usr: > if the packages involved in this installation transaction are part of > the implementation of a transition to merged-/usr, then installing them > might result in the system becoming merged-/usr. > > - The same expectations apply to packages uploaded to testing/unstable > during the development cycle that will lead to Debian 12. > > - Debian contributors who are interested in merged-/usr are invited to > implement a straightforward migration path from non-merged-/usr to > merged-/usr. The Technical Committee will not design this migration path > in detail. If disputes arise related to this migration path, or if > advice on this migration path is requested, we will resolve those by > following our usual procedures. > > + One example of a migration path that might be used is for an Essential > package to add a dependency on the usrmerge package, so that it will > be installed automatically during upgrades. We do not require this to > be the migration path that is chosen; it is presented here merely to > demonstrate that such a migration path can exist. > > - After Debian 12 is released, the transition to merged-/usr is considered > to be complete. Because we do not support "skipping a release" when > upgrading systems, new packages uploaded to testing/unstable during the > development cycle that will lead to Debian 13 may assume and require > that the system onto which they are installed is merged-/usr. > > If a suitable transition mechanism is not available by the time the > Debian 12 freeze is reached, the Technical Committee will rescind our > advice in #978636 and modify our advice in this resolution to reflect > the situation that has been reached. We hope that this will not be necessary. > > [2] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=978636 > > Use of a special upgrade path > ============================= > > Some developers have argued that we should deploy merged /usr by means of > a special upgrade path, with all other upgrade paths being declared to be > unsupported. Some examples of special upgrade paths that might be proposed: > > - upgrading dpkg before carrying out the rest of the upgrade > > - installing the usrmerge package before carrying out the upgrade > > - using a special tool similar to Ubuntu's do-release-upgrade(8) which > encapsulates whatever steps are necessary > > This would imply that upgrade paths other than the recommended one are not > expected to work. > > We are aware that many Debian users do not read the release notes before > upgrading from oldstable to stable, and will expect to be able to upgrade > via whatever apt commands they are used to using, regardless of whether > the release notes recommend something different. Given that there are > alternatives available, we do not think that merged-/usr is sufficiently > good cause to interfere with this. > > During the Debian 12 development cycle, users of testing/unstable will also > need an upgrade path from Debian 11 to testing/unstable, or from older to > newer snapshots of testing/unstable. In general this will be done using > apt, and it needs to continue to work if at all possible, even before a > special upgrade tool has been prepared; introducing a "flag day" at which > all testing/unstable users are expected to carry out additional > non-automatic upgrade steps would be disruptive, and in practice many > testing/unstable users are likely to skip those steps. > > For these reasons, we make the simple, conservative recommendation that a > special upgrade path should not be required, and upgrading via apt in > approximately the same way as previous Debian releases should continue to work > in general. > > Testing and QA > ============== > > We anticipate that during the development cycle that will lead to Debian 12, > it is likely to be useful for testing and QA infrastructure (such as > autopkgtest, piuparts and/or reproducible-builds) to be able to produce > an installation of Debian testing/unstable that is not merged-/usr, > in order to be able to verify that packages targeted for inclusion in > Debian 12 can still be installed and/or built successfully in a > non-merged-/usr environment during partial upgrades. > > As a result, we recommend that if there is an automatic migration from > non-merged-/usr to merged-/usr, it should be possible to prevent that > migration. However, systems where that migration has been prevented are > not required to be fully-supported, and in particular, upgrading them > to Debian 13 or to the state of testing/unstable during development of > Debian 13 should be considered unsupported. > > Building packages > ================= > > In #914897 [3] the Technical Committee resolved that for Debian 11, packages > can validly be built in either a merged-/usr or non-merged-/usr environment. > We understand this to imply that packages built in either a merged-/usr or > non-merged-/usr environment should work as intended in either a merged-/usr > or non-merged-/usr environment. > > There is a class of bugs in which a package embeds the absolute path to > some executable or other file, in such a way that if it is built in an > environment with a unified /usr (either via merged-/usr or symlink farms), > this can result in embedding a non-canonical path such as /usr/bin/sh or > /bin/env which will not work correctly in a non-merged-/usr environment. > The Technical Committee reminds maintainers that packages in Debian > 12 are expected to work correctly in a non-merged-/usr environment, > and therefore this class of bugs needs to be resolved. As a result, we > recommend that these bugs should generally be treated as release-critical > for Debian 12 by maintainers and the release team. > > The Reproducible Builds effort tracks this class of bugs as > "paths_vary_due_to_usrmerge" [4] and many bugs in this class are > tracked with the usertag "usrmerge" [5]. In Autotools-based > build systems, they can often be avoided by passing arguments such as > `SED=/bin/sed` to the configure script. > > Note that although the name of the usertag mentions usrmerge, this class > of bugs is not specific to the usrmerge package, and not completely specific > to merged-/usr; many of these bugs would also manifest if unified /usr was > achieved via symlink farms, or if a local executable such as > /usr/local/bin/sed existed on the build system. > > #993675 [6] is a typical example of this class of bugs. > > [3] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=914897 > [4]: > https://tests.reproducible-builds.org/debian/issues/unstable/paths_vary_due_to_usrmerge_issue.html > [5]: > https://udd.debian.org/cgi-bin/bts-usertags.cgi?user=reproducible-builds%40lists.alioth.debian.org&tag=usrmerge > [6]: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=993675 > > Moratorium on moving files' logical locations into /usr > ======================================================= > > In the past, some package maintainers have taken advantage of the > fact that /usr is now required to be mounted during early boot > (since Debian 9) to move individual files from the root filesystem > to /usr. Examples of this being done during previous development cycles > include the libglib-2.0.so.0 and libgcrypt.so.20 shared libraries, > and the command-line utilities in the iptables package. > > The Technical Committee recommends that during the Debian 12 development > cycle, the maintainers of individual packages should not proactively move > files from the root filesystem to the corresponding locations in /usr in > the data.tar.* of packages. Files that were in /usr in the Debian 11 > release should remain in /usr, while files that were in /bin, /lib* or > /sbin in the Debian 11 release should remain in those directories. > If files were moved from /bin, /lib* or /sbin into /usr since the > Debian 11 release, they should be moved back to their Debian 11 > locations. > > Similarly, during the Debian 12 development cycle, we recommend that > maintainers of tools such as debhelper should not move files from the > root filesystem into /usr. > > For example, debhelper 13.4 started moving systemd units from /lib/systemd > into /usr/lib/systemd, but this was subsequently reverted in 13.5.2. We > consider the revert that was done in 13.5.2 to have been an appropriate > course of action. > > We issue this recommendation for several reasons: > > - The transition to merged-/usr will make the logical locations of these > files irrelevant to their physical locations. > > - The transitional mechanisms necessary to prevent such moves from breaking > other packages that hard-code specific paths are error-prone, and can > themselves interfere with the transition to merged-/usr. > > - After the transition to merged-/usr has completed, during the Debian 13 > development cycle, we expect that maintainers will be able to move these > files without needing transitional mechanisms. > > - On merged-/usr systems, there is a possible failure mode involving files > being moved between packages (with Replaces) during the same release > cycle that their logical location is changed from the root filesystem > to the corresponding aliased directory in /usr, which can result in > the affected file disappearing. This can be avoided by not changing > the file's logical location until the beginning of the Debian 13 > development cycle, after the transition to merged-/usr is complete. > > - On non-merged-/usr systems, a failure mode has been observed in which > older shared libraries in /lib/MULTIARCH are not always deleted as > intended, and interfere with correct loading of the newer shared > library in /usr/lib/MULTIARCH. This can be avoided by not changing > the file's logical location until the beginning of the Debian 13 > development cycle, after the transition to merged-/usr is complete, > at which point ldconfig(8) will choose the newer library even if > this occurs. > > This recommendation applies until Debian 12 is released, or until > a subsequent Technical Committee resolution rescinds it, whichever > is sooner. We intend to rescind this recommendation if mechanisms are > developed to avoid the undesired side-effects of moving files from the > root filesystem into /usr. > > ~~~~ end text to be voted on ~~~~ Christoph
signature.asc
Description: PGP signature
