Hi Osamu! §1 Would you like to join/co-found the nascent "Debian btrfs enablement" team?
In the coming years there will be an increasing number of software that will need a "get all valid bootable rootfs candidates for a btrfs volume". Right now we have GRUB, Debian Rescue (TODO), bootloaders for ARM (TODO) and in the future systemd-boot (aspirationally TODO bookworm), in addition to whatever software will be used for "boot environments". I wonder if os-prober should be the site for this "return list of bootable subvolumes" functionality, rather than duplicating the logic in multiple places? Cyril? I'm not sure because os-prober depends on grub-common, which AFAIK isn't used on ARM. Do we need a NEW package for btrfs-related support functions? Alternatively, btrfs-progs seems like a logical place for functions like "get a list of all bootable subvolumes". Adam, what do you think, is btrfs-progs the right place for helper functions that return volume layout, noting when a subvolume is bootable, perhaps as bundled shell scripts? Could "btrfs subvolume list" be the right place for this? And yes, I think this tooling should ideally exist upstream and not be Debian-specific. I suspect this functionality may be duplicated in Snapper. P.S. Hideki Yamane, who maintains Debian's Snapper package, ACKed installing Debian to a subvolume (without the use of set-default) a long time ago, but it's probably time to check in with him again (CCing him). With truly generic subvolume support for grub that checks for valid bootable rootfs candidates and creates a menu entry for each candidate we will have a working prototype of "boot environments" today. As far as I know, only SUSE has this (IIRC via Snapper, probably with extra grub hooks), plus Arch--on an opt-in basis--with grub-btrfs. Osamu Aoki <os...@debian.org> writes: > Package: os-prober > Version: 1.78 > Severity: normal > > Issue: > Currently Debian os-prober support only btrfs root-filesystem on the root of > the btrfs, i.e., ID 5 (FS_TREE). This makes auto generated grub.cfg to miss > Linux install to btrfs for some Ubuntu and Suse since they put root-system > under @ subvolume. Thank you for bringing this issue to my attention! Yes, I agree, we should fix this. > Existing patch in other distro: > Ubuntu ships patched os-prober 1.77 to address its subvolume use (@ as root- > filesystem) with hardcoded path and very rudamental check for /lib directory. > > -------- > diff -pruN 1.77/linux-boot-probes/common/50mounted-tests 1.77ubuntu1/linux- > boot-probes/common/50mounted-tests > --- 1.77/linux-boot-probes/common/50mounted-tests 2018-08-10 > 19:23:18.000000000 +0000 > +++ 1.77ubuntu1/linux-boot-probes/common/50mounted-tests 2020-11-02 > 11:12:51.000000000 +0000 > @@ -54,6 +54,19 @@ if type grub-mount >/dev/null 2>&1 && \ > mounted=1 > type="$(grub-probe -d "$partition" -t fs)" > [ "$type" ] || type=fuseblk > + > + case "$type" in > + btrfs) > + if [ -x "$tmpmnt/@/lib" ] && \ > + ! mount --bind "$tmpmnt/@" "$tmpmnt"; then > + warn "failed to mount btrfs subvolume @ on > $partition" > + if ! umount $tmpmnt; then > + warn "failed to umount $tmpmnt" > + fi > + mounted= > + fi > + ;; > + esac > fi > > if [ "$mounted" ]; then > ------ > > Discussion: > Since we should offer the choice for the subbvolume name, this hardcoding "@" > here is not elegant. §2 Agreed, that hardcoding is not elegant; although, it does mirror the hardcoding in their installer, so it's somewhat sensible for the Ubuntu-specific case. I imagine their plan was to prototype with hardcoding, and later add user-defined config to their installer...but then configurable subvol layout was never added. My plan is to help solve the "install to a named subvolume" case for bullseye, and to adapt partman-LVM to btrfs semantics for bookworm's partman-btrfs, thus enabling user-configurable subvolume layout in the installer. IIRC users installing Debian to btrfs using Calamares already get the Ubuntu-desktop-style subvolume layout. To solve the Ubuntu-specific case, I must ask: Do you know if Ubuntu's upcoming new installer allows configurable subvolume layout? https://www.omgubuntu.co.uk/2021/02/ubuntu-is-working-on-a-brand-new-installer If not, I'm not sure there's any urgency or benefit to accommodate what they would call an officially unsupported custom config (to see why set-default=@ is unsupported see §3). I just tested the new "curtin" Ubuntu server 20.10 installer, which installed directly to subvolid=5, without creating any subvolumes. I then tried installing Ubuntu Desktop 20.10, where the btrfs-flavour partitioning recipe appears to have been replaced with ZFS, but where manual partitioning still works. Here Ubuntu still uses subvol=@, without set-default=@. So the Ubuntu-specific cases to support are: 1. Hardcoded subvol=@ * used by desktop installer 2. The new direct installation to subvolid=5 * used by new "curtin" server installer on 20.10 3. [Optional] Custom user-configured subvolume layouts 4. Tentative NACK to supporting set-default on Ubuntu * See §3 for rationale Ubuntu installations with set-default=@ are nowhere to be found. Could this be a Timeshift or Snapper-specific behaviour, or perhaps caused by some other tool? This bug might need a severity level bump if Calamares installs rootfs to a subvolume. Fedora is likewise affected. Definitely not RC though. > We should get it as: > ----- > RSUBVOL=$(btrfs subvolume get-default $tmpmnt |cut -d' ' -f 9) > ----- §3 How does this address the Ubuntu case? ie: "The btrfs-tools command 'set-default' will break Ubuntu's layout" https://help.ubuntu.com/community/btrfs#The_btrfs-tools_command_.27.27set-default.27.27_will_break_Ubuntu.27s_layout Given that it's titled "btrfs-tools", this is from an old version, so I wonder if it's still officially broken? Even if that's the case, should we really implement a solution that depends on the expectation that Ubuntu users ignore their distribution's own documentation? > Proposed fix: > So updated patch should be more like > --- > + case "$type" in > + btrfs) > + RSUBVOL=$(btrfs subvolume get-default $tmpmnt |cut -d' > ' -f 9) §4 however, this seems to be a noop for SUSE (set-default=@), and wherever a sysadmin has used set-default. Without this call, @ will be the root of /some/mount/point/suse, which is detected by os-prober, which will then generate a boot entry without subvol=@, which is fully functional because set-default=@ makes subvol=@ unnecessary on the kernel command line. Or does GRUB still not support btrfs' set-default subvol? ie: will GRUB fail to load SUSE's kernel from @/boot/vmlinuz, which links to @/boot/vmlinuz-$version? Also, for a given bootable subvolume, isn't it more accurate to parse @foo/etc/fstab, or @foo/.../snapper-config, when it generates flags into grub for boot environment support, or /e/default/grub (or equivalent), and then generate entries for all bootable subvolumes, nested under a top-level $alternative-distribution GRUB menu entry, (for each partition containing an $alternative-distribution) rather than using "get-default" as a indicator for the blessed subvolume? Unfortunately neither approach guards against the case where Debian is being used to rescue another distribution that crashed during an upgrade, before its fstab could be updated (see §5). ISTM that without something like an LFS 4.0 that specifies a standardised layout for btrfs boot environments, supporting installations on btrfs means that os-probe will eventually need to support all kinds of distro-specific layouts (or layouts created by a snapshotting tool :-o)...not to mention the variety of user-configured layouts btrfs enables. Our Rescue CD also needs bootable subvolume detection logic, and it seems like os-prober should use the same logic. That's why I find this an interesting bug and topic! Apologies that this reply is a bit long, by the way; because I'm invested in solving the Rescue CD case, I'm using this bug for perspective on how that solution ought to look. > + if [ -n "$RSUBVOL" ] && [ -x "$tmpmnt/$RSUBVOL/lib" ] > && \ > + ! mount --bind "$tmpmnt/$RSUBVOL" "$tmpmnt"; then > + warn "failed to mount btrfs subvolume $RSUBVOL > on $partition" > + if ! umount $tmpmnt; then > + warn "failed to umount $tmpmnt" > + fi > + mounted= > + fi > + ;; > + esac > --- §5 I don't think this check is the right approach for the general case. For example, a subvolume named "@var" (or @/var on SUSE) will usually contain "lib", yet will be unbootable. Likewise, ro subvolumes should be excluded. ISTM that the criteria should be 1) rw subvolume, 2) contains "/sbin/init" rather than contains "/lib". Set-default=@ appears to only be relevant to SUSE installations. I also tested a Fedora 33 installation (defaults to btrfs, with no set-default=rootfs, and which uses -o subvol=rootfs). I think it might be a good idea to additionally parse @subvolume/etc/issue, for each bootable subvolume, so that the grub menu entry accurately describes the OS version on a subvolume for the pre and post upgrade case (ie: @20.04 and @, where @ now contains 20.10), but that's arguably wishlist/nice-to-have if all generated entries can be safely booted (ie: never booting a @new_rootfs with @old_var nor @old_rootfs with @new_var. See §4). > This is much simpler patch than ones discussed in > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=688336 > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=699189 > > Since we expect any sane person set-default to the root-filesystem. Citation please. P.S. There's no need to imply that everyone who doesn't hold your view is a person who is mentally ill with a broken ability to process logical problems... Please consider that your claim is also a logical fallacy; it's an appeal to an imaginary authority mixed with a personal attack that fails to rationally address the topic, stated as a conclusion that contains an invalid implied premise. Also, no, this isn't an accurate use of "we". My view is that set-default=@subvolume is an unneeded feature when 1) Grub already supports a default entry for OS, kernel, and subvolume, 2) Grub already supports alternative boot entries for OS and kernel version 3) Grub already supports alternative boot entries where rw snapshots can be booted using subvol=@alt_rootfs (nascent boot environments, or in SUSE's case, actual boot environments). 4) systemd-boot also supports all of this, and is just missing auto-generation of loader entries. Fedora 33 creates "root" (for rootfs) and "home" subvolumes, and also doesn't use set-default=root...so of the major distributions, it would seem only SUSE meets your criteria for "sane". > You could add fall-back with "elif" to use Ubuntu specific hardcoded subvolume > name. > > This should make Debian friendly to older Ubuntu with btrfs. > > What do you think? I think we can, and should, do better! :-) The universal operating system should avoid special casing, whenever possible. Supporting set-default=@user_defined for a second Debian installation (on another partition) seems reasonable, but it isn't something I'm interested in supporting, which is why I've invited you to join/co-found the nascent "Debian btrfs enablement" team :-) Also, we've met, cross signed GPG keys, and I respect you, so I'm sure collaborating would be a pleasure! Thank you, Nicholas
signature.asc
Description: PGP signature
