Re: Change templates: CD -> installation medium - proposal #2
Justin B Rye writes: > Holger Wansing wrote: >> And they all need to covered here. >> Maybe we cannot find a term that works perfectly for all of them, however >> having a suitable coverterm for all would be the major goal. > > Contenders so far: > * "insert another medium", "insert more media", and "insert another >item of media" all run the risk of annoying readers because they're >unidiomatic or "wrong" or clumsy. As far as I can tell "installation media" is the most commonly used term (Eg: RedHat, SUSE, Microsoft, Apple). While it might not be the ideal term, it's worth mentioning that idiosyncratic documentation may also have a "risk" of this kind. My vote is for "installation media" because I've never seen any of the proposed alternatives in print. It definitely needs "installation" for precision, because there are many types of media, and "media" is a generic and neutral term. Oh, and "media" vs "medium". To my mind mediums tend to be things like substrates that are changed by the process being discussed (eg: culture/growth mediums), or as something used to allow a connection (eg: fibre is a medium for high speed communication). Pretty boring I guess...just as long as we don't write something like "insert the bit kazoo now"! signature.asc Description: PGP signature
Re: Change templates: CD -> installation medium - proposal #2
Ben Hutchings writes: > On Tue, 2019-09-10 at 20:21 +0200, Holger Wansing wrote: [snip] > > I don't know exactly what cdrom-detect does, but it may still be > specific to optical drives. In that case you could use more specific > terms here, e.g. "The optical disc drive contains a disc which cannot > be used for installation." > > Otherwise a suitable new text could be something like "The detected > drive does not contain a usable installation disk". > More generally, maybe "The selected/detected source is not valid installation media", plus an error like "missing metadata|corrupt media|unreachable host|wrong suite/release/Debian version|et al"? BTW, I've always found that the strings in Debian Installer were sensible, going all the way back to installation from sets of cdroms or floppies, so everyone's doing a great job! :-) Cheers, Nicholas signature.asc Description: PGP signature
Re: Change templates: CD -> installation medium - final patch
Holger Wansing writes: > Hi, > > Nicholas D Steeves wrote: >> Justin B Rye writes: >> >> > Holger Wansing wrote: >> >> And they all need to covered here. >> >> Maybe we cannot find a term that works perfectly for all of them, however >> >> having a suitable coverterm for all would be the major goal. >> > >> > Contenders so far: >> > * "insert another medium", "insert more media", and "insert another >> >item of media" all run the risk of annoying readers because they're >> >unidiomatic or "wrong" or clumsy. >> >> As far as I can tell "installation media" is the most commonly used term >> (Eg: RedHat, SUSE, Microsoft, Apple). While it might not be the ideal >> term, it's worth mentioning that idiosyncratic documentation may also >> have a "risk" of this kind. My vote is for "installation media" because >> I've never seen any of the proposed alternatives in print. It >> definitely needs "installation" for precision, because there are many >> types of media, and "media" is a generic and neutral term. > > That's a good argument IMHO, not mentioned before. > > And given that most people were fine with this term in the first run > (except of Ben), and there is no perfect choice in this case anyway, > I will move back to "installation media" and to Justins review from > https://lists.debian.org/debian-boot/2019/09/msg00082.html > Thanks :-) I know it's boring and conventional, and that it's an ad populum to advocate for what others are doing. P.S. I'm also passionate about (and educated/indoctrinated into) resisting the trend towards imprecise language, passionate about philology, philosophy, linguistics, and the difficulties of translation--particularly poetics; that said, DUGs, mailing lists, media such as blogs and youtube, conferences, et al have much greater power than the strings we use in the installer. People are going to use whatever feels right to them, and the best we can do is provide a familiar compromise as an official point of reference. signature.asc Description: PGP signature
Bug#935931: Re: Bug#935931: debian-installer: Reinstalling Debian on a current Debian installation without erasing or fomatting the home folder
Geert Stappers writes: > On Fri, Sep 27, 2019 at 05:19:06PM -0400, Daniel wrote: >> Holger Wansing wrote: >> > The debian-installer supports similar use case via the "separate >> > partition for /home" approach. >> to reinstall Debian on top of itself without overwriting the home partition. > > Yes, that is what Holger is telling. I think Daniel is requesting an option that does something like this: find /install-target -maxdepth 1 | grep -v 'home\|lost+found' | xargs rm -rf Maybe this way isn't robust enough, but active mounts shouldn't have their mount points removed, because rm: cannot remove '/install-target/foo': Device or resource busy BTW, Daniel, you can decruft your system with "apt purge --autoremove foo", which also deletes config in /etc and will notify you if any files remain in /var. One of the greatest strengths of Debian is that unlike other operating systems, smooth upgrades between stable versions are taken seriously...gravely seriously...so one never needs to reinstall. The only things that I've seen that have ever required action are packages that needed manual configuration updates in /etc (equally solvable by apt purge), and obsolete/broken configuration in /home/user (not solved if this feature request is implemented). What problem is this feature request intended to solve? FrankenDebian? https://wiki.debian.org/DontBreakDebian Cheers, Nicholas P.S. apt install installation-birthday :-) signature.asc Description: PGP signature
Bug#840248: debian-installer: Add btrfs subvolume setting for snapshot
Control: owner -1 ! Hi, On Sun, Oct 30, 2016 at 02:21:39PM +0100, Philipp Kern wrote: > On 10/11/2016 11:40 PM, Nicholas D Steeves wrote: > > So far, the plan is to default to simple @rootfs and @home subvolumes, > > because I've read that backing up OpenSUSE systems is cumbersome with > > all of those subvolumes, and also because of the KISS principle; [...] > > FWIW, given that I just encountered this myself: rescue(-mode) will need > a fix in this case because by default it mounts the top-level, which > means that the actual chroot is one level down. Although I guess setting > the default subvolume id to the one of whatever you call @rootfs should > also fix this. > I've submitted a tested MR at: https://salsa.debian.org/installer-team/partman-btrfs/merge_requests/1 I decided to leave configuring @home up to the user, because the user may wish to mount /home using another block device, possibly on a non-btrfs volume. Also, adding full-fledged btrfs subvolume configuration (eg: forking partman-lvm) to DI will require a comaintainer. The bus-factor is too high for me to do it alone. Would you like to take care of rescue-mode or shall I? Cheers, Nicholas signature.asc Description: PGP signature
Bug#935931: Re: Bug#935931: debian-installer: Reinstalling Debian on a current Debian installation without erasing or fomatting the home folder
Hi Daniel and everyone reading this, Daniel writes: > I am addressing another case, the one you have not separated partitions > for /, /home and swap. > Len and Daniel, WRT swap, hibernation is useful when you need to preserve the state of applications that aren't aware of a desktop session manager, eg: half-finished work in a terminal. As I see it the primary advantage to a swap file is the ability to expand it after a RAM upgrade. eg: that a reboot using a livecd to resize partitions is not required. Granted, expanding a swap file is only necessary if one cares about hibernation... > As a matter of fact if the installer is able to recognize the home > folder having /home separated in another partition is not necessary > anymore. The advantage respect having the /home separated, specifically > for a desktop use are noteworthy. > The key point is "is able to recognize [safely and reliably]", and that requires someone who values the feature to work on debian-installer (not fun), and also maintain it (not fun). > If the installer, instead of creating /, /home and swap, creates just / > and a swap file and if is able to reinstall itself without overwriting > the home folder I think is a huge improvement. As a matter of fact if > you reinstall Debian, even with /home in another partition, there is not > any assisted aid that explain you how to properly setup the /home > partition. Having the system partitioned is already a setup for advanced > user. > As Len mentioned, Debian users tend to believe that it's a better use of time to learn how to fix things if one is going to track testing/sid/experimental than hitting the panic button and losing state. My mother (who lives 3500km away, runs Debian stable, and installed it herself eight years ago) doesn't need this feature. She's not an intermediate or advanced user and is currently running buster. She also has root on her laptop, so has the power to render it unusable. Daniel, would you please take a look at Bug #941627 (ITP: grub-btrfs -- provides grub entries for btrfs snapshots (boot environments/restore points)? I think that this is a feature that would solve the situation where one ran a testing/sid/experimental upgrade at a time when one didn't have time to do the work required to fix the brokenness. Here's how it will work: 1. Install to btrfs (after my MR is merged this will automatically create @rootfs subvolume, eg: special directory kind of like a pseudo logical volume). Reboot. 2. Run a one-line command and add one line to fstab to create a @home subvolume--this is necessary to exclude /home from the snapshots. I can write a beginner friendly helper script if necessary. 3. Take a snapshot before running a dangerous upgrade (easy one-line command). Eventually this may be automatic (eg: something like apt-btrfs-snapshot) 4. If the upgrade produces a broken state the user doesn't have time to fix, simply boot into a known-good copy of / using the grub menu to select the correct entry. If the top-most one isn't good, reboot and try with the second top-most one until a good one is found. After confirming all is well, rename the @rootfs subvolume and create a new read-write snapshot named @rootfs based on the current boot environment. This step is only necessary if you want to reboot into the old copy of the rootfs automatically. You also get to keep the @rootfs-does-not-work copy. 5. The logical progression of this feature is to create a snapshot, dist-upgrade the snapshot, test it (without rebooting), and if everything looks good then mark it as the default boot environment. Eventually there will be a GUI! While wiping and reinstalling may be the best other OSs can aspire to, Debian can, and will, do better. I hope you'll agree the project I'm working on will solve the root issue you're reporting, and that you'll agree it's a more elegant and time-efficient solution :-) In the meantime, to remove everything but /home and reinstall without reformatting you can reboot into a Debian rescue system or using the Debian Live image, mount your volume, use the solution I provided in my initial reply (p.s. I consider that a risky approach), then follow: https://www.debian.org/releases/buster/amd64/apds03.en.html https://wiki.debian.org/Debootstrap Sorry for the belated reply, I've been swamped with work. Cheers, Nicholas signature.asc Description: PGP signature
Bug#939070: removing gnome desktop in tasksel has little or no effect
Hi Nicolas and Joey, Nicolas Braud-Santoni writes: > On Sat, Aug 31, 2019 at 03:57:07PM -0400, Joey Hess wrote: >> I accidentially installed debian 10.0 with gnome rather than xfce, so >> after the installation, I re-ran tasksel, unselected gnome, and selected >> xfce. >> [...] >> Tasksel probably removed task-gnome-desktop, but many of its >> dependencies appeared to still be installed. > > Yes, removing task-gnome-desktop won't do much if you do not run > `apt autoremove` or somesuch. > > Of course, making tasksel run autoremove would be a terrible idea, > since it might remove unrelated packages. > Agreed. > I'm not sure how that can be addressed, TBH. Maybe this?: 1. use changes their selection of desktop task. 2. tasksel gets a list of recursive dependencies for the task that is being removed. 3. the task is removed, but tasksel still has work to do... 4. the new desktop task is installed. 5. use a dry run of autoremove to get a list of packages that can be autoremoved 6. cut anything from the list at #5 that isn't in the list at #2 7. remove autoremovable packages that match the removed task What are the pitfalls with this method? Sadly I cannot volunteer for this work as I don't know Perl. Cheers, Nicholas signature.asc Description: PGP signature
Bug#944632: override: color-theme-modern:editors/optional
Package: ftp.debian.org Severity: normal Dear ftpmasters, I was not aware that the Emacsen Team had standardised on Section: editors, moving away from Section: lisp. This particularly makes sense for things that affect UI like themes and new modes. David Bremner notified me after I had uploaded. A section change would be very much appreciated :-) Also, please let me know if the use of reportbug's "override" submenu for ftp.debian.org bugs was appropriate when filing this bug. Thanks! Nicholas
Bug#949626: busybox-static: Please include less and ftpput in busybox-udeb
Witold Baryluk writes: > On Wed, 22 Jan 2020 at 22:03, Philipp Kern wrote: >> > Please include functional less, just like in busybox-static with the same >> > build options. >> >> There is nano though. (I'd still second less. I think we can spare the >> space.) > > ~17kB from my estimates and looking at build logs. > >> > ftpput to transfer files out would good option too. >> >> The age of FTP has long passed. > > You think you can fit the scp or ssh then? :D I doubt so. > > In the lack of ftpput, one would still use netcat / nc, > which is actually available in busybox-nc to accomplish > the same task, just in more convoluted way. > +1. Also, could dropbear-initramfs solve this? Cheers, Nicholas signature.asc Description: PGP signature
Bug#951387: Please accept https://salsa.debian.org/installer-team/console-setup/merge_requests/3
Mantas Baltix writes: > Tags: patch > > Hi, why such simple patch isn't accepted - 3 weeks already passed... > Don't feel bad! Here's one I've been waiting three years for (tested with custom install media in a VM): https://salsa.debian.org/installer-team/partman-btrfs/-/merge_requests/1/commits Cheers, Nicholas signature.asc Description: PGP signature
Bug#951387: Please accept https://salsa.debian.org/installer-team/console-setup/merge_requests/3
Hi Adrian, John Paul Adrian Glaubitz writes: > On 3/9/20 10:33 PM, Nicholas D Steeves wrote: >>> Hi, why such simple patch isn't accepted - 3 weeks already passed... >>> >> >> Don't feel bad! Here's one I've been waiting three years for (tested >> with custom install media in a VM): >> >> >> https://salsa.debian.org/installer-team/partman-btrfs/-/merge_requests/1/commits > > I can have a look tomorrow. > Thank you! Best, Nicholas signature.asc Description: PGP signature
Bug#615646: [installation-guide] How do I know which CD has the package I need?
On Sun, Jul 29, 2018 at 11:59:34AM +0200, Wouter Verhelst wrote: > On Sun, Jul 29, 2018 at 07:50:18AM +0200, Holger Wansing wrote: > > +Also, keep in mind: if the CDs/DVDs you are using don't contain some > > packages > > +you need, you can always install that packages afterwards from your running > > +new Debian system (after the installation has finished). If you need to > > know, > > Drop the comma (you do need it in German, but English doesn't need it) If the subordinate clause precedes the main clause, then the comma is mandatory. Likewise, even if "then" is implied, the comma is mandatory. That said, a comma separating the subordinate and main clause is not mandatory if the subordinate clause follows the main clause. /\ no comma If you need to know, on which CD/DVD to find a specific package, visit... /\ /\ no comma, because/ "on which...package" mandatory comma =/ is a propositionalfor the "if...package" phrasesubordinate clause Are prepositional phrases enclosed by commas in German grammar? Cheers, Nicholas signature.asc Description: PGP signature
Re: Creating my own Preseeded ISO with partman replaced by a ZFS step
On Sat, Aug 11, 2018 at 01:32:40AM +0200, Cyril Brulebois wrote: > Hi, > > Bailey Parker (2018-08-10): > > Is there a sane way to go about adding ZFS root support to my preseeded > > install or should I abandon this and wait for better support? If the > > latter, are there steps I could take to add better support given my > > limited knowledge of d-i? > > I'm afraid we're not going to support ZFS as that would mean supporting > out-of-tree kernel modules, which we migrated away from years ago. Automated Debian on ZFS installation seems like a problem that FAI should be able to solve. https://wiki.debian.org/FAI They don't yet have ZFS support, it's on the roadmap, and they might be looking for someone to implement it. https://fai-project.org/roadmap Cheers, Nicholas signature.asc Description: PGP signature
Bug#907704: choose-mirror: default to deb.debian.org
On Mon, Sep 03, 2018 at 08:54:56PM +0100, Ben Hutchings wrote: > On Mon, 2018-09-03 at 20:13 +0200, Karsten Merker wrote: > > On Mon, Sep 03, 2018 at 04:41:10PM +0200, Julien Cristau wrote: > > > Control: tag -1 + patch > > > > > > On 08/31/2018 06:27 PM, Julien Cristau wrote: > > > > Package: choose-mirror > > > > Severity: wishlist > > > > X-Debbugs-Cc: tfh...@debian.org > > > > > > > > I think it's time for choose-mirror to stop asking by default. AFAIK > > > > deb.debian.org works well enough now that we don't need users to > > > > manually select a mirror close to them. [...] > > > > Hello, > > > > I can see the argument for not asking to select a mirror when > > there is a well-working mechanism for automatically choosing a > > "near" (in networking terms) mirror. Does deb.debian.org fulfill > > everybody's needs in this regard? ISTR that there were some > > discussions in the past that deb.debian.org didn't resolve to > > particularly useful mirrors for some parts of the world, but I > > have no idea whether that is still a problem. My personal > > experience with deb.debian.org hasn't been that great - instead > > of redirecting me to the Debian mirror that is run by my local > > ISP (and that is in d-i's mirrorlist), it redirects me to an AWS > > instance hosted rather "far" away in networking terms. > [...] > > The existing mirror network has several longstanding problems: > > 1. Many mirrors don't reliably update > 2. Some mirrors aren't reliably available at all > 3. Many mirrors don't carry all release architectures (even a few >of the "primary" ones don't) > 4. Most mirrors don't support TLS > > httpredir.debian.org attempted to solve the first 3 problems while > still doing what you want: it redirected to local mirrors known to have > up-to-date files. This would have been almost ideal as a default. But > apparently it required a lot of maintenance work, which no-one was > prepared to continue doing. > > That's why deb.debian.org is a plain CDN which doesn't rely on the > existing mirror network. It also supports TLS (which I think should > also be enabled by default in the installer). > > If deb.debian.org still doesn't provide reasonably fast service in some > countries, then maybe we should still ask—but then we should put > deb.debian.org at the top of the mirror list for most countries. /\ +1 /\ Like Karsten, my experience with deb.debian.org has been inconsistent. With a 50 Mb/s ADSL line in Montréal, most of the top candidates mirrors from netselect will consistently deliver ~6200 kB/s, but deb.debian.org often connects to an AWS instance where the download proceeds no more than 350 KB/s... Additionally, I think that it is reasonable that users look at the mirror list for the following reason: Our mirrors are a list of organisations and universities who donate storage and bandwidth. Having users look at this list provides the opportunity for the user to recognise their donation--something like "oh, these are the entities who support FLOSS in my country". Thus, I believe that hiding this from the user reduces the reciprocity with these donors, and reduces the incentive to donate storage/bandwidth. That said, I think there should be some sort of mechanism to reward those mirrors who provide TLS. It's becoming normal for a browser to display "insecure site" for those which don't support SSL... Cheers, Nicholas signature.asc Description: PGP signature
Bug#907910: debian-installer: Not possible to reset root password
On Tue, Sep 04, 2018 at 12:48:48AM +0200, Tuxicoman wrote: > Package: debian-installer > Severity: normal > > Dear Maintainer, > > I tested Debian testing installer the 4 september 2018 > > At one step, the installer asks for setting the root password. > I pressed Enter, without entering any password, and the installer went to the > next step (creating user accounts) > > I tried to fix this by restarting at a previous step (network configuration) > but the root password step doesn't show anymore. It jumps to user account > creation step directly after network configuration. > > Bugs are : > - maybe empty root password should not be allowed > - the root password setting step should be replayable If the root password is unset/blank, root is disabled and the first user is added to sudoers. Perhaps this should be made explicit in the installer? Cheers, Nicholas signature.asc Description: PGP signature
Re: Bug#908834: please build libzstd udeb so that btrfs-progs can use zstd in Debian Installer
Control: retitle -1 please build libzstd udeb so that btrfs-progs can use zstd in Debian Installer Control: reassign -1 src:libzstd/ 1.3.5+dfsg-1 On Tue, Oct 09, 2018 at 06:24:47PM +0200, Alex Mestiashvili wrote: > On 10/09/2018 04:32 PM, Dimitri John Ledkov wrote: > > is o > > On Tue, 9 Oct 2018 at 12:23, Alex Mestiashvili > > wrote: > >> > >> On 09/14/2018 08:04 PM, Nicholas D Steeves wrote: ... > >>> > >>> Would you please build a zstd udeb so that btrfs-progs can use zstd in > >>> Debian Installer and Rescue System? It uses zstd for transparent > >>> filesystem compression. > >>> > >>> eg: `chattr +c`, or `btrfs filesystem defrag -c`, or via a mount > >>> option `compress=zstd`. I believe the first and last of these use the > >>> kernel's libzstd, and that the udeb is primarily required for > >>> `btrfs-repair` to handle zstd extents in the Rescue System. Also, please > >>> continue to CC Dmitri Ledkov, Debian's btrfs-progs maintainer. > >>> ... > >> > >> As far as I see it's enough to add udeb stanzas in d/control in order to > >> build the udebs[0]. > >> Is there anything else to consider before uploading lisbzstd with udebs? > >> > > > > No, that's not at all enough. It ends up creating two empty packages, > > without any files in them. > > Oh, I see. I thought there is some debhelper magic involved and didn't > check the generated packages.. > I'm also surprised it didn't run a second dh_install run with $DESTDIR set to debian/udeb-package. Or alternatively dh_install debian/udeb-package. > > > > One needs to actually install a library into the library udeb and > > tools into tools-udeb. > > Note for fbtrfs only library-udeb is needed. > > Does that also apply for btrfs-repair? Initial bug report is about zstd > udeb as I see. Sorry for the inaccuracy; I've retitled and reassigned this bug. Initially I thought zstd was the name of the source package. > > > > Also do get it reviewed, as last time unwritten rules w.r.t. udebs got > > enforced and above patch was rejected on ground of not strict enough > > alternative shlibs deps generated. > > Thank you for clarifying, but I didn't understand the reason of reject :). Ditto, me neither. If I had to guess maybe ftpmasters want a manually generated symbols file for libzstd and libzstd-udeb? https://wiki.debian.org/UsingSymbolsFiles > @debian-boot folks, please review and please either fix it or explain > what is required. > Please read what Cyril (Debian Installer Team) wrote at these bugs in case these questions have already been answered: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=898410 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=886968 > Are there any udeb related docs available? > Sorry, I don't know of any. Kind regards, Nicholas signature.asc Description: PGP signature
Re: Bug#908834: please build libzstd udeb so that btrfs-progs can use zstd in Debian Installer
On Tue, Oct 09, 2018 at 08:41:35PM +, Holger Wansing wrote: > > > > > Are there any udeb related docs available? > > > > > > > Sorry, I don't know of any. > > Maybe the d-i internals? > https://d-i.debian.org/doc/internals/ Thank you Holger! Yes, that's the one: https://d-i.debian.org/doc/internals/ch03.html Bookmarked! Cheers, Nicholas signature.asc Description: PGP signature
Bug#840248: debian-installer: Add btrfs subvolume setting for snapshot
Control: noowner -1 Hi, Update: I've learned that Debian Installer work needs to be completed about four months before the freeze. As it looks like I'll be swamped with work for the next month or two I'm unsetting myself as owner. If no one finishes the work in time for buster freeze I'll resume work sometime this spring. It's not nearly as simple as I'd hoped it would be. Cheers, Nicholas signature.asc Description: PGP signature
Re: Fix references to KDE
Replying from my phone. I think the project now uses "KDE" to refer to a combination of the organisation and its developer community. The project now uses the phrase "KDE Frameworks" to refer to the libraries and frameworks for the desktop environment. Finally "KDE" seems to have been rebranded "Plasma Desktop"; however, major revisions are sometimes branded "KDE Plasma x", where x in an integer. I suspect that "KDE Plasma" is meant to signify "Plasma, by KDE" (Eg: by the organization and greater Dev community) or maybe symbolically "community desktop". It might alternatively be transitory branding, eg: it makes sense to keep "KDE" for people who know what it is, and add "Plasma" for the new generation who only know the DE by that name. So I think the change is sensible, although I find it confusing that the KDE acronym seems to be in the process of being redefined. I liked the old name: Kopernikus Desktop Environment 😊 On Sun, Oct 28, 2018, 17:18 Holger Wansing wrote: > Hi, > > there is a merge request on Salsa: > > > https://salsa.debian.org/installer-team/debian-installer/merge_requests/4/diffs > > to change some references of KDE into KDE Plasma. > > Do we want to change this? > > Holger > > -- > Sent from my Jolla phone > http://www.jolla.com/
Re: Bug#908834: please build libzstd udeb so that btrfs-progs can use zstd in Debian Installer
Hi Alex, Cyril, and anyone else reading this, On Fri, Oct 12, 2018 at 10:09:48AM +0200, Cyril Brulebois wrote: > Hi, > > Alex Mestiashvili (2018-10-12): > > Fixed all the mentioned above issues in the repository. > > That's looking good indeed. > > Please note that by building a udeb you'll be subject to this once in a > while: > > https://lists.debian.org/debian-devel-announce/2014/08/msg3.html > > (That hasn't happened in a long while because I've been otherwise busy, > but I still hope to resume regular d-i releases at some point.) > > > Thank you for the detailed answer! > > No problem, always easier/happier to catch such issues before packages > reach the archive. ;) It seems libzstd 1.3.5+dfsg-2 hasn't yet reached the archive. Maybe it was not uploaded, or maybe it was rejected for some reason? https://salsa.debian.org/med-team/libzstd/commit/9b865b77d2bfc41c5865f255cf3e4aae18bbe934 Thanks you for working on this! Nicholas signature.asc Description: PGP signature
Re: Bug#908834: please build libzstd udeb so that btrfs-progs can use zstd in Debian Installer
Hi Alex, Cyril, Dimitri, and anyone else reading this, On Wed, Nov 28, 2018 at 08:41:18PM +0100, Alex Mestiashvili wrote: > > Hi Nicholas, it is in the new queue: > > https://ftp-master.debian.org/new/libzstd_1.3.5+dfsg-2.html > > We just need to wait or ? I fear that waiting will put us too close to the freeze, and then it will be an inappropriate time to make these changes. I've asked #ftp-masters on irc.oftc about the status of libzstd in NEW (2 months and counting) Cyril and Dimitri, if either of you have a chance to ask an ftp-master for feedback, and it wouldn't be too much of a bother, would you please? Thanks! Merry Christmas, Nicholas signature.asc Description: PGP signature
Re: Bug#908834: please build libzstd udeb so that btrfs-progs can use zstd in Debian Installer
On Sat, Dec 29, 2018 at 04:53:02PM +0100, Cyril Brulebois wrote: > Hi FTP team, > > I've just been reminded (see below) of the zstd udeb addition currently > sitting in NEW; the udeb addition was reviewed (even amended) and should > be ready for use in other d-i components. Could you please let this > package through? Thanks already! > > Cheers, > Cyril. Thank you Cyril! Yay, libzstd_1.3.5+dfsg-2 (with udeb) is in sid, and has also migrated to testing! I took a look at partman-btrfs, which brings in the btrfs udeb. Does partman-btrfs need to be modified to also depend on the new zstd udeb, or will it be added as a recursive dep when btrfs-progs is rebuilt? Happy New Year! Nicholas signature.asc Description: PGP signature
Re: Bug#908834: please build libzstd udeb so that btrfs-progs can use zstd in Debian Installer
On Sat, Dec 29, 2018 at 04:51:49PM +1100, Dimitri John Ledkov wrote: > On Sat, 29 Dec 2018 at 15:54, Nicholas D Steeves wrote: > > > > Hi Alex, Cyril, Dimitri, and anyone else reading this, > > > > On Wed, Nov 28, 2018 at 08:41:18PM +0100, Alex Mestiashvili wrote: > > > > > > Hi Nicholas, it is in the new queue: > > > > > > https://ftp-master.debian.org/new/libzstd_1.3.5+dfsg-2.html > > > > > > We just need to wait or ? > > > > I fear that waiting will put us too close to the freeze, and then it > > will be an inappropriate time to make these changes. I've asked > > #ftp-masters on irc.oftc about the status of libzstd in NEW (2 months > > and counting) > > > > Cyril and Dimitri, if either of you have a chance to ask an ftp-master > > for feedback, and it wouldn't be too much of a bother, would you > > please? > > > I'm happy. From btrfs-progs point of view, it simply needs a binNMU to > pick up and start using libzstd udeb which release team can schedule. Brilliant! :-) It looks like we're finally good to go. I'll also ask the backports team if that (is_udeb_available) conditional will now violate backports policy, eg: if it's a problem that the the stretch-backports btrfs-progs will not have zstd support when the buster one does. I worry it might, and will follow up with a btrfs-progs bug if it does. Happy New Year! Nicholas signature.asc Description: PGP signature
Re: btrfs related option in the debian Installer
Hi Pierre-Yves, Thank you for your interest :-) Reply follows inline: Cyril, if you have time to skip to the bottom for a problem/question I'd really appreciate it. Pierre-Yves David writes: > Hi Nicolas, > > Cyril pointed to you as the person to talk to about btrfs. > Thank you for CCing debian-boot. My username is "sten" and my IRC nick is "sten0". > I am a btrfs and Debian user and there are a couple of option I would be > happy to have avaible from the installer. > > The first one is quite simple, withing the "partition management", there > is a dialog to select mount option. However, the > `user_subvol_rm_allowed` option is not available there. The option is > handy and it would be nice to have a line for it. > I think we should wait a while before doing this one and should not be encouraging user-initiated snapshots at this time. At a minimum we should have support for a /home subvolume first. If I understand btrfs correctly a malicious user can still DOS the I/O from the rootfs subvolume, similarly to a fork bomb. I'd possibly like to see a "user_subvol_mk_allowed" mount option, where the default would be "no", if finer-grained access controls don't materialise soon. > The second one is a bit trickier (but not crazy either). I usually > prefer to install my system in a dedicated subvolume (set as the default > subvolume) to allow me to easily do clean snapshot of the system content > with clean snapshot management (ie: outside of the things we > snapshot). Agreed, and also the current state of no subvolumes is too hard for many users to migrate from. At some point I should probably write up a doc, if only for users who installed to btrfs before the future point in time when we'll install to a rootfs subvolume and have fancy tools to do next-gen things. > I would be nice to have an option to specify a subvolume that should be: > > - created > - set as default subvolume for the btrfs volume This isn't necessary, because "subvol=@rootfs" can be automatically added to the mount options of the volume chosen for "/". I believe this approach is better because it's less opaque, and because it appears to better support future boot environments/bootable snapshots/system restore points. > - used by the Debian installer to install the system too (should be > trivial once the first 2 are done). > > What do you think and how should I proceed if I want to contribute this > kind of changes ? > Well, I'd like to form a btrfs task force team! You're welcome to join :-) I have an initial proof of concept MR here: https://salsa.debian.org/installer-team/partman-btrfs/-/merge_requests/1 It worked in March, but started failing in August or so, and I've had trouble finding the time to debug the debian-installer. Basically my MR just creates a non-configurable "@rootfs" subvolume and installs to that. Everything else can be done post-install. As for the big plan of making btrfs layout and features configurable in d-i: the three models that come to mind are the d-i code for mdadm, LVM, and ZFS (if still exists and has been kept up-to-date), with integration with cryptsetup. Fedora's now defaulting to installing on btrfs, so it might soon be time to work on this. Previously the problem with making btrfs volume configuration as visible as mdadm and LVM was that this would have been an arguably premature implicit endorsement for general use. Cyril, now the question: Did I miss the memo for a big d-i change that limited what partman-foo packages could do or make a mistake somewhere? For btrfs with a rootfs subvolume, the volume needs to be unmounted, and then remounted. Could this be triggering a d-i error condition? Thank you, Nicholas signature.asc Description: PGP signature
[Attn Zinoviev] I'd like to adopt partman-btrfs
Hi, I noticed there's been no active development on partman-btrfs since 2016, and I've had an MR open for over a year https://salsa.debian.org/installer-team/partman-btrfs/-/merge_requests/1 Does anyone have any objections to me adopting it? Anton? Regards, Nicholas signature.asc Description: PGP signature
Bug#980782: Info received (Bug#980782: Acknowledgement (os-prober: linux-boot-prober returning "root=/dev/dm-X" line instead of expected "root=UUID=[UUID128]))
Mirko Vogt writes: > Looking at /usr/share/initramfs-tools/scripts/local-top/lvm2 more > closely, passing a UUID also wouldn't trigger a `vgchange -ay` here. > But a path like /dev/mapper/X would. > So maybe the question is rather: how to make os-prober return a > "root=/dev/mapper/X" line instead of one containing a UUID(?) The first thing that comes to mind is: For a given UUID run blkid, and exclude all lines that do not match the UUID count lines and error if there are duplicates (it probably already does this, and I think the risk of collisions most significant for short UUIDs like FAT has) as part of that regex, check for ^/dev/mapper, with that anchor use /dev/mapper/X for the truly unique UUID Cheers, Nicholas signature.asc Description: PGP signature
Bug#983107: os-prober: generic subvolume support for btrfs
Hi Osamu! §1 Would you like to join/co-found the nascent "Debian btrfs enablement" team? In the coming years there will be an increasing number of software that will need a "get all valid bootable rootfs candidates for a btrfs volume". Right now we have GRUB, Debian Rescue (TODO), bootloaders for ARM (TODO) and in the future systemd-boot (aspirationally TODO bookworm), in addition to whatever software will be used for "boot environments". I wonder if os-prober should be the site for this "return list of bootable subvolumes" functionality, rather than duplicating the logic in multiple places? Cyril? I'm not sure because os-prober depends on grub-common, which AFAIK isn't used on ARM. Do we need a NEW package for btrfs-related support functions? Alternatively, btrfs-progs seems like a logical place for functions like "get a list of all bootable subvolumes". Adam, what do you think, is btrfs-progs the right place for helper functions that return volume layout, noting when a subvolume is bootable, perhaps as bundled shell scripts? Could "btrfs subvolume list" be the right place for this? And yes, I think this tooling should ideally exist upstream and not be Debian-specific. I suspect this functionality may be duplicated in Snapper. P.S. Hideki Yamane, who maintains Debian's Snapper package, ACKed installing Debian to a subvolume (without the use of set-default) a long time ago, but it's probably time to check in with him again (CCing him). With truly generic subvolume support for grub that checks for valid bootable rootfs candidates and creates a menu entry for each candidate we will have a working prototype of "boot environments" today. As far as I know, only SUSE has this (IIRC via Snapper, probably with extra grub hooks), plus Arch--on an opt-in basis--with grub-btrfs. Osamu Aoki writes: > Package: os-prober > Version: 1.78 > Severity: normal > > Issue: > Currently Debian os-prober support only btrfs root-filesystem on the root of > the btrfs, i.e., ID 5 (FS_TREE). This makes auto generated grub.cfg to miss > Linux install to btrfs for some Ubuntu and Suse since they put root-system > under @ subvolume. Thank you for bringing this issue to my attention! Yes, I agree, we should fix this. > Existing patch in other distro: > Ubuntu ships patched os-prober 1.77 to address its subvolume use (@ as root- > filesystem) with hardcoded path and very rudamental check for /lib directory. > > > diff -pruN 1.77/linux-boot-probes/common/50mounted-tests 1.77ubuntu1/linux- > boot-probes/common/50mounted-tests > --- 1.77/linux-boot-probes/common/50mounted-tests 2018-08-10 > 19:23:18.0 + > +++ 1.77ubuntu1/linux-boot-probes/common/50mounted-tests2020-11-02 > 11:12:51.0 + > @@ -54,6 +54,19 @@ if type grub-mount >/dev/null 2>&1 && \ > mounted=1 > type="$(grub-probe -d "$partition" -t fs)" > [ "$type" ] || type=fuseblk > + > + case "$type" in > + btrfs) > + if [ -x "$tmpmnt/@/lib" ] && \ > + ! mount --bind "$tmpmnt/@" "$tmpmnt"; then > + warn "failed to mount btrfs subvolume @ on > $partition" > + if ! umount $tmpmnt; then > + warn "failed to umount $tmpmnt" > + fi > + mounted= > + fi > + ;; > + esac > fi > > if [ "$mounted" ]; then > -- > > Discussion: > Since we should offer the choice for the subbvolume name, this hardcoding "@" > here is not elegant. §2 Agreed, that hardcoding is not elegant; although, it does mirror the hardcoding in their installer, so it's somewhat sensible for the Ubuntu-specific case. I imagine their plan was to prototype with hardcoding, and later add user-defined config to their installer...but then configurable subvol layout was never added. My plan is to help solve the "install to a named subvolume" case for bullseye, and to adapt partman-LVM to btrfs semantics for bookworm's partman-btrfs, thus enabling user-configurable subvolume layout in the installer. IIRC users installing Debian to btrfs using Calamares already get the Ubuntu-desktop-style subvolume layout. To solve the Ubuntu-specific case, I must ask: Do you know if Ubuntu's upcoming new installer allows configurable subvolume layout? https://www.omgubuntu.co.uk/2021/02/ubuntu-is-working-on-a-brand-new-installer If not, I'm not sure there's any urgency or benefit to accommodate what they would call an officially unsupported custom config (to see why set-default=@ is unsupported see §3). I just tested the new "curtin" Ubuntu server 20.10 installer, which installed directly to subvolid=5, without creating any subvolumes. I then tried installing Ubuntu Desktop 20.10, where the btrfs-flavour partitioning recipe appears to have been replaced with ZFS, but where manual partit
Bug#983107: os-prober: generic subvolume support for btrfs
Hi Osamu,
Correction for previous email: Fedora 33 does not use "subvol=rootfs",
it uses "subvol=root". I'm not sure if they changed this sometime in
the last few years, or if I misremembered and typed "rootfs" by habit.
Reply follows inline:
Osamu Aoki writes:
> If you want to use timeshift (Ubuntu origin?) or snapper (Suse origin
> ?), they seem to use non-ID-5 subvol for the system install, i.e.,
> root-FS. People use these tools on Debian too.
>
Doesn't this mean the new subvol=@rootfs (without set-default=@rootfs)
actually fulfills the assumptions required by Timeshift (Ubuntu
subvol=@, no set-default=@) and Snapper?
Hideki, would you please confirm that the changes introduced in
partman-btrfs 53 do not break Snapper? If so, is it difficult to fix
this in Snapper, and if necessary, do you have time to do so before
bullseye's release or would you like help? I still worry that Snapper
might have SUSE-specific assumptions in its design (see previous message
at bug #983107 for more context), and if the "subvolid=5" aka "subvol=/"
(Debian pre-bullseye) has hidden a "set-default=@" (SUSE) assumption
that might now manifest as broken functionality in Snapper.
Alternatively, maybe snapper installs a grub config hook?
> === Additional Tips ===
>
> You can convert a system from EXT2/3/4 to btrfs as follows by booting
> system from another system on the multiboot set-up.
>
> File system conversion is trivial as described in
> https://btrfs.wiki.kernel.org/index.php/Conversion_from_Ext3
>
> -
> $ sudo fsck.ext3 -f /dev/xxx
> $ sudo btrfs-convert /dev/xxx
> $ sudo mount -t btrfs /dev/xxx /mnt
> -
Ahhh. Is the ext4-to-btrfs via btrfs-convert case the basis for your
preference for the use of subvol=/ or set-default=@foo?
> Also, you need to update many relevant parts of grub.cfg, too. Roughly
> as ...
> -
> --- grub.cfg-orig 2021-02-17 09:32:35.855910912 +0900
> +++ grub.cfg2021-02-19 14:26:12.728005239 +0900
> ...
> -insmod ext2
> +insmod btrfs
Given "You can convert a system from EXT2/3/4 to btrfs as follows by
booting system from another system on the multiboot set-up":
From a live boot with live/rescue media, update-grub already does the
right thing for subvolume renames, or moving rootfs from subvolid=5 to a
named subvolume mounted with "-o subvol=@foo", and I don't know why the
btrfs-convert case is special. Why not, post-btrfs-convert:
1. umount the just-converted partition
* suppose it's /dev/sda2
2. mount /dev/sda2 -o subvol=/ /target
3. btrfs sub snap /target /target/@rootfs
4. btrfs sub sync /target && btrfs fi sync /target
* hasn't been necessary since linux-4.4 IIRC
5. umount /target
6. mount /dev/sda2 -o subvol=@rootfs /target
7. (make bind mounts)
8. chroot to /target as usual
9. update-grub as usual
ISTM that update-grub will do the right thing ("-insmod ext2; +insmod
btrfs") if the partition is unmounted, then remounted before running
update-grub.
Cheers,
Nicholas
signature.asc
Description: PGP signature
Bug#983107: os-prober: generic subvolume support for btrfs
Hi Osamu, Sorry, I think I misunderstood what you meant by "Since we expect any sane person set-default to the root-filesystem". I thought you meant this: since we expect any sane person will run set-default [@] to / Which really surprised me, because I didn't think anyone had this position, and I struggled to find reasons to value that perspective (hence the research!). Thank you for your patience with that long reply; it would have been *much* shorter had I not misunderstood. Now I think you meant this: since we expect any sane person will set-default to subvol=/ One good thing that came out of this investigation is knowing that SUSE uses set-default=@. Obviously their grub supports this, because their /boot is the OS partition, and this is btrfs, and it successfully boots...but I'm not sure if ours does, and if os-prober will need extra work to support this, or if our grub would benefit from the hypothetical SUSE patch. Just now, to be thorough, I checked to see if anything in Debian might be recommending set-default, or actually executing it, so I did a codesearch: https://sources.debian.org/src/systemd/247.3-1/docs/DISCOVERABLE_PARTITIONS.md/?hl=176#L176 * Recommends using it https://sources.debian.org/src/btrbk/0.27.1-1.1/doc/FAQ.md/?hl=144#L144 * Lists it as an equal option to the subvol=@foo method Luckily we don't have anything unexpected in codesearch that executes set-default. Cheers, Nicholas signature.asc Description: PGP signature
Bug#985481: debootstrap: Detection of docker container is broken with cgroup v2
Attention LXC Team: Does a functioning /sys always exist in an LXC
container, or is it absent/disabled in some configurations?
Hi Arnaud,
Reply follows inline.
Arnaud Rebillout writes:
> Package: debootstrap
> Version: 1.0.123
> Severity: normal
> Tags: patch
> User: de...@kali.org
> Usertags: origin-kali
>
> Dear Maintainer,
>
> The code that is meant to detect if debootstrap is running from within a
> docker container is broken with cgroup v2. Talking about this particular
> function and line in the file `functions`:
>
I agree that Bullseye should have working LXC with cgroups v2, since (as
far as I know) we have new enough packages from upstream to support
it :-) Thank you for your interest and motivation to fix this!
> detect_container () {
> [...]
> elif grep -qs '[[:space:]]/docker/.*/sys/fs/cgroup'
> /proc/1/mountinfo; then
> CONTAINER="docker"
>
> This code only works for cgroup v1.
>
> After some research, and also after looking into the code of
> systemd-detect-virt, it seems that the right way to detect a docker
> container these days is to check for the file '/.dockerenv'.
>
> Hence I'm proposing this patch:
> https://salsa.debian.org/installer-team/debootstrap/-/merge_requests/52
>
I'm not sure that systemd-detect-virt and your patch are
forward-compatible in light of
Originally, ".dockerenv" was for transmitting the environment
variables of the container across the container boundary -- I would
not recommend relying on its existence either (IIRC, that code
you've linked to is the only reason it still exists). There's
likely something incriminatory inside /sys/fs/cgroup, but I haven't
checked recently.
https://github.com/moby/moby/issues/18355#issuecomment-220484748
This makes it sounds like ".dockerenv" may be deprecated and later
removed. It seems reasonable to contact Debian's systemd maintainer[s]
about this probable future issue, because if checking for ".dockerenv"
is robust enough for Bullseye's systemd, then it might be robust enough
for debootstrap. That said, I still wonder if this method could pose a
problem when using debootstrap, lxc, and docker from future
bullseye-backports, if ".dockerenv" support is removed sometime during
Bullseye's life-cycle.
Cgroup v2 is also mounted at /sys/fs/cgroup, so I wonder if the original
check should be rewritten to check for something under this path instead
of mountinfo? Also, using this /sys/fs/cgroup method, I'm not sure if
it's better debootstrap style to express the OR logical operator in the
regex or a shell "||" (ie: seems to be needed because the tree under
/sys/fs/cgroup is different between v1 and v2).
Thanks again!
Regards,
Nicholas
signature.asc
Description: PGP signature
Bug#985481: debootstrap: Detection of docker container is broken with cgroup v2
Control: affects -1 release-notes Hi Arnaud! Adding src:docker.io maintainers and Shengjing Zhu (recent uploader) to CC list. Arnaud Rebillout writes: > Hello Nicholas! Thanks for your feedback here, see replies below. > You're welcome :-) > On Sun, 11 Apr 2021 11:51:20 -0400 Nicholas D Steeves > wrote: > > > I'm not sure that systemd-detect-virt and your patch are > > forward-compatible in light of [snip] > > This makes it sounds like ".dockerenv" may be deprecated and later > > removed. > > That's a good point, but it's also a 5 years old comment, and the > .dockerenv file is still present these days. > > I would think that if Docker plans to remove it, they will issue a more > formal deprecation warning that will give us enough time to fix things > on our side. Also the fact that systemd checks for this file gives me > more confidence that it's not just me doing something fancy here: it > seems that this is the "de facto" solution to detect docker containers. > > FWIW, it's also the most common solution on Q&A sites like > stackoverflow. Other people do that, because there is no better solution > provided apparently. Unless I missed it. > Yes, I agree; It appears to be the defacto solution, and might very well be the only solution for Bullseye in the sense that "perfect is the enemy of the good", ie: that it's better to solve this issue in a non-future compatible way to solve a bonafide issue in Bullseye; Later, a future alternative to /.dockerenv can be documented in Debian.NEWS and/or release-notes for Bookworm. > > Cgroup v2 is also mounted at /sys/fs/cgroup, so I wonder if the original > > check should be rewritten to check for something under this path instead > > of mountinfo? Also, using this /sys/fs/cgroup method, I'm not sure if > > it's better debootstrap style to express the OR logical operator in the > > regex or a shell "||" (ie: seems to be needed because the tree under > > /sys/fs/cgroup is different between v1 and v2). > > I just had a quick look in /sys/fs/cgroup from within a container. > Nothing obvious stands out, there's no file named docker, and nothing in > the content of those files mentions docker. I'll attach the output below. > > I will CC Tianon, as he was the author of the comment mentioned above, > and he might know better, 5 years after :) > > In short, Tianon, if you're reading those lines, our question is: what > would be the right way to detect that we're running from within a docker > container, apart from checking for the existence of the file > `/.dockerenv` ??? > Thank you for this investigation! I was also unable to find an alternative is_running_in_docker cgroupv2 check using /sys/fs/cgroup. Hopefully one of the src:docker.io maintainers knows! I've also added "affects release-notes" (and filed separate release-notes bugs) to defend against a worst-case scenario where this bug isn't resolved in time. Regards, Nicholas signature.asc Description: PGP signature
Bug#1034095: mount options could be cause
Hi James, James Abernathy writes: > I reran an install but this time when I remounted the @ and @home > subvolumes I only used the default, compress=zstd, and subvol= options. > Thank you. > This time it worked. After booting successfully, I edited fstab to add in > noatime and it still worked. At this point. it still could be the discard > or ssd options. > noatime is safe, and recommended, as noted in our wiki. compress=zstd is considered to be safe by many people--including Fedora. They're using "compress=zstd:1", and with different mount option than you're using. If "widely-tested" is an objective, then it may be worth keeping an eye on what config they use, and not introducing additional options that trigger corner cases. "ssd" see the Debian btrfs wiki on this topic (tldr: it's not necessary). > I'd look at what has changed in this area since Debian 11. Since Debian 11 (bullseye), space_cache v2 became the new btrfs-progs default, activated when a device is formatted, so it should be obvious why attempting to force space_cache v1 is wrong. At the same time, when upgrading from bullseye to bookworm, a volume made with space_cache v1 will still work (ie upgrades won't break), even with that mount option, because conversion to v2 does not yet happen automatically. discard=async isn't ready to use yet imho. Here is some upstream reading material on the topic: One year ago: https://www.spinics.net/lists/linux-btrfs/msg126838.html Last month: https://www.spinics.net/lists/linux-btrfs/msg133128.html Maybe it will be ready for linux-6.3? I don't know if the future fixes will be backported to 6.1, so I'm inclined to continue to advise against the use of discard=async for bullseye. Regards, Nicholas signature.asc Description: PGP signature
Bug#1012859: Info received (Syslog)
Dear Leslie, I'm sorry no one noticed your bug. Reply follows inline: Jeremy, thank you for following up on this bug! This brought the bug to my attention :) Leslie Rhorer writes: > On 7/13/2023 5:18 PM, Jeremy Davis wrote: >> [Just a random passer-by that might have an idea?] >> >> It looks to me like you are missing the firmware-realtek[1][2] >> (non-free) package?! > > No, I am not missing it. The package is broken in Bullseye. The > firmware is there, but does not work. It worked just fine in Buster, > but when I upgraded to Bullseye, the 10G NIC completely quit working. > It's been over a year, so I don't recall everything I did, but I spent > many, many hours trying to get the new firmware working, and many more > hours trying to extract the firmware from the oldstable package, and > then quite a few more hours trying to compile from source, but nothing > worked. I could not even get the source code to compile. Given your intention to upgrade from buster to bullseye, here is what you can try (please read to the end of this email first, because an alternative method is a better use of your time): 1a. Enable bullseye-backports (non-free), and 'apt install -t bullseye-backports firmware-linux firmware-misc-nonfree' which is currently 20230210-4~bpo11+1 2a. Reboot 3a. If both your NICs work, then it's a firmware bug. If this is the case, please report a bug against firmware-linux-nonfree 20210315-3. -- 1b. If the steps above are insufficient, 'apt install -t bullseye-backports linux-image-amd64' which is currently 6.1.20-2~bpo11+1 2b. Reboot. GRUB should automatically choose the backports kernel. 3b. Your NICs should work at this point. If they do, and you'd like to report a bug against the linux-image-amd64 version in bullseye, then please go ahead and do so, but please make sure that you've tested 5.10.178-3 before doing so:) > The bottom line is the firmware from the Buster non-free distro > works perfectly well, but noone has come forth with a fix for Bullseye, > and I have no reason to believe the firmware from Bookworm will work. > The NIC is an Asus PEB-10G/57811-1S 10GbE SFP+ Network Adapter which > employs a BCM 57811S controller. Maybe nobody knows that this specific hardware is broken? It may be that the Asus PEB-10G/57811-1S has some hardware quirks that Broadcom doesn't know about. In your original bug log you'll notice this snippet RTL8211E Gigabit Ethernet which is the Realtek one, Gibabit, RJ45 copper. I wonder if this one is a completely different NIC built into your motherboard. ie: the historically very buggy Realtek 8111E? Alternatively, if the 10GbE SFP+ PCIe adapter NIC uses a Realtek for gigabit PHY, then the nature of the bug could be that both both Realtek and Broadcom broke your NIC (either in the firmware on in the driver, or both). >> If you haven't already tried, I'd suggest that you try a clean >> bookworm install from ISO. FWIW bookworm now includes a separate >> "non-free-firmware" repo that is enabled by default. So the official >> installer should "just work". > I can try, but I really would not be well advised to do so until I > can get the dead system working again. Jeremy is right about how bookworm includes non-free-firmware by default, and also that the state of your hardware with bookworm should be tested first. The best use of your time will be to test with live media (USB or DVD). If you'd like to have a GUI for your test, please choose a variant you recognise and are comfortable with. The "standard" flavour is CLI only, which--alternatively--might be what you want (it's a smaller download ). https://cdimage.debian.org/debian-cd/current-live/amd64/iso-hybrid/ If the problem still exists in bookworm, then it needs to be fixed in bookworm before the fix can be backported to buster, and the live media is the fastest way to test this. Regards, Nicholas signature.asc Description: PGP signature
Bug#1017762: incompatible after "btrfs subvolume set-default ..."
Hello, Osamu Aoki writes: > It is great to have btrfs support with @rootfs. Thanks. I wish if it > is a bit more verbose on what it does in installer dialogue. This is > more important if we want to use existing btrfs with something like > @home-uid1000 in it ;-) > You are welcome, and yes, I agree, the current state is definitely incomplete! By the way, it's cool to hear from someone who is using user $HOME subvolumes :) > Anyway, I experienced an unsuccessful install to the existing btrfs > partition. I had @rootfs-broken-backup in it and I set "btrfs subvolume > set-default ..." to it. Don't ask me why I did. But this caused d-i > to stop installation without much error report. > Agreed, silent failure is a major problem. > EXPECTED BEHAVIOR: > > 1. Clearly mention the use of subvolume @rootfs in d-i dialogue. > (This is for both fsck or fsck-less install cases.) > The entirety of my reply supposes that you intended 's/fsck/mkfs/'. Yes, I agree this should be more verbose. > 2. Check "btrfs subvolume get-default " to be "ID 5 > (FS_TREE)". If not, > * stop installation with clear message or (reasonable?) Yes, and this will not break other installed operating systems that use set-default (eg SUSE, last I checked). Have you investigated whether Snapper rollbacks necessarily use set-default as well? If so, we will need to coordinate with Hedeki Yamane. > * set-default to fix this. (better?) > (This is for fsck-less install) > As you know, I am categorically against this approach. Within a Debian context, it seems to me that the most typical use-cases for a shared volume are: 1. Boot environments (like system restore checkpoints). This is a project that I used to have a lot of energy for, and why I joined Debian, but I have suffered significant demotivation for a variety of reasons. 2. A rootfs subvolume for stable, and another rootfs subvolume for sid, or possibly some other Debian-derivative distribution. 3. Please share what you use them for :) Why not just: * Always mount a btrfs volume with subvolid=5 during the subvolume creation step of a Debian installation to btrfs. The debootstrap and bootloader steps occur after remounting subvol=@rootfs, so the bootloader subvol autodetection generates the correct config for the new installation. If os-prober fails to find other OS on other bootable subvolumes then that is a bug in os-prober. To put this option another way: If you want to hide something from debian-installer, use LUKS, not a default-subvolume...that said, I seem to remember that the use of default subvolumes, plus multiple installed OS plays havoc with GRUB. * To support this policy in an optimal way may also suggest the creation of a new subvolid=5 view in the default install. By this I mean the creation of something like /btrfs-admin, which is subvolid=5 of the device that contains @rootfs, by default, in all installations. > 3. Check existance of @rootfs. If exists, >* stop installation with clear message or (simple) I believe this is the current best option, and maybe go one step farther in an advanced installation and emit a message that advanced users will use to prepare the volume. (ie something like "The Debian Installer currently requires @rootfs; however, this subvolume already exists. Please switch to a console and move the existing @rootfs out of the way) >* make a backup snapshot of @rootfs to some other name and > remove @rootfs to have clean start. (better) >(This is for fsck-less install) > The Debian Installer cannot guess what the correct action is, and it is wrong to automatically make an existing working installation unbootable. Last I checked we didn't even do that to Windows. Regards, Nicholas
Bug#757182: debian-installer: Please provide a warning about BTRFS
P.S. > Dimitri John Ledkov writes: > >> On 6 August 2014 03:46, Russell Coker wrote: snip >>> be that we should have a warning. BTRFS isn't at the stage where someone >>> with >>> little knowledge of it can just use it. To have it work reliably the >>> sysadmin >>> needs to know more about it than for other filesystems. >>> >> >> I disagree and the assessment here is unjust. I agree. That was supposed to be in the last email!
Bug#757182: debian-installer: Please provide a warning about BTRFS
Dimitri John Ledkov writes: > On 6 August 2014 03:46, Russell Coker wrote: >> Package: debian-installer >> Severity: normal >> >> http://www.spinics.net/lists/linux-btrfs/msg36461.html >> >> BTRFS has some issues that can cause system lockups, filesystem deadlocks >> that >> prevent writing to disk, and other problems. After some discussion on the >> BTRFS mailing list (see the above URL for the archive) the consensus seems to >> be that we should have a warning. BTRFS isn't at the stage where someone >> with >> little knowledge of it can just use it. To have it work reliably the >> sysadmin >> needs to know more about it than for other filesystems. >> > > I disagree and the assessment here is unjust. By default we offer > ext4, [ with lvm2 [ with cryptsetup LUKS ] ]. mdadm raid needs > additional setup. > For none of the above, we show any warnings. > In the manual partitioning, again ext4 is the default. To get to > BTRFS, one needs to change from ext4 to it, which imho there is a > sufficient amount of hoops to jump through. > I wouldn't want to loose ability to install on to btrfs, since > developers have need to have working installers with btrfs. > From UX perspective, users don't read warnings =) > When people ask me if they should use btrfs, or if btrfs is ready my > reply is usually "if you have to ask, you shouldn't use it. Instead > study and benchmark it to know for sure what you are getting into with > your workload." > > ext4 is Debian's and Ubuntu's default filesystem for upcoming releases. Nine years since this bug was filed, and three years since Fedora has been using btrfs by default, I think this bug can be closed. Any objections? Nicholas signature.asc Description: PGP signature
Bug#777578: initramfs-tools: update-initramfs fails with btrfs
Hi, jnqnfe writes: > On 11/02/2015 18:42, Ben Hutchings wrote: >> So the root (no pun intended) of the problem is that btrfs-tools was >> not installed. Ben. > > Ah ha, you're absolutely right, I assumed it was but it is indeed not > installed. Thanks for that. > > Yep, now it boots successfully :D Are you still able to reproduce the state where Debian is successfully installed to a btrfs rootfs, but where btrfs-progs is not installed? From what I can tell, that is the nature of this [by now most likely historical] bug. Regards, Nicholas signature.asc Description: PGP signature
Bug#1053678: partman-crypto: Requires separate /boot partition, even if not required
Jonathan Hettwer writes: > Package: partman-crypto > Version: 121 > Severity: normal > Tags: d-i > X-Debbugs-Cc: j24...@gmail.com > > Dear Maintainer, > > The `crypto_check_mountpoints` script prevents you from setting up an > encrypted root filesystem without an additional unencrypted /boot > filesystem. > While this may be a requirement for e.g. grub2, it is not > necessarily required when not using grub2 but using UKIs to build EFI > executables that can directly mount the encrypted root filesystem. > While UKIs aren't currently supported, I would still expect partman-crypto > to let me partition an encrypted root filesystem without separate /boot > filesystem, at least after having ignored the warnings or in combination > with the `nobootloader` udeb. Quick note: systemd-boot works with kernel images + initramfs, without UKI. After the systemd-boot menu, the user is prompted for the master LUKS password, as usual, and I use the derived key script to then unlocks a couple LUKS volumes. No LVM, no /boot. It seems to work well, but yeah, it's not possible to do this with fresh install (I manually migrated an old installation to new hardware). Regards, Nicholas signature.asc Description: PGP signature
Re: Bug#1070085: Enhancing default home folder permissions for improved privacy
Chris Hofstaedtler writes: > On Tue, Apr 30, 2024 at 12:16:23AM +0200, Håvard F. Aasen wrote: >> Could default home folder permissions lean towards greater privacy, >> while administrators can adjust permissions to be less strict if >> necessary? Isn't it worth noting that normal users can adjust the permissions of their own home directory? >> What I would like is to have 'HOME_MODE' [4] uncommented. >> [4] >> https://salsa.debian.org/debian/shadow/-/blob/master/debian/login.defs?ref_type=heads#L156 > > I vaguely remember the installer has an option for this, possibly > only on expert mode. If so, it would be good if the installer > default would change (or so). > > If the installer has no option for this, please reassign to > src:shadow, so we can think about it more. For what it's worth, the last time this topic came up (sometime in the last eight or nine years) the consensus was that enhancing default home folder permissions for improved privacy would break sites that depend on default home folder permissions that facilitate collaboration. I don't have a preference either way by the way. Best, Nicholas signature.asc Description: PGP signature
Bug#646699: btrfs: Installer offers BTRFS an optional filesystem
Hi, On Thu, Oct 27, 2011 at 10:24:05AM +0200, Gaudenz Steinlin wrote: > On Wed, 26 Oct 2011 23:21:33 +0530, Christian PERRIER > wrote: > > > > > > > > Maarten writes: > > > > > > > Package: btrfs > > > > Severity: critical > > > > Justification: causes serious data loss > > > > > > > > BTRFS shouldn't be offert as a option filesystem in the debian > > > > installer. > > > > It is unsafe to use. Quallity is poor. No recovery possible on > > > > filesystem errors. (The btrfs driver will even crash on a filesystem > > > > error) > > > > The provided tool btrfsck doesn't actually do anything. > > > > There doesn't seem to be any progres on a working btrfsck. > > > > > > > > Atleased users should be warned to not use it, unless they don't > > > > care about dataloss > > Do you have any real world cases to support these claims using a recent > kernel version (at least the version currently in testing). > > > > > > > There is no btrfs package in Debian, thus, this report did not reach any > > > developers. Furthermore, since it is the installer that is allegedly at > > > fault, it should be filed against the debian-installer package. > > > > > > I went ahead and reassigned it there. > > > > > > Well, if btrfs is in such a bad shape, then partman-btrfs should be > > made optional so that only those people who really want it will have > > it as an option. > > > > I don't think that dropping the package entirely is the best > > option. But making it less "visible" in D-I is probably good if I > > believe in the above claims (I have no idea about this to be true or > > not). > > With my own experience with BTRFS I can not support the above claims. In > several tests and while running my laptop with BTRFS I never saw any > data loss. While it's true that there is no external filesystem checker > (aka "btrfsck") as this is a journaling filesystem such a tool is much > less needed than for a non journaling filesystem. Also a btrfsck tool > is in the works, but it's unclear when it will be released. > > The main reason why I would not recommend btrfs on Debian for / is it's very > poor fsync performance which makes apt runs a pain in the ass if you > don't use "eatmydata" which disables fsync. But that's a performance and > not a corectness issue. > > BTRFS might be unreliable with the current stable kernel. I did not test > this. So if someone really belives that BTRFS should be less visible, > just do that for the stable installer (if thats possible wrt stable > update policies). > I've been using btrfs since mid 2014, and it has been problem-free since sometime in 2017 with linux-4.4.x. Fsync performance seems to have improved (still not great), but IO latency under load for an aged filesystem is still sometimes awful. Tracking the latest stable upstream kernel (incl. testing/sid) rather than LTS kernel is still risky and presents increased exposure to new bugs, but this is documented on our wiki. Imho this bug is no longer relevant and should be closed. Regards, Nicholas signature.asc Description: PGP signature
Bug#964818: Enable basic subvolume management for rootfs
Package: partman-btrfs Version: 50 Severity: normal Control: patch -1 Control: block 840248 by -1 https://salsa.debian.org/installer-team/partman-btrfs/-/merge_requests/1 I have tested the proposed changes and confirmed that they produce the desired change. Briefly, the problem: Installing Debian directly to subvolid 5 rather than to a "rootfs" (like Fedora) or "@" (like SUSE and Ubuntu) subvolume causes the following problems: 1. It makes it unreasonably difficult to move to a flat subvolume structure 2. It means that any snapshots of the rootfs creates a nested subvolume rather than flat subvolume structure. 3. It blocks development of "Boot Environments" (consult the web for how these are used in Solaris and FreeBSD and how these would be useful in Debian). 4. It blocks the ability to stage a system upgrade in a @rootfs-sid or @rootfs-stable+1 subvolume and then pivot into this during a reboot; this is an example of how "boot environments" are useful. There is precedent in Ubuntu for using a non-configurable method (they additionally add a "@home" subvolume), and the Ubuntu installer does not offer user configurability of subvolumes during installation. My plan is thus: 1. After we have installation to subvolumes, add subvolume listing support to the rescue cd. This has the side-effect of being able to test "boot environments" from the rescue cd. 2. Activate boot environment support via grub-btrfs (#941627). 3. Long-term: add debian-installer support for user-configurable subvolume layout like Fedora and OpenSUSE have. Ideally I'd like to work on this as part of a btrfs-enablement team Thanks, Nicholas CCing Cyril for Kibi ACK :-)
Bug#964818: Enable basic subvolume management for rootfs
Hi Cyril! On Sat, Jul 11, 2020 at 11:15:03PM +0200, Cyril Brulebois wrote: > Hi Nicholas, > > Nicholas D Steeves (2020-07-10): > > My plan is thus: > > > > 1. After we have installation to subvolumes, add subvolume listing support > > to the rescue cd. This has the side-effect of being able to test "boot > > environments" from the rescue cd. > > 2. Activate boot environment support via grub-btrfs (#941627). > > 3. Long-term: add debian-installer support for user-configurable subvolume > > layout like Fedora and OpenSUSE have. Ideally I'd like to work on this as > > part of a btrfs-enablement team > > > > Thanks, > > Nicholas > > > > CCing Cyril for Kibi ACK :-) > > Please don't block on me. I'd assume you have much more experience with > btrfs than I do, and I'm not really concerned about possible breakages > with that particular filesystem (upon checking, it appears it doesn't > even appear in the installation guide for some reason). > Sorry for the delay replying, this was a case of "gmail silently ate/bounced your email" :-( Re: installation-guide: If I remember correctly someone recommended striking mention of btrfs from the installation guide because they believed the fs was RC buggy. Given that Fedora 33 is allegedly going to default to btrfs (with subvolumes for rootfs and home), for real this time, I think it's clear that this is no longer a just assessment. Thank you, I appreciate your vote of confidence! :-) The most significant potential issue I'm aware of is if Debian begins to default to using a persistent journald. Upstream journald defaults to using "chattr +C" (eg: nocow, no checksums, no protection), and there was a historical bug where this combination caused problems, but given the Fedora news I think it's fair to suppose that this potential issue has been resolved upstream. I'll wait a month to Andrew Hayzen a chance to test, then retest, then merge. Oh, and there is a fourth long-long-term objective: enabling advanced btrfs features such as btrfs-native raid profiles and compression in the installer. This one is without a shadow of a doubt post-bullseye, because util-linux and coreutils aren't yet sufficiently aware of btrfs' freespace accounting peculiarities when using these features (said another way, btrfs doesn't export expected info). I'm not blaming one group or the other, just saying I don't believe these features are not yet "Debian stable" quality in terms of user experience. I expect it will be ready for bookworm! Ideally I hope Adam Borowski, Hideki Yamane, and maybe Andrew Hayzen help found the future btrfs-enablement team :-) Cheers, Nicholas signature.asc Description: PGP signature
Re: debian-installer for mac
"Andrew M.A. Cater" writes: > As mentioned on debian-user: the debian-11.1.0-amd64-netinst.iso or the > debian-11.1.0-amd-DVD-1.iso are the appropriate ones to use. > > The debian-mac images are for a couple of specific models from 2008/2009 > which had problems recognising El Torito images. > Oh yeah! I remember having to ask about this for a 2014 installation I did on a 2012 MBP. Has it not been documented yet? Seems like the website and installation guide could benefit from bug reports requesting documentation of this fact. I believe it's worth the effort, because recent macOS versions don't work well with older Apple hardware. Regards, Nicholas signature.asc Description: PGP signature
Bug#988472: Bug#604839: [installation-guide] Planned overhaul of chapter 4.3 "Preparing Files for USB Memory Stick Booting"
Holger Wansing writes: > Hi, > > David wrote (Sat, 9 Oct 2021 21:56:24 +1100): >> I see that the suggestion to use 'cat' comes >> from #604839. >> >> Yes, 'cat' will "work", however I feel there is no >> good reason to use 'cat' there. >> >> Because the purpose of 'cat' is for concatenating >> multiple files, and it also requires a shell redirection >> from stdout. Both are unnecessary here. >> >> I suggest this command should be used: >> # cp /usr/lib/syslinux/mbr.bin /dev/sdX > Wow that's a new capability :-) IIRC cp couldn't historically write directly to block devices, and DEST had to be either a target file or directory. It makes me wonder if install(1) has also gained spooky new capabilities :-p > The documentation in the syslinux package also has > > A simple MBR, roughly on par with the one installed by DOS (but > unencumbered), is included in the SYSLINUX distribution. To install > it under Linux, simply type: > > cat mbr.bin > /dev/XXX > > ... where /dev/XXX is the device you wish to install it on. > > so I guess there is some good reason to do it this way. > Holger, do you think this could be from the days of cat bootloader.bin kernel.image userspace.bin > /block/device ? AFAICT these semantics aren't totally totally anachronistic, because of systemd-boot's "unified image" or "unified kernel image" support...but that said, I'm not sure if this is an example of simple appended/concatenated images. Best, Nicholas signature.asc Description: PGP signature
Bug#1000239: Rescue system won't find root partition, but insists on /usr
Control: tag -1 + moreinfo Hi Steve, I've written my reply assuming this isn't a usermerged system, because if it is one, then I wonder if this is a usrmerge-related bug. Ie: I wonder if a splitting / and /usr onto different partitions is never supported on usrmerged systems. Alternatively, maybe the rescue mode isn't usrmerge-ready? Hi Tom, TomK writes: > Package: debian-installer > Version: 20210731+deb11u1_amd64 > > Errors, "No suitable shell found on /dev/sda1" > [snip] > I was attempting to reinstall grub, to get the system to boot. I have also > used the rescue syatem to change a lost password. It appears this bug may > apply only to a gpt partition table and a fat32 boot partition (efi?). > > I've had this same problem with 2 systems, a lappy and a desktop. I can > always work around it. I don't understand what could be going on, for lack of > experience. I've only been a Debian user since Woody was in testing. > > It's easy to reproduce. Do an expert install with defaults, but partition > with gpt. Boot the system with the install media, launch a rescue shell, and > try to open a root shell. An incorrect device should be identified as '/'. > Perhaps this occurs only when /usr is on a different partition that '/'. > Given the information you provided, this is what I suspect may be occurring: 1. You have a EFI system 2. On an EFI system, /dev/sda1 is usually the ESP (EFI System Partition) 3. The ESP is not the rootfs, and it won't contain /bin/init, /bin/sh, etc. 4. Therefore choosing /dev/sda1 as the rootfs will never boot 5. due to 'Errors, "No suitable shell found on /dev/sda1"' If you're dual booting with another OS, the rootfs might be on sda2. Would you please share note the error[s] when selecting sda2, sda3, or whichever partition ? Other relevant info is if LUKS (encrypted partition) or an configuration using LVM was selected. Optional: Feel free to unset the "moreinfo" tag when providing this information (see the first line of this email for the model, and change the "+" to a "-" in your reply). P.S. It's likely that I won't be able to follow-up on this bug until after the holidays, Regards, Nicholas signature.asc Description: PGP signature
Bug#1000239: Rescue system won't find root partition, but insists on /usr
Control: severity -1 serious Control: tags = confirmed CCing the release team, and CTTE because I don't know who else is tracking issues related to the usrmerge effort. I've consciously chosen not to pour gasoline on the flame war by CCing anyone else (nor will I contact anyone else about the existence of this bug). Steps I used to try to reproduce: 1. Downloaded debian-testing-amd64-netinst.iso 2021-12-03 16:21 408M 2. Installed to EFI-enabled qemu eg: kvm -bios /usr/share/ovmf/bios.bin -m 2G \ -hda debian-separate-usr-sda.raw \ -cdrom debian-testing-amd64-netinst.iso 3. Guided partitioning with separate /home, then changed the mount point to /usr. Partition layout is: sda1: ESP fat32 sda2: /ext4 sda3: swap swap sda4: /usr ext4 4. Selected only "Standard System Utilities" 5. Rebooted Result: SUCCESS Then, to test the rescue functionality: 1. I discovered qemu+OVMF boot order is broken without changing the command to: kvm -L /usr/share/ovmf/ -m 2G -boot menu=on \ -hda /scratch/debian-separate-usr-sda.raw \ -cdrom /scratch/debian-testing-amd64-netinst.iso 2. Selected sda2 3. Yes, mount /boot/efi 4. Execute a shell in /dev/sda2 5. No usable shell was found on your root file system (/dev/sda2) 6. Changed virtual terminal 7. cd /target && ls bin ls: bin: No such file or directory Result: FAILED Conclusion: This is a usrmerged system, and the rescue system does not support usermerged systems. The options are, as I see them, ranked from least to most work-hours: 1. Debian isn't yet ready for usrmerge. Revert to normal system installation. 2. Reassign to src:rescue, and fix the rescue system. a) before chrooting, test for the presence of /target/bin/sh b) if /bin/sh is not found, either emit error to the user and present a dialogue for selecting /usr partition, or c) parse /target/etc/fstab, and attempt to mount other partitions d) b & c will be difficult to implement when attempting to accommodate the heterogeneity of possible MD, LUKS, and LVM layouts, not to mention the complications introduced by the possibility of a user-configured btrfs subvolume name "@usr" on any valid device. Fstab parsing might make the btrfs case easier with: i) Display a dialogue selector if a btrfs partition is detected The dialogue will list all detected subvolumes ii) If the user cannot find a subvolume for "@usr", then iii) Allow the user to escape to the partition selection screen, and iv) Unmount the partition 3. Disallow configuring of a mount for "/usr", and *Prominently* declare that Debian no longer supports separating /usr from /, declare this in *many places*, and reply to bugs on this topic for many years. I put this one last because I believe the cost to work-hours is unbounded, and because I believe there may be a negative social cost associated with this action. Also, if Fedora/RHEL/SUSE/Ubuntu support a separate /usr partition, then this action could make Debian look inferior. Regards, Nicholas signature.asc Description: PGP signature
Bug#1000239: Rescue system won't find root partition, but insists on /usr
Thank you for the discussion! I found it educational and motivating, and hope that everyone found the same. It's really refreshing to see this. Sorry for the naivety in my analysis and in the delay replying; I've been drained/busy, but I read each of your emails carefully, wanted to reply to each of them, and chose not to CC everyone because that would be noisy. Steve, thank you for the quick fix :-) Reply follows inline: Steve McIntyre writes: > On Sat, Dec 04, 2021 at 10:42:29PM +, Steve McIntyre wrote: >>On Sat, Dec 04, 2021 at 11:37:28PM +0100, Pascal Hambourg wrote: >>>Le 03/12/2021 à 22:08, Nicholas D Steeves a écrit : >>>> >>>> >>>> c) parse /target/etc/fstab, and attempt to mount other partitions >>> >>>The rescue system already offers to do it for separate /boot and /boot/efi, >>>so why couldn't it do for separate /usr too ? >> >>That's exactly what I'm about to do. > > In fact, it needed more work than that - the code chrooted into > /target and ran mount there. That didn't work for a separate /usr. But > I've refactored the code and made things work more cleanly inside d-i. > I had also attempted a fix at the same time, but my approach wasn't nearly as nice as yours. In particular, I appreciate how your fix produces insight into debian-installer/rescue function, architecture, and style, and I hope to say thank you more substantially with an eventual MR for btrfs subvol support :-) It will be much less ugly and "tacked on" as a result of studying your changes! Regards, Nicholas signature.asc Description: PGP signature
Bug#1018894: rescue-mode: mounts wrong btrfs subvolume
Pascal Hambourg writes:
> On 03/09/2022 at 06:32, Philip Hands wrote:
>> Ansgar writes:
>>
[snip]
>>>
>>> However, mounting the root filesystem failed: /target contained only a
>>> "@rootfs" subdirectory. So running a shell in the target fs failed.
>>> Manually mounting the filesystem with `-o subvol=@rootfs` worked.
>>>
>>> This was with Debian 11.4.
>
>> I just pushed 1.88 including this MR:
>>
>>https://salsa.debian.org/installer-team/rescue/-/merge_requests/1
>>
>> which seems like it ought to address the problem you're experiencing.
>
> If I understand correctly, the problem is in mounting a btrfs root
> filesystem. However the change in rescue 1.88 addresses only mounting
> extra filesystems such as /boot/efi or separate /usr.
>
> I guess that handling btrfs subvolumes or other root filesystem mount
> options would require changes in the root file system selection step. I
> do not see how it could be automated (rootflags are defined in the boot
> loader configuration which may be located anywhere).
This can be automated with limitations, or semiautomated (with a menu),
like LVM.
First, the automated case: Currently DI statically sets the rootfs to be
"subvolume=@rootfs", which is a value unique to Debian. The simple
thing to do is to detect that a device has been formatted to btrfs
before mounting, and then try to mount "-o subvol=@rootfs" as Ansgar
suggested. One step more flexible is to have a Rescue policy of
something like only supporting read-write subvolumes with a name that
matches '.*\@rootfs.*', and that exist in the / of the volume. Ie:
maxdepth=1.
To do more, we need a mechanism to detect [bootable] subvolumes and then
present a menu of candidates while excluding read-only snapshot and
non-bootable and irrelevant subvolumes. Ie: Rather than try, and bail on
failure, create a whitelist and present that to the user. A user with
poor subvolume hygiene may have thousands of read-only snapshots of
their rootfs, and it is not useful to overflow the terminal with a
selection menu of these almost entirely irrelevant items.
Would (/bin/mount AND /etc/fstab) be useful for this? Ie: This may be
enough for Pascal's work to find everything else that is needed (other
than the subvolume=foo specification). If the sysadmin/user botched
their fstab then this will fail; however, the discussion I had on
debian-boot (possibly on IRC) indicates that this is a non-issue,
because "rescue" is only expected to reinstall the bootloader. What do
you think?
ISTM that the rescue media should test for an actual Debian installation
with something like:
awk '{print $1}' < /MOUNTPOINT/etc/issue
or perhaps
grep 'ID\=debian' < /etc/os-release
as well as to test for /bin/mount, and /etc/fstab. These days I'm not
sure what the minimally complete criteria would be...
At any rate, regarding the btrfs subvolume detection (ie: lvdisplay
equivalent, and no, not a PITA at all, this is the easy part.), there
are three approaches:
1. Use the "btrfs" tool, and parse its output
* I can do this, but will need help/review for the DI idioms
! Prerequisite: mount the FS someplace temporary, or teach rescue how
to remount /target. Bind-mounting rootfs with *not* work.
2. Use `find` various btrfs-specific-type detection functions
* I wrote these long ago, because I saw a future need for them
* probably limit search depth to not waste time!
! Prerequisite: mount the FS someplace temporary,or teach rescue how
to remount /target. Bind-mounting rootfs will *not* work.
3. Write something that uses python-btrfs
* Someone else can do this if it looks like it would be the most
robust approach
For now, I also recommend declaring that a specific set and type of
layouts are supported. When DI gains subvolume layout support, it may
be wise for it to exclusively support a particular schema. The
definition of supported configurations could of course be expanded in
the future.
The hard part: For example, when mounted without "-o subvol=@rootfs"
(ie: the "top-down admin view that I recommend using /btrfs-admin for on
a running system), a volume mounted at /MOUNTPOINT may contain the
following completely valid and bootable subvolumes/environments:
@rootfs # which is presumably Debian stable with a default installation
@rootfs_buster # a RO snapshot made before upgrading to bullseye?
* This one is probably still bootable, with warnings
@rootfs_sid # custom bootable subvolume probably created with
debootstrap
* Like a schroot with installed kernel...actually, with a separate
/boot, a subvolume dedicated to a sid schroot may actually be bootable.
@# Ubuntu
rootfs # Fedora
Given /\, the menu presented by Rescue should look something like like:
||
\\sda1
||
1) @rootfs
2) @rootfs_sid
Select a subvolume (and press Return/Enter):
But a user may have customised things to look more like the
request for review of proposed changes
Hi, I've pushed minimal changes to the git branch "proposed" of partman-btrfs. Would someone please take a look at them and let me know if they look good? I'm sure I'm forgetting something... That said, the solution I'm proposing doesn't require translation ;-) Kind regards, Nicholas signature.asc Description: Digital signature
Bug#840248: debian-installer: Add btrfs subvolume setting for snapshot
On 30 October 2016 at 07:21, Philipp Kern wrote: > On 10/11/2016 11:40 PM, Nicholas D Steeves wrote: >> So far, the plan is to default to simple @rootfs and @home subvolumes, >> because I've read that backing up OpenSUSE systems is cumbersome with >> all of those subvolumes, and also because of the KISS principle; [...] > > FWIW, given that I just encountered this myself: rescue(-mode) will > need > a fix in this case because by default it mounts the top-level, which > means that the actual chroot is one level down. Although I guess > setting > the default subvolume id to the one of whatever you call @rootfs > should > also fix this. Hi Philip, So sorry for the delay. Life stuff that my plan couldn't accommodate for :-( Which rescue mode, and where? Please tell me so I can fix it! From what I've read, setting a default subvolid != 5 was explored by other distributions, and abandoned. As I hadn't received any feedback from debian-boot@, and it seemed like development has shifted to providing translations only, I thought that a minimal change that didn't require translation would be more appropriate. From this proposed default configuration, in single user mode, rootfs' partition can be mounted without subvol=@subvolume somewhere like /btrfs-admin and subvols can be created as children of subvolid 5 and peers to @rootfs (eg: @var), then you replicate the data from /btrfs-admin/@rootfs/var, and finally edit fstab and mount the new /var subvolume, go multiuser or reboot. I would very much appreciate it if you would take a look at it. I understand it needs to be rebased ;-) https://anonscm.debian.org/cgit/d-i/partman-btrfs.git/log/?h=proposed Concerning the naming of the rootfs subvol, is there something you would prefer? I've since learned that LXC's btrfs backend follows the Fedora/CentOS/RedHat convention of a simple "rootfs" albeit by nesting it in whatever subvolume /var/lib/lxc belongs to. I plan to keep working on this even if it's now too late for Stretch's initial release! Cheers, Nicholas signature.asc Description: Digital signature
Bug#851526: Please provide command-line option to disable ipv6
On Sun, Jan 15, 2017 at 02:43:45PM -0800, Josh Triplett wrote: > Package: netcfg > Severity: wishlist > > netcfg provides an option to completely disable all automatic > configuration, but no option to disable ipv6 autoconfig (SLAAC) while > leaving DHCP enabled. Putting ipv6.disable=1 on the kernel command line > will cause netcfg to realize the network has no ipv6, but only after > waiting a similar timeout for a link-local address, defeating the > purpose. > > Please either detect disabled ipv6 and skip those steps, or provide a > command-line option to disable ipv6 in netcfg. > > (Context: repeatedly testing preseed installs in a virtual machine, and > I don't want to keep waiting on ipv6 autoconfig timing out.) > From what I've read, ipv6.disable=1 hasn't been sufficient for quite some time, and one requires something like the following in /etc/sysctl.d/: 00-disable-ipv6.conf: net.ipv6.conf.all.disable_ipv6 = 1 net.ipv6.conf.default.disable_ipv6 = 1 net.ipv6.conf.lo.disable_ipv6 = 1 Cheers, Nicholas signature.asc Description: Digital signature
Bug#840248: debian-installer: Add btrfs subvolume setting for snapshot
Hi Philipp, Thank you for the clarification, and sorry for my tardy reply. On Wed, Jan 04, 2017 at 12:04:09AM +0100, Philipp Kern wrote: > On 12/19/2016 05:49 AM, Nicholas D Steeves wrote: > > Which rescue mode, and where? Please tell me so I can fix it! From > > what I've read, setting a default subvolid != 5 was explored by other > > distributions, and abandoned. > > rescue-mode is in [0]. That presents you with a menu where you can > select local root filesystems. That should somehow DTRT instead of > mounting the top-level btrfs filesystem with the root filesystem being > below. I suppose it'd be also ok to mount it as-is, as long as the shell > is spawned in the right place. (Although that might be surprising.) > > The mode is triggered by passing "rescue/enable=true" on the kernel > command-line. d-i ships with boot menu items that do this. > > Kind regards > Philipp Kern > > [0] https://anonscm.debian.org/cgit/d-i/rescue.git/tree/ > Oh, there! I had already checked that out in debian-installer/packages/rescue. :-) From what I gather, DTRT looks something like one of the following: 1. Use existing choose partition menu * select partition menu * test if selected partition is a btrfs volume - if there are no subvolumes, use present behaviour * if subvolumes exist - install btrfs-progs udeb - use btrfs subvol list to read subvols - present a menu How is this currently handled for LVM? There is very little code in "rescue" itself, and I haven't yet managed to figure out how everything fits together. 2. Alternatively, duplicate the existing LVM code, then modify it for btrfs. If you could point me to whatever 'rescue' ties into for LVM support I would be very grateful! From what I've gathered so far, "rescue" dependency on the btrfs application is provided by the btrfs-progs udeb and not through initramfs Cheers, Nicholas signature.asc Description: Digital signature
Bug#840248: debian-installer: Add btrfs subvolume setting for snapshot
Ah...the logic is in debian/rescue-mode.postinst; I had assumed it would be elsewhere. I'll take some time to study this thoroughly, and to do a VM install and rescue to see how the LVM case works. If you know if it's closer to (1) or (2) in my last email. Is Feb 5th (Full freeze) the final deadline for this work? Deadline being, it needs to have been submitted to ftp-masters before then. Cheers, Nicholas signature.asc Description: Digital signature
Re: Bug#861263: debian-installer: zfs support
On 5 May 2017 at 15:27, Sam Kuper wrote: > On 05/05/2017, Ben Hutchings wrote: >>On Fri, 2017-05-05 at 19:50 +0100, Sam Kuper wrote: > >>> 2. Add ZFS to a Debian Installer that is not the *default* Debian >>> Installer. Does Debian distribute such an installer, to which the >>> facility to compile and run ZFS could be added? >> >> Yes, there is already an (officially unofficial) installer that >> includes non-free firmware. > > Thanks for the information. Can the non-free aspect of that installer > be disabled by the user during installation? If not, then it would be > no use to anyone I know who would be interested in running ZFS under > Debian. That is because a key reason to use Debian in preference to > other distros is that Debian's blob-free kernel and DFSG-compliant > main and contrib repositories make it easy to avoid installing > non-free software. If a person doesn't mind the risk of installing > non-free firmware then they may as well just skip Debian and use > Ubuntu or FreeBSD instead, which ship with ZFS in the installer by > default. > I would recommend the second of the following options: 1. Install using the non-free media with "Advanced options" -> "Expert install" 2. Install using the non-free media, then cleanup #!/bin/sh apt-get install aptitude sed -i 's/ non-free//' /etc/apt/sources.list apt-get update aptitude search ?obsolete -F '%p' --disable-columns \ | apt-get purge ...and the non-free packages should be gone. And if you don't want aptitude you can purge that too. It's faster than an "Advanced options" -> "Expert install", where I believe it is also possible to install a system which pulls uniquely from main and contrib. There are a more reasons to use Debian than just default package selection... eg: updates policy, minimal sysadmin headaches, smooth upgrades even from major version to major version, very high quality packaging standards, etc. These are pragmatic reasons to prefer Debian. In my opinion embracing CDDL constitutes ideological compromise, because it forbids "mixing" with with GPL--the most socioally conscious and not neoliberal license. And if Debian isn't 'pure' enough, there are always these: https://www.gnu.org/distros/free-distros.html Cheers, Nicholas
Debcamp 2017?
Hello, I'm wondering if Debian Boot will have a Debcamp this year. My small project would be 1) making a couple of improvements to Rescue Mode 2) adding btrfs subvolume support to the installer 2 is currently blocked by 1. My plan for 2 is to model it off of existing LVM support. Cheers, Nicholas signature.asc Description: Digital signature
WRT Bug #818687
Hi, I'm working on a bug (#818687) that pre-dates Jessie's release, and I'd like to get it resolve it before Stretch goes into freeze. En résumé it's a rename of btrfs-tools to btrfs-progs, and this affects debian-installer. I imagine that a patch can be generated with a simple substitution of btrfs-progs for btrfs-tools. Please take a look at it and let me know what I can do to help. Sincerely, Nicholas
Re: WRT Bug #818687
Please note that the discussion seems to have shifted to Bug#818687 Thank you, Nicholas
Re: WRT Bug #818687
ack! I mean, the original bug is: #780081 Sorry for the noise, Nicholas
partman-btrfs, RFS involved with two other bugs
I am working on problems associated with bug #780081, where it was planned to have a fix staged in experimental after Jessie was released. Associated bugs: #780081: btrfs-tools: rename package to btrfs-progs #818687: RFS: btrfs-progs/4.4.1-1.1 [NMU] #820479: Acknowledgement (accommodate rename of btrfs-tools to btrfs-progs (patch included) To access further information about this package, please visit the following URL: http://mentors.debian.net/package/partman-btrfs Alternatively, one can download the package with dget using this command: dget -x http://mentors.debian.net/debian/pool/main/p/partman-btrfs/partman-btrfs_20+nmu1.dsc On #debian-boot jcristau said he prefered "a git tree [which] would be easier than a source package tbh". I don't have alioth permissions to push to partman-btrfs or debian-cd so I've published it here. Please let me know if you'd prefer a diff. The repo can be found here: https://github.com/sten0/partman-btrfs.git I'd like this change to go through soon, so plan to file an RFS bug for partman-btrfs if I don't hear back soon. It took a lot of work to learn enough to get this done, but it was fun. :-) Kind regards, Nicholas
Bug#820483: accommodate rename of btrfs-tools to btrfs-progs
Package: partman-btrfs Version: 20 I am working on problems associated with bug #780081, where it was planned to have a fix staged in experimental after Jessie was released. To access further information about this package, please visit the following URL: http://mentors.debian.net/package/partman-btrfs Alternatively, one can download the package with dget using this command: dget -x http://mentors.debian.net/debian/pool/main/p/partman-btrfs/partman-btrfs_20+nmu1.dsc On #debian-boot jcristau said he prefered "a git tree [which] would be easier than a source package tbh". I don't have alioth permissions to push to partman-btrfs or debian-cd so I've published it here. Please let me know if you'd prefer a diff. The repo can be found here: https://github.com/sten0/partman-btrfs.git Thanks, Nicholas
Bug#820483: upgrade priority
control: severity -1 important Hi, I'm raising the severity here on Gianfranco Costamagna's recommendation. 'hope I'm doing this right for a project that has four bug reports! Best regards, Nicholas
Bug#820483: accommodate rename of btrfs-tools to btrfs-progs
On 9 April 2016 at 03:04, Christian PERRIER wrote: > Changes seem to be OK, but I guess that we can't upload before the > btrfs-tools package hasn't been renamed, isn't it? > > From what I see, we need both btrfs-progs and the udeb in the archive > before partman-btrfs is uploaded, or it will: > - FTBFS > - fail when running > > I'm not sure if an upload to experimental is really useful, as the > installer and its components doesn't make good use of experimental. > > I'd say "go ahead for the package rename and we'll apply the changes > to partman-btrfs as soon as btrfs-progs is in unstable". Other > advices? I can't think of anything else, and that sounds good to me. I've updated my git tree and the packages I uploaded to debian-mentors to release for unstable instead of experimental. Also, I'm not 100% sure how to correctly manage bug dependencies and priorities. The original btrfs-tools bug is #780081 and my RFS bug is #818687. Best regards, Nicholas
Re: Use newer debian installer with old distro
Hi Alan, On 22 April 2016 at 19:13, Alan Evans wrote: > Is there a way I can _easily_ get a new kernel into debian-installer. > Bearing in mind that I normally use on Fedora. So apt-get install > debian-installer, make etc, even in a chroot, is tedious. > > Please help, > -Alan My first instinct is to refer to you https://wiki.debian.org/Simple-CDD I'm also interested in this question, specifically because I'm considering maintaining a minimal installation cd that comes with both a jessie-backports kernel and btrfs-tools. My concern is ease of maintenance over time. Cheers, Nicholas
joining the team
Hi, I'd like to join the Debian Installer installer team to work on better btrfs integration. Recently I've been working on a rename of btrfs-tools to btrfs-progs, and I submitted at patch for partman-btrfs. The #1 feature I'd like to work on is support for installing to a btrfs subvolume. The #2 feature is btrfs-style multiple device support in the installer. I imagine #1 will be fairly easy. At this point in time I believe that #2 should be limited to the raid1 profile, with mandatory duplication of both metadata and data. Also, at this point in time I do not believe that compression should be supported. Additionally, I've read bug reports recommending displaying a notice in the installer as to the experimental nature of btrfs. That would be #3, but I'd be happy to re-prioritise it as #1. It might also be worthwhile to support the mount options ssd_spread and mkfs.btrfs --mixed in the installer; however, the usefulness for these is limited to filesystems that are < 16GiB, so probably only used for usb flash, satadom, netbook, and embedded. The goal is to ship a "safest possible configuration", to enable those wish to use this next-gen filesystem to try it, while at the same time reducing bug reports that are caused by the current behaviour. For example one of the "killer features" of btrfs is the ability to dump a subvolume as a FAR data stream. This doesn't work out-of-the-box on Jessie, because the feature depends on a named subvolume. I'd also like to discuss whether the default subvolume naming scheme should follow Ubuntu, Fedora, OpenSUSE, or something else. Best regards, Nicholas P.S. I have been subscribed to this list since 5 April 2016
Re: Issues with Installation?
Hi Jen, On 22 April 2016 at 18:50, Jen Longstreet wrote: > For the past 3 or 4 days I have tried to install Debian. I do not think it > is installing correctly because once it finishes the installation, my > computer restarts and then it does not boot up. This is always after > inserting disc one. Do I insert disc two after it does this? (Note: I have > used both a CD and a Flash Drive that has just Disc One on it) Could you please indicate if any error messages appear on the screen when you reboot, or perhaps upload a photo of the screen somewhere and provide a link to it? Kind regards, Nicholas
Re: Issues with Installation?
On 22 April 2016 at 21:36, Jen Longstreet wrote: > https://www.youtube.com/watch?v=u4A4r6wixN4 > > I do apologize for any shaking, background noises and fingers in the video. > This is what it has been doing since I installed from the first disc of > Debian. On 22 April 2016 at 20:47, Jen Longstreet wrote: > There are no error messages...it goes as if it will boot up but instead > of > booting, it goes to a black screen with a blinking cursor...this is any > time > after the installation this occurs...I can provide a short video of that > I > can take using my phone if It would help you understand better It looks like it isn't making it to GRUB; GRUB is the bootloader that lets your BIOS load the Linux kernel that Debian uses 99% of the time. To provide a recommendation on how to fix this I'll need more info. Could you please boot your installation media, and select rescue at the boot menu? Alternatively, type rescue at the boot: prompt. It will ask you some questions to find out where on your hard drive Debian is installed, and will then boot into it. From there you will be able to fix GRUB. But we're not there yet. The first thing I need to know is if you only have one hard drive, if there are no other operating systems installed, and if you're using the MBR or EFI bootloader. From the rescue shell, you can get this info with the command: parted -l (and press enter) <- this will show everything at once. If the text is legible, a photo is fine. or fdisk -l /dev/sda (press enter) fdisk -l /dev/sdb (...) If you get a graphical environment, then you can install gparted, or use the disk tool or partition manager that is already installed. Alternatively, if you're not comfortable with the command line you can use this live boot disk ( http://downloads.sourceforge.net/gparted/gparted-live-0.25.0-3-i686.iso ). Just boot it, find and open gparted (the graphical version of parted), and take a photo of what it finds. Also, in the upper-left of the window there will be a button that specifies which hard drive it looks at. You'll need to click that button and take a second photo if you have more than one hard drive. Oh, and actually fixing this is faster than gathering the info you need to fix it! Cheers, Nicholas
Re: btrfs subvolume naming scheme
On 23 April 2016 at 07:38, New Thread old subject joining team wrote: > On Sat, Apr 23, 2016 at 01:28:43PM +0200, Philipp Kern wrote: >> On Fri, Apr 22, 2016 at 08:30:35PM -0400, Nicholas D Steeves wrote: >> >> > I'd also like to discuss whether the default subvolume naming scheme >> > should follow Ubuntu, Fedora, OpenSUSE, or something else. >> >> What scheme are they using? > > Or a proposal for default subvolume naming scheme? > Ubuntu avoids using the default subvolume (subvol ID 5). For the rootfs their installer creates a subvol called @, for /home it creates @home, etc. In fstab the device is specified and subvol=@ is added to the mount option to specify which subvolume gets mounted. When the volume is mounted without a subvol option, it mounts the whole tree. The tree would be /btrfs/@ and /btrfs/@home if mounted from a rescue disk. I think the symbol is visually striking, but it can be cumbersome because @+ sometimes autocompletes as ipv6 addresses for root. I'm also not sure if @ ever needs to be escaped \@. Oh! I just booted OpenSUSE Leap (their LTS), and it looks like they've now adopted the Ubuntu convention of using @. OpenSUSE has also, to my knowledge, always avoided using the default subvolume. Furthermore, OpenSUSE creates subvolumes for just about everything @opt, @srv, @tmp, @usr/local, @var/crash, etc. Fedora 23 Workstation: When btrfs-style partitioning is selected, their installer creates two subvolumes, home, and root. When the volume is mounted to /btrfs without a subvol= option, the tree would be /btrfs/home and /btrfs/root. Like Ubuntu and OpenSUSE default subvolume is also not used. From what I gather this is a necessary configuration to support btrfs send and receive. Eg: Bug #764056 is a result of our current policy. Unlike LVM or disk partitions, all free space is shared between subvolumes. In the future it will be possible to use qgroups (quota groups) to prevent /var or /home from using up all available free space in rootfs, but at this time I don't think we should support it in the installer, because of the volume of associated bugs and code churn on the linux-btrfs mailing list. Also, in the future it will be possible to mount subvolumes with different options, but at this time the first subvolume mounted sets the mount options for all members of the volume--I'm not sure how to address this the D-I. In consultation with https://btrfs.wiki.kernel.org/index.php/SysadminGuide#When_To_Make_Subvolumes , a subvolume for rootfs and for /home seems sane, and sysadmins can be instructed to consider making a subvolume for /var/www in documentation. This brings us to a concern I have for documentation. How should /var/www appear when it's mounted using a rescue disk? Should it be /btrfs/var_www? If it was /btrfs/var/www, then the two possibilities are: a)/btrfs/var is its own subvolume and /btrfs/var/www is a child subvolume of /btrfs/var note: strictly speaking, all subvolumes what seems to be the root volume are actually children of the default subvolume...the semantics get tricky very quickly! or b)/btrfs/var is a normal directory and in the case of /btrfs/var/www, www is actually a child of /btrfs. Finally, because subvolumes are partitions in POSIX namespace, it's safe to mount a subvolume to two locations, and also to have a /btrfs-admin directory where the whole volume is mounted, at the same time as individual subvolumes are mounted. eg: you have your rootfs mounted at /, and also at /btrfs-admin/rootfs or /btrfs-admin/@. The primary reason to do this is because most of the btrfs tools operate on mountpoints rather than on devices. It also allows centralisation of snapshots. eg: /btrfs-admin/snapshots is a normal directory that holds snapshots of /btrfs-admin/rootfs, /btrfs-admin/home, etc. Résumé: Do we follow Ubuntu and OpenSUSE with the @ convention and work through the issues in bash-completion, or we follow Fedora's plain text/alphanumeric convention, or do we do our own thing? Secondly, Do we want to limit the difficulty of supporting complicated configurations by establishing simple conventions and recommendation early on? eg: all subvolumes created in the installation are peers, and a subvolume that will be mounted at /var/www is named var_www. A default delimiter convention would also need to be chosen. I look forward to your replies, Cheers! Nicholas
Re: joining the team
On 23 April 2016 at 07:28, Philipp Kern wrote: > Hi, > > On Fri, Apr 22, 2016 at 08:30:35PM -0400, Nicholas D Steeves wrote: >> I'd like to join the Debian Installer installer team to work on better >> btrfs integration. Recently I've been working on a rename of >> btrfs-tools to btrfs-progs, and I submitted at patch for >> partman-btrfs. The #1 feature I'd like to work on is support for >> installing to a btrfs subvolume. The #2 feature is btrfs-style >> multiple device support in the installer. > > that would be nice. I think submitting patches to the BTS is the right > way forward. Ok, I'll open two bugs for the two features once I've identified all the different parts that are affected. I've checked out a copy of debian installer according to the instructions on the wiki. >> I imagine #1 will be fairly easy. At this point in time I believe >> that #2 should be limited to the raid1 profile, with mandatory >> duplication of both metadata and data. Also, at this point in time I >> do not believe that compression should be supported. Additionally, >> I've read bug reports recommending displaying a notice in the >> installer as to the experimental nature of btrfs. That would be #3, >> but I'd be happy to re-prioritise it as #1 > > I think optimally the subvolume to install on could be preseeded for #1 > and be able to reuse an existing btrfs volume. Similarly it'd be nice > if one could provide some sort of list of subvolumes to create at > installation time. (Although touching the partman receipe stuff for this > might be a bit messy.) To break down the #1 goal "add subvolume support": a) Add ability to list existing, create, and delete subvolumes in partman-btrfs i. study partman-LVM to learn how to do this ii. alternatively, is partman-zfs in a state where I could use it as a base? -- I noticed that partman-zfs is based off of partman-LVM, and uses a lot of LVM terminology instead of ZFS -- This surprised me, because I thought partman-modules were independent from each other b) Provide defaults for this support in a preseed file i. study partman-LVM and example-preseed.txt to learn how to do this ii. alternatively, is partman-zfs in a state where I could use it as a base? c) How is "the subvolume to install on could be preseeded" distinct from "provide...list of subvolumes to create"? d) Tie into the module that manages fstab mount options, so a subvolume created for /var gets mounted with subvol=a_subvolume_for_var. This seems like the tricky part to me, and afaik there is no equivalent in LVM or zfs. Would this work also be limited to partman-btrfs, or are other modules/packages affected? > I don't think there needs to be such a scary warnings on the kernel > version stretch will ship with. What version is this likely to be? 4.4.x? 4.6.x? From a btrfs-perspective, I hope it's an LTS. >> The goal is to ship a "safest possible configuration", to enable those >> wish to use this next-gen filesystem to try it, while at the same time >> reducing bug reports that are caused by the current behaviour. For >> example one of the "killer features" of btrfs is the ability to dump a >> subvolume as a FAR data stream. This doesn't work out-of-the-box on >> Jessie, because the feature depends on a named subvolume. > > But right now it installs into "@", no? According to debian-installer/packages/partman-btrfs/TODO, support has not yet been added. I just installed debian-stretch-DI-alpha5-amd64-netinst; it failed at package selection after the base system was installed. I dropped to a shell to find out how the btrfs volume was set up. btrfs sub list /target (no output) mkdir /btrfs ; mount /dev/sda1 /btrfs btrfs sub list -a /btrfs (no output) <- if any subvolumes were created, they should have shown up here, without exception mount | grep btrfs /dev/sda1 on /target type btrfs (rw,noatime,space_cache,subvolid=5,subvol=/) /dev/sda1 on /dev/.static/dev type btrfs (rw,noatime,space_cache,subvolid=5,subvol=/dev) /dev/sda1 on /btrfs type btrfs (rw,relatime,space_cache,subvolid=5,subvol=/) subvolid=5 is the default subvolume, and subvol=/ further specifies that this is not a subvolume. Unfortunately, like Jessie, it doesn't install into "@". @ is an Ubuntu convention (apparently adopted by OpenSUSE; stapp...@stappers.nl moved this discussion to the thread "New Thread old subject joining team"). Best regards, Nicholas
Bug#820483: accommodate rename of btrfs-tools to btrfs-progs
On 9 April 2016 at 03:04, Christian PERRIER wrote: > Quoting Nicholas D Steeves (nstee...@gmail.com): >> Package: partman-btrfs >> Version: 20 >> >> I am working on problems associated with bug #780081, where it was >> planned to have a fix staged in experimental after Jessie was >> released. > > > Changes seem to be OK, but I guess that we can't upload before the > btrfs-tools package hasn't been renamed, isn't it? > > From what I see, we need both btrfs-progs and the udeb in the archive > before partman-btrfs is uploaded, or it will: > - FTBFS > - fail when running > > I'm not sure if an upload to experimental is really useful, as the > installer and its components doesn't make good use of experimental. > > I'd say "go ahead for the package rename and we'll apply the changes > to partman-btrfs as soon as btrfs-progs is in unstable". Other > advices? An update: the btrfs-tools to btrfs-progs rename was uploaded to NEW earlier today. I've applied to join debian-boot both by email and on Alioth. Please approve the request if you'd like me to push my changes to Alioth. Otherwise, please let me know you'd prefer a diff, or to pull from my github tree. It seems I forgot to CC you my 9 April reply. Sorry about that! Sincerely, Nicholas
Re: joining the team
On 23 April 2016 at 02:57, Christian PERRIER wrote: > That's welcomed. The team, nowadays, is a bit small and most active > members have other commitments, either in Debian...or in many things > in RL, that makes the life of the team a bit less active than it has > been. Thank you :-) In the long-term, I'm not sure I'll have the time to do more than work on a very specific niche with very specific goals. > So, new energy and contributions are much welcomed. I kinda witnessed > your work on btrfs from far away and I thinnk the best to do is to > apply (on Alioth) for commit access to git so that you can directly > work on components where you'd like to see improvements. > > On behalf of the "team admins" (which shrinks to Cyril and /me) I > encourage you to do so. > Hi Christian! Yes, I remember you replied to my partman-btrfs RFS, concerning the rename of btrfs-tools to btrfs-progs. From what I gather, you prefer changes to be submitted with git, and Philipp Kern prefers diffs. As a result, I'm not sure what I'm I should be using... Also, is it a problem to publish Debian work on github? I've read that it is non-free, and wonder if I shouldn't be using it; it seemed like the only way to give you a git tree to pull from when I only have RO access to debian-installer on Alioth. Best regards, Nicholas
Re: joining the team
On 23 April 2016 at 02:43, Geert Stappers wrote: > On Fri, Apr 22, 2016 at 08:30:35PM -0400, Nicholas D Steeves wrote: >> to work on better btrfs integration. > > My respect for the itch scratching Thank you! I also hope it will help limit the number of btrfs issues submitted to the BTS as adoption of the fs increases. And also, the spirit of Debian is very much to give GPL software its best chance when competing against CDDL alternatives, no? ;-) Cheers, Nicholas
Bug#820483: accommodate rename of btrfs-tools to btrfs-progs
Hi Christian, On 23 April 2016 at 23:05, Nicholas D Steeves wrote: > On 9 April 2016 at 03:04, Christian PERRIER wrote: >> Changes seem to be OK, but I guess that we can't upload before the >> btrfs-tools package hasn't been renamed, isn't it? >> >> From what I see, we need both btrfs-progs and the udeb in the archive >> before partman-btrfs is uploaded, or it will: >> - FTBFS >> - fail when running >> >> I'd say "go ahead for the package rename and we'll apply the changes >> to partman-btrfs as soon as btrfs-progs is in unstable". Other >> advices? Btrfs-progs and the udeb are in the archive now. Cheers, Nicholas
Bug#820483: accommodate rename of btrfs-tools to btrfs-progs
I just noticed that I had Alioth write access, so I pulled my changes to partman-btrfs from my github tree, and pushed them to Alioth. We're 100% ready to go, as far as I can tell. Cheers, Nicholas
Bug#810582: [partman-base] hangs when reformatting existing fs
On 26 April 2016 at 09:48, Andreas Beckmann wrote: > On Sun, 10 Jan 2016 02:21:48 +0100 Cyril Brulebois wrote: > >> > I guess it should run mkfs.ext4 in non-interactive "yes, really do it and >> > don't ask me any questions" mode or something like that. >> >> That's unfortunately a known issue but noone has taken the time to fix >> it yet… :( >> >> Possibly to be fixed in partman-ext3's commit.d/format_ext3 which I >> believe is what we're using even if we're now using ext4. > > That was #767682, and is already fixed in partman-ext3 in stretch. > >> [ To be backported to stable when fixed, too… ] > > I filed a PU request for this: #822678, but wouldn't mind if the D-I > team takes over from here ... Rather than a per-filesystem partman-fs fix, I'd like to propose the following general policy: user chooses a partition if there is already a filesystem possibly notify "this partition is already in use, really wipe it out?" if yes then use wipefs -a to remove all the magic numbers from that filesystem. elsewhat should the alternative be? Old backup superblocks still cause weird problems with btrfs, for example, and I've read it's also an issue for zfs. Isn't it also advantageous to wipe out ext4 magic numbers and backup superblocks when making a fresh start, because this gives tools like testdisk and photorec their best chance of success if something, or someone wipes out the partition table? For completeness, this would also need to be implemented when starting with a fresh partition table. eg: wipefs -a each existing partition, then make a new MBR/GPT. Additionally, it might be wise to wipefs -a the raw device, in case an fs was used on it without a partition table... I am willing to work on this wipefs solution. What do you think? Nicholas
Re: unusability issues
Hi Christoph, On 1 May 2016 at 18:02, Christoph Trunk wrote: > The long version: Your live-CDs and live-CDs seem to regularly request a > username and a password. This is something I have never encountered with > other Linux distributions. > > Even when I seem to have solved that problem (via a laptop running parallel > to my computer), I only get to the console level. With no instructions how > to proceed. I'm sorry to hear you've run into this issue. A google for: debian 8 live-cd username password produced the following top result: https://stackoverflow.com/questions/30842216/debian-8-live-cd-what-is-the-standard-login-and-password So username=user and password=live. Ubuntu has the identical behaviour of needing a password when the screensaver comes up on the live-cd, but their username is ubuntu and their password is blank. > Or you might call it kafkaesque: trying your best in a battle against strange > unknown forces. Nice reference! Are you referring to The Castle? If so, I hope you will find that there are a number of Olga-type characters within the community who will help you on your quest ;-) Kind regards, Nicholas
Bug#787279: d-i screen blanking (more info)
On 11 May 2016 at 09:09, Philip Hands wrote: > Philipp Kern writes: > > Do modern screens still actaully suffer from burn-in? > To my knowledge only screens with a phosphorus layer do, and the only modern (and recently no longer manufactured) ones are plasma TVs...and those eventually suffer loss of contrast or faded black level because they burn the whole screen so that the background region has the same black point as the burnt-in region. In my opinion, older LCDs with CCFL backlighting are the issue, because their life is measured in on/off cycles. CCFL screens were still being manufactured in 2008. Here's a decent, but old (with dead links) article on the topic: https://gblargg.wordpress.com/2009/09/30/ccfl-lcd-backlight-lifespan/ https://www.google.com/search?q=CCFL%20life%20on%20off%20cycle%20lcd&rct=j > Do we realistically expect to do significant damage to the screen (or > the environment) by very occasionally leaving a screen on for 12/24 hours? I don't think so, especially if LED backlit LCDs are the norm... In an ideal world where hardware standards were truly and ubiquitously implemented we could use DDCcontrol to dim any connected screen. This would solve burn-in for CRT/plasma/phosphorus layer screens, this would maximize the useful life of CCFL backlit LCDs, and this would benefit the planet in a small way. ;-) Honestly, I wish DDCcontrol got more love, because in theory all decent commonly-available x86 hardware made post-1999 should support it--LVDS connected displays excepted. Cheers, Nicholas
Bug#787279: d-i screen blanking (more info)
On 11 May 2016 at 20:51, Ben Hutchings wrote: > On Wed, 2016-05-11 at 23:16 +0200, Philipp Kern wrote: >> I also like the dimming idea. It sounds to me like there could be some >> filter be applied to the output of X... > > xrandr --brightness > > But it is implemented in individual drivers by setting hardware gamma > correction, and the generic fbdev driver we use in the installer > doesn't support that. That seems like it would help prevent burn-in on CRTs. It kinda of looks like the hardware is failing when the gamma is wrong ;-) What I like about xbacklight (for laptops, if supported in firmware or with ACPI tricks, or another method for quirky hardware) and DDCcontrol is that they actually dim the backlight or power down the ray gun. Cheers, Nicholas
Re: accessing efivarfs in debian-installer
Hi Francesco, On 24 May 2016 at 06:47, Francesco De Vita wrote: > As explained here [2], the wifi device requires a proprietary firmware > and its nvram-file, a UEFI configuration variable from > /sys/firmware/efi/efivars. Loading the firmware in the DI is not a > problem but I'm unable to access the efivarfs interface. > > Using a tty console in the DI, I can see that the directory > /sys/firmware/efi/efivars is there but it is empty. Mounting the > efivarfs in this path [2] fails with the "no such device" message, and > lsmod doesn't show the efivars and efivarfs modules, which I suppose > are required. > > [1] https://wiki.debian.org/InstallingDebianOn/Asus/T100TA > [2] https://wiki.debian.org/InstallingDebianOn/Asus/T100TA#WiFi > I seem to remember someone else writing about EFI not working poorly with alpha5... Could you please try alpha6? The T100TA wiki states either i386 or amd64+i386 grub-efi need to be used. Here are links to the netinst isos for your convenience. http://cdimage.debian.org/cdimage/stretch_di_alpha6/i386/iso-cd/debian-stretch-DI-alpha6-i386-netinst.iso http://cdimage.debian.org/cdimage/stretch_di_alpha6/multi-arch/iso-cd/debian-stretch-DI-alpha6-amd64-i386-netinst.iso Alternatively, could someone please comment on how the following is different from the multiarch iso? Or does the multiarch iso install a preconfigured amd64+i386 sources.list and packages out of the box, while using a 64bit grub-efi? The reason I mention this mac iso is because I seem to remember that macs are infamous for using mixed-mode EFI. http://cdimage.debian.org/cdimage/stretch_di_alpha6/amd64/iso-cd/debian-mac-stretch-DI-alpha6-amd64-netinst.iso Cheers, Nicholas
Re: touchpad
On 1 June 2016 at 01:54, Geert Stappers wrote:
> On Tue, May 31, 2016 at 10:21:16PM -0400, MY wrote:
>> Dear Team,
>>
>> I apologise for writing to the list, as this is very minor, but since
>> I had thought linux 4.5 would support touchpad ELAN1000:00 04F3:0401,
>> I was very surprised to find a motionless pointer upon trying the live
>> non free firmware iso. This being "unofficial" iso, I then used the
>> official cd, as graphical install, with the same result. Did not
>> complete install as I am drowning in work this week, but have seen this
>> with most current install media from Arch to Ubuntu.
I bet this can be fixed with a one-line patch to hid-core.c. I'm not
sure if :0401 is in the version linux linux-4.5.x you have, but it's
definitely in the linux-4.4.11 I'm using. Alternatively ELAN1000:00
04F3: is somehow not matching "{ HID_I2C_DEVICE(USB_VENDOR_ID_ELAN,"
so it might be a two line kernel patch.
Does the unofficial iso ship with both xserver-xorg-input-synaptics
1.8.3-2, and xserver-xorg-input-multitouch 1.0~rc2+git20110312-2?
Concerning *-input-multitouch, I just noticed that
xf86-input-multitouch v1.0-rc3 is the most recent upstream version,
from 2012. I'm not sure if that driver is specific to Macbooks, but
I've filed a new version available bug.
>> Also, and I have no idea why this would be true, but, since
>> purchasing a bleeding edge laptop, I have tried many distros with
>> kernels greater than 4.2 which would appear to support hardware and
>> one behaviour is always the same, and I am sure you are all aware of
>> it, but I wish merely to re-iterate it:
Out of curiousity, does the touchpad work with the latest OpenSUSE
Tumbleweed (bleeding edge rolling release) LiveCD? It's a great way
to check to see if a bleeding-edge laptop will work with a bleeding
edge software stack. In particular, I'd use the GNOME one, because
I've found KDE5 to be buggy with Intel graphics. If everything works,
then Debian can be made to work, with a little help ;-) My experience
with Arch and Ubuntu is that they have, at most 3/5 to 4/5 of
OpenSUSE's out-of-the-box laptop support.
https://en.opensuse.org/openSUSE:Tumbleweed_installation
http://download.opensuse.org/tumbleweed/iso/openSUSE-Tumbleweed-GNOME-Live-x86_64-Current.iso
>> As to the touchpad, the hid controller seems to prevent the
>> recognition of the actual physical touchpad as a real device, and this
>> has been true in all live trials as well as my current ubuntu install.
>> (I have made a kludge to type quickly)
So the touchpad is detected by the kernel after all? Was the
"motionless pointer" issue only a problem on the installation media?
I'm not sure if the following is still applicable for fully enabling
Elan touchpads, because I think the driver has been integrated into
the kernel, but these links are possibly worthwhile:
http://www.dahetral.com/public-download/alps-psmouse-dlkm-for-3-2-and-3-5/view
http://forums.debian.net/viewtopic.php?f=16&t=95940
>> If there is anything I can send you to help, please let me know,
>
> For starters name and shame the vendor that told
> you "Yes, it has full Linux support".
> That is based upon asking before buying.
Geert, if you're joking, the joke was above my head...If it works with
an out-of-tree driver, I think it's a grey area for what they can
claim :-p On the other hand, if it's a Linux-specialised vendor like
Emperor Linux, or Zareason, I totally agree...but only if the
out-of-tree driver they integrated is flaky, or if they didn't bother
to integrate one into the installation they shipped the laptop with.
And then, only if they're not willing to fix the flakyness, because
that would be a breach of the TOS--the expectation being that paying a
premium for out-of-box functioning provides reliable out-of-box
functioning.
> Consider returning the laptop.
> Make the problem a problem of the vendor.
If it works reliably with an out-of-tree driver, or reliably with a
driver in staging, maybe not? Maybe I'm cynical, but given the
purportedly extremely low profit margins for all commodity hardware
(Apple and possibly Panasonic excluded), I would expect the vendor to
conclude "those demanding Linux users aren't worth it, frack 'em, they
can figure this out on their own." IBM purportedly sold off both
their Thinkpad and Thinkserver lines due to the razor-thin profit
margins.
Now for braindead ACPI and/or UEFI firmware, yes, make a problem of
the vendor! Also for firmware that enables DMA for the WLAN chipset
in such a way that it corrupts arbitrary segments of memory, vilify
them, and spread the word. Or a cooling solution that is so utterly
dependent on OS drivers that taking a long phone call after entering
the BIOS will permanently damage the hardware? Yeah in these cases, a
storm of bad PR is not only just, but maybe even a duty. ;-) A storm
of bad PR is never just between the buyer and the vendor...
But for that grey-area of companies who are half-way
Re: btrfs subvolume naming scheme
Thank you for the replies, and I'm so sorry this email is long. I had also hoped that by taking time to think through the issues I would place less of a "this email is so long and taking so much of my time!" burden on everyone reading this thread. On 27 April 2016 at 07:43, Philipp Kern wrote: > On 2016-04-23 23:51, Nicholas D Steeves wrote: >> >> Ubuntu avoids using the default subvolume (subvol ID 5). For the >> rootfs their installer creates a subvol called @, for /home it creates >> @home, etc. In fstab the device is specified and subvol=@ is added to >> the mount option to specify which subvolume gets mounted. When the >> volume is mounted without a subvol option, it mounts the whole tree. >> The tree would be /btrfs/@ and /btrfs/@home if mounted from a rescue >> disk. > > > I think avoiding the default subvolume is the sensible approach. It should > be possible to reuse the volume by multiple root filesystems if needed. > (Just like this is possible with LVM today.) And in the future, multiple boot environment support! I think openSUSE might already have integrated it, but this would be an early-to-mid fall project for me. At a bare minimum the creation of a boot environment should someday occur before running a dist-upgrade. I've read BEs are quite popular with Archlinux users, whose systems break more often than most ;-) > I'd personally prefer if we would only mount the btrfs filesystem once, but > I don't know what the best guideline here is. If we mount it multiple times > at different subvolumes, the output of mount is pretty confusing to the > user. The user would of course still be free to mount additional arbitrary > subvolumes later and end up with this state. Hmm, ok, I guess the default should be like other distributions (1 subvolume -> 1 mountpoint), and then address the two different topologies of snapshots with pros and cons in the wiki, and let the sysadmin configure it how he/she likes. (TODO) I definitely need to more explicitly, address the dangers of going snapshot crazy, or using a loose and easy snapper config, because performance crashes somewhere between at 250 and 300 snapshots per subvolume, and also sometimes wedges the volume into an unmountable state. > I think /var/log would be very sensible as a separate subvolume by default. > Usually if you want to snapshot your rootfs, you really don't want log files > to take part in the snapshotting. I suppose the same argument can be made > about /home and a non-tmpfs /tmp. There are also people who want to push for > all OS content to be located in /usr, but we still have a lot of content in > /var so that doesn't seem feasible in the near time. Will omitting /var/log from backups cause any systemd or journald voodoo to cause errors on restore? If /usr and /var should be on the same subvolume, and /var and /etc/ need to be, then rootfs (for future enabling of boot environments) should encompass everything needed to boot. User data should probably be separate, because it would be invisible/appear to be "lost" when reverting to an older BE. Likewise, if BEs are supported, the wiki (note to myself) would need to recommend creating a subvolume for /var/www, location of a major database, and/or any exported samba or nfs mounts. The primary argument I've read against doing this at installation (like openSUSE does) is that it increases the chances of something going wrong or performing poorly in the same way as "too many snapshots." > Kind regards and thanks for spawning these discussions > Philipp Kern You're welcome. Sorry for the delay in following up. On 24 April 2016 at 02:30, Geert Stappers wrote: > On Sat, Apr 23, 2016 at 05:51:25PM -0400, Nicholas D Steeves wrote: >> On 23 April 2016 at 07:38, New Thread old subject joining team >> wrote: >> > On Sat, Apr 23, 2016 at 01:28:43PM +0200, Philipp Kern wrote: >> >> On Fri, Apr 22, 2016 at 08:30:35PM -0400, Nicholas D Steeves wrote: >> >> >> >> > I'd also like to discuss whether the default subvolume naming scheme >> >> > should follow Ubuntu, Fedora, OpenSUSE, or something else. >> >> >> >> What scheme are they using? >> > >> > Or a proposal for default subvolume naming scheme? >> > >> >> Ubuntu avoids using the default subvolume (subvol ID 5). For the >> rootfs their installer creates a subvol called @, for /home it creates >> @home, etc. In fstab the device is specified and subvol=@ is added to >> the mount option to specify which subvolume gets mounted. When the >> volume is mounted without a subvol option, it mounts the whole tree. >> The tree would be /btrfs/@ and /btrfs/@home if mounted from a rescue &
Re: Next d-i alpha release: late June
On 24 June 2016 at 18:22, Cyril Brulebois wrote: > Hi, > > I've just checked with Ben, it seems we could be getting a 4.6 kernel > suitable for testing (no regressions reported from previous version + > mips* FTBFS fix) shortly. We could think about urgenting it into testing > and releasing a new d-i early in the week, which seems OK on the -cd > side too. > > Glibc maintainers (esp. Aurélien): you should then have a clear path for > the new glibc in unstable. I'm not sure how much time it'll need to be > ready, that's why I'd slightly prefer if we could go for a d-i release > first (as outlined above). In case major blockers pop up, we would > probably let you go ahead with the new glibc upload and postpone d-i > until glibc reaches testing. > > Having checked with -release already, I'm freezing udebs right away. Could someone please tell me what the deadline is for adding expanded partman-btrfs functionality? My proposal is in a thread on debian-boot@lists.debian.org, subject: "Re: btrfs subvolume naming scheme". A résumé of the read is: add the volume-manager-like subvolume setup, and hopefully also add btrfs-style raid1 profile support, and also the question of whether Debian should follow Ubuntu and openSUSE subvolume naming conventions or Fedora/CentOS/RHEL ones. The upstream wiki advocates Fedora/CentOS/RHEL-style. Thank you, Nicholas
Re: Thinking about a "jessie and a half" release
On 4 July 2016 at 09:12, Cyril Brulebois wrote: > Hi, > > Steve McIntyre (2016-07-04): >> There's something I've been pondering for a while, along with some >> other folks - it might be useful to do a "jessie and a half" release, >> similarly to what we did in the etch days. That's *basically* just >> like a normal jessie release, but with a few key updates: >> >> * backports kernel > > That's a given. I still wonder if a fork of the last linux:src=4.4, updated to bring it to linux-4.4.14 would be a lower support burden? I'm still finding that there are a fair number of issues reported with 4.5.x and 4.6.x on various mailing lists. Does anyone know how Skylake support is like for the 4.4.x branch? What is arm64 support like? I've corresponded with Ben Hutchings, and he tells me an LTS kernel effort is ok to do, but unofficial. Personally, I believe following bpo kernel is a bit of a rodeo in comparison to what one expects from Debian Stable, which is why I'm looking into this project. I would very much prefer to do it as a team, because I do not believe that I am qualified to maintain kernel packages. Philipp, what do you think? >> * rebuilt d-i to match that kernel > > You know there are patches around for that. > >> * X drivers > > I don't see backports for them. libdrm2: 2.4.68-1~bpo8+1 libgl1-mesa-.*: 11.1.3-1~bpo8+1 xserver-xorg-video-intel:2:2.99.917+git20160522-1~bpo8+1 I backported the libva stuff and required dependencies yesterday. Some time ago, there was also a user who requested a xserver-xorg-video-savage bpo on IRC. I made a quick one, sent him the link, but he never filed a bug report and I forgot to officially upload it. In private email correspondence with Andreas Boll, with respect to backported X drivers and mesa, it came to light that On 21 March 2016 at 14:49, Andreas Boll wrote: > > Actually radeonsi requires LLVM >= 3.6 but since we currently use Mesa > with LLVM 3.7 in unstable and testing it would make sense to backport > LLVM 3.7. So for radeon hardware enablement, there is 1) the proprietary driver 2) a need to backport llvm. >> * ... (other things that might be needed for consistency) Maybe pinning a list of drivers that a user would expect would come from backports instead of main? eg: presumably a user would expect that this Jessie+hardware enablement bpos would choose the latest broadcom-sta-dkms in backports. I could be wrong here, and I recognise that backports are officially opt-in only, but if we're already opting the user into a bpo X-stack, wouldn't he/she also assume that installing some-non-free-dkms-driver will also pull in the bpo version? Also, I imagine this might one day be necessary, particularly if tracking backported linux-src due to ABI changes between kernels, eg: if the out-of-tree drivers in testing begin to require >> linux-src=3.16.0 ABI. Ben, could you please lend your insight into this point? >> all rolled up with a small installer image build (netinst, maybe >> DVD#1). > > That'd probably make it easy to decide how to resolve open questions > with my "d-i vs. backported kernel" patches. > >> A lot of arm64 machine users would benefit from this, and maybe owners >> of very recent amd64 machines too, with better support for things on >> the Skylake platform. Those are the only two architectures I'm >> thinking of supporting at this point. >> >> Is anybody else interested in helping? Thoughts/comments? Yes, it's a project I'm already working on ;-) Is this project a candidate for a new Debian Team? > Questions: > 1. Is it going to pick pieces from backports only? (See X question > above.) Please see Philipp and my concern wrt rolling kernel updates. > 2. Does it have to be called "jessie and a half"? (How much is the > concept understood across users? Wouldn't it be a better idea to > squeeze the "backports" concept into the name somehow?) Maybe something like jessie-fresh-unofficial? > 3. What about security support once the system is installed? (Which > can be answered along with 1., I suppose.) Given the current status of backports, security support would be "Unfortunately not" ( https://backports.debian.org/FAQ/ ). This is one point in support of a team that would monitor a subset/list of supported/shipped/blessed backports for security issues. Sorry this is so long, I'm excited to see that there are other people interested in this project! On 4 July 2016 at 13:13, Hideki Yamane wrote: > > Just a comment. I don't have any objection for this proposal. > > However, not only half but also updates with some point is better > to deliver value for users, I hope it'll be in Stretch cycle. > > Recently I've read "lean software development" and it's quite > impressive for me. "deliver value to users" is one of the most > important thing in Debian (it means "do continuous improvement > for stable"), IMO. Agreed! Also, OpenSUSE has been doing this with their post-42.x release model. Mind you
Re: Thinking about a "jessie and a half" release
On 5 July 2016 at 08:40, Samuel Henrique wrote: > > 2016-07-05 7:43 GMT-03:00 Jose R R : >> >> We're getting to the point where there's a fairly pressing need for >> arm64 - the more useful hardware is starting to get a wider distribution >> and we don't really have anything for people who want to run Debian >> that gets them a supported system with an upgrade path. > > > We have Debian Stretch, which is what i recommend to anybody who wants do > install Debian as a desktop. > > I understand the difference of running Debian Testing and Debian Stable with > some backported packages, but is it really worth it? > Shouldn't we discuss more the usability of Testing as a solid release, or > maybe start doing a stable release and another release kinda like Testing > but with more stability guarantees? > > I'm not really sure, but i think opensuse has a model like that. > ... > They always end up using Debian Testing, knowing that the main risk comes > when the unfreeze happens and that while the freeze is rolling they will > have a more stable debian (compared to when unfreezed). I personally like to test stable+1 on my laptop by changing stable to stable+1_codename about a month after the freeze happens; it then transitions to stable automatically, and no trouble with the unfreeze. As for building off of a [semi]rolling release model, from testing, I'm pessimistic because of historical precedents for success. For example: http://cut.debian.net/ https://www.reddit.com/r/debian/comments/3yeg6z/what_happened_with_debian_cut/ Tanglu seems to have stalled, and SolydXK is now stable+bpos, but maybe they will go back to using stable+1 once it freezes? And a recent discussion on running testing -> http://forums.debian.net/viewtopic.php?f=30&t=128598&start=0 Of course, I'd also love to see it work! :-) I'm guessing it requires a substantial investment of time and a very dedicated—and large enough—team. Cheers, Nicholas
Re: Thinking about a "jessie and a half" release
On 4 July 2016 at 18:38, Ben Hutchings wrote: > On Mon, 2016-07-04 at 16:01 -0400, Nicholas D Steeves wrote: > [...] > [...] >> So for radeon hardware enablement, there is 1) the proprietary driver > > fglrx is dead upstream and removed from unstable. (It's still in > jessie-backports, but should be removed there as well.) Ok, so a very real need for the new radeon driver bpo is/has probably emerging/ed. And Bug #828087 on others mentioned in another email to this thread are blockers to a bpo of xorg-core, which would need to happen first. > [...] >> Also, I imagine this might one day be necessary, >> particularly if tracking backported linux-src due to ABI changes >> between kernels, eg: if the out-of-tree drivers in testing begin to >> require >> linux-src=3.16.0 ABI. > > I think you mean API. We don't ship out-of-tree drivers as binaries. > Yes, you're absolutely right, I meant API. Given that people with, for example broadcom wifi chipsets will be using this new Debian spin (Maybe it should be called Debian Fresh Spin?) in the hopes of hardware enablement, do you think it should pin the bpo versions of out-of-tree drivers, and also include bpo non-free firmware? Cheers, Nicholas
Re: btrfs subvolume naming scheme
On Sat, Apr 23, 2016 at 01:28:43PM +0200, Philipp Kern wrote: > On Fri, Apr 22, 2016 at 08:30:35PM -0400, Nicholas D Steeves wrote: > > > I'd also like to discuss whether the default subvolume naming scheme > > should follow Ubuntu, Fedora, OpenSUSE, or something else. > > What scheme are they using? Please consult the following thread for a new upstream discussion on this debian-boot thread. It consistitutes something of a bug report + established alternatives and is much more succinct than my writing! : http://www.spinics.net/lists/linux-btrfs/msg56959.html Where should I ask D-I questions as I familiarise myself with partman-btrfs? Also, what procedure do you use to test the installer and updated modules before pushing your commits? Thank you, Nicholas
Re: Thinking about a "jessie and a half" release
Hi Jeffrey, On 12 July 2016 at 09:28, Jeffrey Walton wrote: > > I think it would benefit more than Skylake users. The last few > processors are missing support. Below is from a Core i5-5300U (5th > gen) and a 3.19.0-64-generic kernel. > > ** > > $ dmesg | egrep -i '(error|failed)' > ... > [35679.953137] mce: [Hardware Error]: Machine check events logged > [35980.749723] mce: [Hardware Error]: Machine check events logged > [36280.594838] mce: [Hardware Error]: Machine check events logged > [36580.439940] mce: [Hardware Error]: Machine check events logged > [37029.202190] mce: [Hardware Error]: Machine check events logged > [37179.118743] mce: [Hardware Error]: Machine check events logged > [37629.831878] mce: [Hardware Error]: Machine check events logged > [37779.748453] mce: [Hardware Error]: Machine check events logged > [38229.510127] mce: [Hardware Error]: Machine check events logged > > $ sudo mcelog > mcelog: Family 6 Model 3d CPU: only decoding architectural errors > Out of curiosity, what does the mcelog version in jessie-backports output when running this kernel version? Cheers, Nicholas
Re: Thinking about a "jessie and a half" release
On Tue, Sep 06, 2016 at 11:27:49PM -0400, Jeffrey Walton wrote: > > There's something I've been pondering for a while, along with some > > other folks - it might be useful to do a "jessie and a half" release, > > similarly to what we did in the etch days. That's *basically* just > > like a normal jessie release, but with a few key updates: > > > > * backports kernel > > * rebuilt d-i to match that kernel > > * X drivers > > * ... (other things that might be needed for consistency) > > > > all rolled up with a small installer image build (netinst, maybe DVD#1). > > > > A lot of arm64 machine users would benefit from this, and maybe owners > > of very recent amd64 machines too, with better support for things on > > the Skylake platform. Those are the only two architectures I'm > > thinking of supporting at this point. > > > > Is anybody else interested in helping? Thoughts/comments? > > Sorry to bump an old thread > > Please consider moving to Clang 3.8 or 4.0 as the LLVM front end for > the platform. > > Clang 3.5 and 3.6 are no longer maintained. The bugs we are > discovering and reporting are being closed as "invalid" and "won't > fix" because Clang is outside its freshness date. > > Also pick up this for glibc: > http://stackoverflow.com/questions/17775390/clang-3-3-in-c1y-mode-cannot-parse-cstdio-header/17776548#17776548 > . Though it was first seen in Clang 3.3, its still a problem today. > > Jeff > Hi Jeff, Actually, good timing to bump the thread! Gianfranco has backported LLVM and Clang 3.8. Cheers, Nicholas signature.asc Description: Digital signature
Bug#838031: task-laptop: Add 'ntp' to list of recommended packages.
On Fri, Sep 16, 2016 at 08:55:19PM +0300, Michael Tokarev wrote: > 16.09.2016 20:51, Ben Hutchings wrote: > >We should install a minimal NTP client by default. Not ntp, it's far > >more complex than needed and (partly as a result of that) has a poor > >security record. > > Systemd comes with systemd-timesyncd these days, JFYI. Is the addition systemd-timesyncd documented somewhere? 'just something along the lines of "Simple ntpdate-like time synchronisation is now provided by systemd". I read the NEWS when upgrading and didn't notice this change. Does the laptop task support non-systemd inits? If so, would there be a benefit to a systemd | ntpdate (or alternative) dependency for the task? Cheers, Nicholas P.S. Is openntpd the defacto standard these days, for servers? signature.asc Description: Digital signature
Bug#838919: Proposed documentation, please comment! [was Re: Bug#838919: debian-installer: please calculate swap parition according to max RAM...]
On Thu, Sep 29, 2016 at 02:51:24PM +0300, Martin-Éric Racine wrote:
> 2016-09-29 14:15 GMT+03:00 Ben Hutchings :
> > On Wed, 2016-09-28 at 16:20 +0300, Martin-Éric Racine wrote:
> > [...]
> >> The thing is, right now, the user has two choices:
> >>
> >> 1) Trust d-i to make the right choices once, even though more RAM is
> >> likely to be added later on, at which point there won't be enough swap
> >> to save the suspend image;
> >
> > Why do you think it's likely? Hibernation is mostly used on laptops
> > and I don't believe they often get RAM upgrades; in fact increasingly
> > they don't have even have sockets for RAM upgrades.
>
> Because it's what everyone I know does: as soon as the price for the
> type of RAM their computer needs drops, they max it out.
[...]
> As for hibernate, it in fact is used a lot on office desktops as well.
> It was the preferred way of leaving the office at my two previous
> workplaces.
[...]
> > How many 'lay users' that would be overwhelmed by the partitioner
> > nevertheless understand how to upgrade RAM, what swap space is, and why
> > they might need more than the default? I think that's a very small
> > group indeed.
>
> Pretty much everyone I know knows how to download the upgrade
> instructions from the manufacturer's website, remove a couple of
> screws and insert a memory module. By comparison, very few people I
> know understand the intricacies of LVM partitioning.
Dear M. Racine and DDs,
This seems like an opportunity for a useful HOWTO, or page in the
Debian wiki, or a new section in wiki.debian.org/Partition.
Page title or section heading of "I've upgraded my RAM and my
swap partition/LV is no longer big enough to hibernate".
1. Notice to refresh one's backups before attempting this.
* For lay users, this might require additions to the
"BackupAndRecovery" wiki page.
* Needs consensus on the type of backup. I imagine a bare-metal
backup/restore from a live cd would be easiest for lay users...
2. Officially recognised boot/rescue cd
* for lay users, it must contain system-config-lvm and gparted
- Is GNOME Disks sufficient?
* Is everything needed for this already on the gnome live disk?
3. Try system-config-lvm first; if it can't find any LVM-managed
disks, then close it and use gparted or GNOME Disks.
--
4. Resize a partition or LV.
5. Delete swap partition or LV.
6. Create new swap partition or LV.
* Does LVM swap need to be contiguous for hibernate?
* What is the official minimum size of swap file for hibernation?
- I've always used something like:
echo "`free -m | grep Mem | awk '{print $2}'` * 1.15" | bc -l
--
7. There would also need to be a section on reinitialising encrypted
swap. Consideration of this is why I went with the delete, resize
create method rather than attempting to resize, because I'm not
sure if resizing encrypted swap is supported...and I have no
experience with encrypting disks/partitions/LVs.
If the format is for lay users, using a GUI, then including
screenshots would make sense? Where can I upload images to for use in
the Debian wiki? I'd be happy to work on this documentation if no one
else wants to. Please comment on the proposed document structure and
the issues I've raised in the sub-points. Also, if someone with
experience with LUKS would like to write this documentation (or the
LUKS section), please let me know! :-)
Regards,
Nicholas
signature.asc
Description: Digital signature
Bug#840248: debian-installer: Add btrfs subvolume setting for snapshot
Control: owner -1 Control: tags -1 pending On Mon, Oct 10, 2016 at 09:22:14AM +0900, Hideki Yamane wrote: [...] > debian-installer can format disk with btrfs now, but it is NOT appropriate > setting with btrfs. We can just format partion with btrfs but cannot create > btrfs "subvolume" at that time. > > Subvolume is a bit special idea, you can slice one btrfs partion to some > subvolume and can set quota for each subvolume, also mount directory and > get snapshot for each. > > So, I'll propose debian-installer to add btrfs subvolume setting menu or > add subvolume setting like SUSE by default. [...] Hi Hideki, Thank you for the reminder! I've been planning to do this work for quite some time. ( https://lists.debian.org/debian-boot/2016/04/msg00277.html ) So far, the plan is to default to simple @rootfs and @home subvolumes, because I've read that backing up OpenSUSE systems is cumbersome with all of those subvolumes, and also because of the KISS principle; That said, one will have the flexibility to choose this style, if desired. I absolutely agree that, for example, there needs to be a mechanism in place to allow configuration of a separate subvol for /var/www and /var/database_of_choice at the time a Debian system is installed. The second half of better Debian btrfs support is better documentation. ( https://wiki.debian.org/Btrfs ) To the best of my knowledge, the following is still applicable: https://btrfs.wiki.kernel.org/index.php/Mount_options As a consequence, per-subvolume btrfs mount options set in the installer would not be honoured, because the fstab mount option for @rootfs are applied to all other subvolumes . The three workarounds are 1) Create a separate physical partition for @nodatacow_subvol. 2) chattr +C the mountpoint, relying on the application's own COW and checksumming implementation. 3) Disable the application's COW and checksumming implementation and rely on btrfs'. I'll try to having something ready for testing by Oct 24th, with a hard deadline of Nov 1st. Cheers, Nicholas signature.asc Description: Digital signature
Re: Next d-i release
On Wed, Oct 19, 2016 at 03:33:03PM +0200, Cyril Brulebois wrote: > Hi, > > Since linux vs. fat/efi is no longer an issue, I'm tempted to prepare > a new d-i release soonish. I'll probably freeze udebs in the upcoming > hours or days, and try to figure out what to do with packages sitting > in unstable for the time being. > > Feel free to mention packages you want to see in testing, in case I go > for a conservative approach (and only hand-pick a few packages); feel > free to cc me explicitly to make sure I read your replies. > > > KiBi. What is my deadline for adding minimal subvolume support to partman-btrfs? If it needs to be done in the next 24h, my plan is to default to creating a subvolume called @ for each btrfs volume and silently add subvol=@ to the mount options and fstab entries for these volumes. With this change there is a 1-to-1-to-1 partition-to-volume-to-subvolume mapping. I'd like to start with this, just to make sure the various bootloaders handle it cleanly; I'm 95% certain the existing grub-install logic will handle this change without incident, but I'm not sure about other bootloaders on other architectures. That might take some time to shake out...essentially, the bootloader needs to detect that /target is mounted with -o subvol=@ and add rootflags=subvol=@ to the kernel command line. What is my deadline for more complete btrfs support? What documentation can I consult when working on this? I've been spending time reading everything in partman-lvm, and partman-zfs, but I feel like I'm going to miss the deadline at this rate. Specifically my plan is to: 1. Add a "Configure btrfs volumes" option underneath "Configure encrypted volumes": 2. Partitions that are configured to be used as "btrfs journaling file system" appear in this menu, just like a raid-configured partition appears in "configure software raid" 3. Then it offers two options "configure profile" and "configure subvolumes. 3.a - The following "configure profile" menu appears. i. Single (similar to lvm append) data with raid1 (two copies on different devices) metadata ii. Raid0 data with raid1 metadata iii. Raid1 data and raid1 metadata p.s. In the future, when raid5/6 stabilise they can be added as option. I believe the above options strike the balance between what is currently most featureful and as safe as possible. p.p.s. If a user wants to live dangerously, he/she can rebalance a live system to raid0 metadata for maximum performance, or any other possible btrfs profile after installation completes and is rebooted iv. The next menu is like the "active devices for the raid1 array" * default to creating @rootfs, and configure / on @rootfs; then mount -o subvolume=@rootfs /target. In the main "Partition Disks" screen there will be a btrfs volume heading which displays this configuration. 3.b - Configure subvolume menu; this functions similarly to "Configure the Logical Volume Manager" menu, after an PV has been designated, and after a VG has been created. i. When the user enters this menu, @rootfs is preconfigured and a list of existing subvolumes is shown. iii. The only two options are create and delete subvolume (equivalent to LVM create LV and delete LV) -- 3.b.alt Would it be better UI design to skip the branch from 3 -> 3.a || 3.b inside of btrfs options, and instead write a new "Partition disks" mode which appears under an "available btrfs volumes". If we use this method instead, then once a btrfs volume's profile (equivalent to [mdadm +] pv+vg) configured in 3.a it will appear with its "btrfs volume" heading above the physical partitioning section on the main "partition disks" screen. By default, the item "create subvolume" will appear under the volume heading. Like the "Partition Disks" "Partition settings" it allows configuration of Name, Mount point, but in addition to these only the Remove subvolume and Done setting up subvolume menu options are provided. If there is not something configured to be mounted at /, then the first time the "create subvolume" screen is invoked Name will be preconfigured to @rootfs. Because per-subvolume mount options are unsupported or poorly defined, by default btrfs volumes will be configure with -o default in /etc/fstab. The 3.b or 3.b.alt configuration is what generates the subvol=@probably_should_be_limited_to_ASCII_name. mount option used in fstab. So yeah, with this plan Debian Stretch's installer finally have equivalent btrfs support to other major distributions :-) Any advice or reference material would be very much appreciated! Sincerely, Nicholas signature.asc Description: Digital signature
Bug#866085: At least it should be the option to not download
On Wed, Jun 28, 2017 at 09:13:05PM +0200, Narcis Garcia wrote: > > > > If you are doing an install party, set up a proxy server. That really > > really helps a lot. > > > > Not only experts should be able to do an install party; some more people > wants to share small knowledge and experiences. > Could this be solved with a proxy server installation task? With the move to https-only apt I worry that configuring one will become too difficult... Additionally I wonder how difficult it would be to add some sort of mdns magic so that something like apt-mirror.local could be used for the mirror? After that, update sources.list on all the new systems so that they function without the install party mirror and Bob's your uncle. Cheers, Nicholas signature.asc Description: Digital signature
Bug#690763: installation-guide: sudo and no password for root user situation
Hi Holger, On 11 July 2017 at 19:11, Holger Wansing wrote: > Baptiste Jammet wrote: >> Hello, >> >> I don't know the implementation detail of this (no root password >> installs sudo and allow user to use it), but I suspect that only the >> first user (created in the next step) will be allowed to use sudo. >> >> Maybe add something like : >> By default, only the first user, that will be created in the next step, >> will be allowed to use sudo to perform these >> administrative tasks. > > Yes, you are right, it works exactly this way. > > I have committed a change to document this clearer: > https://anonscm.debian.org/viewvc/d-i?view=revision&revision=70801 Quick question about this change: Is the first user given sudo privileges now when the root account is enabled (eg: first user always has admin privileges irregardless of root account status)? IIRC when the root account is enabled the first user needs to be granted permissions with visudo. I'm not up-to-date with the new installation procedure though! ;-) Cheers, Nicholas
where can I find the team?
Dear Kibi and Debian Boot team, Where can I find the team this evening? I have two questions relating to a bug I'd like to close that probably have short answers...but might not be so short :-) Sincerely, Nicholas signature.asc Description: PGP signature
Re: bts reassign 878722 partman-auto
On Fri, Nov 10, 2017 at 12:32:59PM -0500, Lennart Sorensen wrote: > On Fri, Nov 10, 2017 at 04:19:14PM +, Ben Hutchings wrote: > > This is true, but I don't think it's a good reason not to implement a > > mostly-reliable heuristic. > > > > If there are multiple disks, there are usually going to be just 2 of > > them, one of which contains the installer. In any installer build > > other than netboot, it will look for its own disk in order to load > > udebs. Once it has done that, it can determine that the other disk is > > the one to install on. That's a pretty good heuristic. > > I think more than one disk in the machine isn't that unusual. Is there any reason why the following method wouldn't be an improvement?: 1) get a list of disks 2) identify the disk used by the installer 3) exclude the disk found at #2 4) present modified list as target disks for installation 5) unless in expert mode, where a user could use one partition of a disk as the installation source and another partition as the installation target. I'm not sure how important #5 is, but maybe some users want to be able to do this? Cheers, Nicholas signature.asc Description: PGP signature
Re: bts reassign 878722 partman-auto
On Mon, Nov 13, 2017 at 10:35:19AM -0700, Ben Hildred wrote: >On Fri, Nov 10, 2017 at 12:50 PM, Cyril Brulebois <[1]k...@debian.org> >wrote: > > Nicholas D Steeves <[2]nstee...@gmail.com> (2017-11-10): > > 1) get a list of disks > > 2) identify the disk used by the installer > > 3) exclude the disk found at #2 > > How do you do 2? > > Last I touched this, nothing obvious appeared in d-i to know what the > installer was booted from. ISTR having suggested at the time that > bootloaders could set something to help d-i figure out where it booted > from, but I don't think anything happened in this area since then. > >OK, This Is a crazy Idea, but . . . When generating Installer images, they >get various readmes and so on, and I believe one of them includes version >information, so we can parse that file for the version number, compare it >with the one in the initrd (to be added). This handles the common cases of >cd (overkill as cds are read only) and USB (presumably most important), >but fails on net-install (where it is not needed). We can have installer >loader scripts copy the version info file to mark the drives they are >using which should catch most of the rest of the cases. >A variation of this is to have a pseudo random token in the version file >which is passed on the command-line to the installer instead of modifying >the initrd. This has the advantage that we could have special case values >for net-boot to skip the scan. (ie if the token was a hexadecimal value >but the special case was the word netboot. > >both cases make identifying and protecting partitions used to store >archives and iso images easy by manually placing the version file. Why does net-boot need to be special cased? By default, shouldn't the net-boot media be excluded as an installation target--except for the expert case. Another feature that could be piggybacked on the "mount block device and identify if it's Debian installation media" is OS identification. It's been so many years since I dual-booted with another OS that I don't know if this functionality already exists. If it does exist, I'm guessing that is where this new "Identify installation media" could be added. Re: the identification step: I don't think it's crazy :-) Do you know if such a magic file already exists (especially if the installer build was recorded)? Can busybox-provided cat, grep, cksum and cmp provide strong enough magic file match, or do you think we also need a /sys/.../something/uuid check as well? Cheers, Nicholas signature.asc Description: PGP signature
Re: Bug#861263: debian-installer: zfs support
On Wed, May 10, 2017 at 05:24:48PM +0200, Wouter Verhelst wrote: > On Sat, May 06, 2017 at 03:35:52PM +0100, Sam Kuper wrote: > > On 06/05/2017, Ian Campbell wrote: > > > It would in theory be possible to arrange build and install modules > > > during installation using the in-progress target installation (where > > > the normal toolchain packages could be installed) such that they are > > > available during the latter parts of the install procedure -- that is > > > clearly of limited use for any modules which are required for the > > > filesystems which you want to install to (certainly the root fs and > > > perhaps any separate /usr or /var partition(s)). > > > > Given that a machine intended to run ZFS is likely to be provisioned > > with >2GB of RAM (much more than the Debian Installer would normally > > require), might a viable workaround be for the Debian Installer to > > take the following steps? > > > > - Ensure that some minimum amount of RAM is available, and refuse to > > proceed with a root-on-ZFS installation if not; > > > > - install the toolchain, ZFS source, and any other packages from > > "main" or "contrib" necessary to support these, to a RAM disk; > > > > - compile and load ZFS functionality; > > Add partitioning here, and add an extra partition used for holding the debootstrap system. [1] > > - format the target HDD/SSD using ZFS; > > > > - copy the toolchain binaries and ZFS source packages, etc, to the > > relevant places on the target HDD/SSD (/usr/bin , > > /var/cache/apt/archives/ , etc); > > > > - resume the installation as usual. Before installing the bootloader, check if it's a ZFS installation, and if so, then replicate the debootstrapped system from the extra partition to the ZFS dataset marked as rootfs, using something that supports --numeric-owner. [2] Then delete the extra partition, expand the ZFS one to the end of the device. eg: [1] sda1: 256MB /boot sda2: 930GB zfs-vdev And then the installer substracts 4GB from sda2, creates sda3 for debootstrap, and takes care of the removal transparently. Using this method, sda2 only gets expanded to the size that is requested, instead of the whole disk--allows underprovisioning. > I don't see why not. We already need the ability to run debootstrap; > this would just change that to a need to do so twice. > > It wouldn't be terribly efficient for a netinstall, since it requires > that you download the base system twice; but it could imagine a special > "non-free netinstall" image which would contain zfs-dkms and its > dependencies on top of the base system, so that you don't need to > download things more than once. I'm not sure if [2] is the right place in the installation sequence for this, but it seems like something like this method could make efficient use of bandwidth for a netinstall :-) Cheers, Nicholas signature.asc Description: PGP signature
Re: Bug#886968: btrfs-progs-udeb: depends on non-udeb: libzstd1
On Mon, Jan 15, 2018 at 02:20:28PM +, Dimitri John Ledkov wrote: > On 15 January 2018 at 00:27, Cyril Brulebois wrote: > > Hi, > > > > Cyril Brulebois (2018-01-12): > >> Your package is no longer installable (along with its rev-dep > >> partman-btrfs) because it now depends on libzstd1, which isn't > >> a udeb. > > > > It seems zstd is only an option for btrfs-progs, and I've just confirmed > > that setting --disable-zstd on the dh_auto_configure line lets btrfs-progs > > build just fine, without the libzstd1 dependency. As far as I can tell, > > there's no absolute need for this feature in d-i, and we could consider > > building the udeb without zstd support, instead of requesting the addition > > of a libzstd1-udeb. What do you think? > > > > That's an oversight on my part. From the recovery point of view, it > would be desired to have zstd compression support built-into > btrfs-progs-udeb such that one can use d-i recovery mode to > backup/restore btrfs filesystems with zstd compression. > > -- > Regards, > > Dimitri. Hi Cyril! I agree that it's not essential for d-i; although, it's worth mentioning that zstd seems like it will more likely satisfy users who are coming from zfs' "lz4 compression is usually beneficial" school of thought than lzo will. Dmitri, does btrfs send/receive actually require userspace libzstd? Or is it needed for defrag? (rewrites files, and also optionally applies, removes, or changes compression type) I'm guessing that btrfs-check requires it. (check --repair is, broadly speaking, equivalent to 'fsck.ext4 -p') The worst-case d-i bloat for an official arch seems to be 193.4kB for i386, before compression. Would you please 'reassign -1 libzstd' if you agree that Debian's rescue mode should support cloning/restoring subvolumes with compressed files, and/or being able to repair the (unmounted) rootfs? It seems strange that buster's initramfs has this functionality and that rescue mode doesn't. Alternatively, the btrfs binary in the udeb could be statically linked... Thanks, Nicholas signature.asc Description: PGP signature
Bug#1102604: debian-installer: Rescue mode can not execute shell when root filesystem is btrfs
clone 1102604 -1 retitle -1 "debian-installer: Rescue mode cannot execute a shell for any default btrfs installations" thanks Pascal Hambourg writes: > On 11/04/2025 at 03:21, Nicholas D Steeves wrote: >> >> Rather than reimplement our own thing for Debian Rescue, I think that it >> would be maximally beneficial to talk to the grub-btrfs >> (https://github.com/Antynea/grub-btrfs) project (and maybe >> btrfsmaintenance, and/or the new systemd-based one). or even btrfs-progs >> upstream about where this logic should be centralised. For example, >> what are the requirements for a bootable rootfs subvolume? >> >>/sbin/init, /etc/init, /bin/init, /usr/bin/init?, /bin/sh, /usr/bin/sh? >> >> I've read that's the order the kernel looks for things. Should we also >> look for /lib/modules? Now /usr/lib/modules because of usrmerge? Not >> to mention /lib/firmware...now /usr/lib/firmware because of usrmerge. > > This is not specific to btrfs and d-i rescue mode does not even do any > of these checks with any filesystem type. To be clear, I intend to implement this. It is not specific to btrfs; however, at this time there are only two file systems that support persistent alternative boot environments: ZFS and btrfs (I don't know if the new ntfs driver can do it). Ext4 support seems like it might be on the way, since some of btrfs' complexity was generalised to the whole fs layer upstream, and is now supported there. Would you please share what you think are useful (and/or not useful) criteria for these heuristics? Btrfs doesn't have fancy tooling like ZFS, LVM, Stratus, etc, so the question of "is this a rootfs" is established either with our choice of magic cookies and/or with heuristics. The objective and user-desired feature is a menu. The OP send me a PM instead of replying to this bug, so this may not yet be clear; multiple users have asked for a menu; it's common for users to have dozens, sometimes even thousands of subvolumes. I hope it's obvious why the candidates need to be automatically filtered and why a large unfiltered menu won't work for any but the most basic cases. Also, can you point me towards any documentation about the arcane d-i design? Most of my free time ends up going towards this object ends up going towards studying d-i rather than making the fixes. >> If grub-btrfs and Debian Rescue use different logic for determining >> viable boot environments, or if they order them differently, then many >> users will be confused, and various red-eyed Reddit avatars will gripe >> about our our "half-baked" solution. > > Do they serve the same purpose ? Is d-i rescue mode intended to be a > generic tool or primarily for reasonably "standard" Debian systems ? Yes. See below. >> What solutions have you thought of? > > For now, what about just this : > if the selected device filesystem is btrfs, then try to mount in order > - the @rootfs subvolume (Debian/Fedora style) If we're doing minimum defaults, then '@rootfs'. Fedora uses "root" rather than @rootfs. > - the @ subvolume (SUSE/Ubuntu style) But why? Neither SUSE nor Ubuntu are 'reasonably "standard" Debian system'. Also, rescue doesn't know whether these are non-standard Debian rootfs or if they're SUSE or Ubuntu (or Arch) installation, so do no harm. > - the default (or top level ?) subvolume (buster and older Debian style) There are two cases here. I've fixed one. If you mean a custom set-default-subvolume, then this is not supporting "reasonably standard" Debian installations either, because it's the definition of overriding a default, as well as an established consensus. > Patch: <https://salsa.debian.org/pham/rescue/-/tree/pham/btrfs_subvol> Thanks, I've merged and updated. > It should work with most standard Debian systems installed with d-i. There are only two standard cases, and now they're now covered. > PS: There is a flaw in partman-btrfs @rootfs creation. If using an > existing filesystem whose default subvolume is not the top level > subvolume, then @rootfs is created in the default subvolume instead of > the top level subvolume. If another @rootfs subvolume does not already > exist in the top level subvolume, then mounting @rootfs silently fails > and d-i writes target files in its own rootfs. > I admit that this is an edge case and probably nobody has ever hit it, > but the fix seems trivial: mount the top level subvolume instead of the > default subvolume before creating @rootfs. Patch available. Yes, there was consensus against supporting custom default-subvolumes, so I chose not to spend time supporting this foot-gun. Asamu Aoki had a system with the unsupported preconditions for what you're describing, and d-i aborted with an error as it should; however, the error was opaque. We don't attempt to support custom default-subvolumes at this time, and d-i should error usefully and abort. Kind regards, Nicholas signature.asc Description: PGP signature
Bug#1102604: debian-installer: Rescue mode can not execute shell when root filesystem is btrfs
Dropping kibi from CC, since he's super busy Holger Wansing writes: > Would it be possible, to ask the user for input, if the automatic tries > mentioned > in [1] above all fail? Cyril Brulebois writes: > Pascal Hambourg (2025-04-21): > >> Or a bigger change like what Nicholas intends to implement or Tyler >> suggested, which allows the user to select a subvolume ? > > I'd rather not. > It's been NACKED so I'm going to upload our two supported cases. I've tested them both. My understanding of solidarity in D-I Team right now is that less is more <3 Sorry for all the rubber-ducking over the years. Today is actually the first day when it became clear that if we have a strong grub-plugin and/or os-prober thing, then rescue-mode can remain simple. Kind regards, Nicholas signature.asc Description: PGP signature
