Hi Santiago, Quoting Santiago Vila (2019-03-12 23:43:02) > > > Do any of them still don't know that base-passwd should be configured > > > first because otherwise any other package using root (be it base-files or > > > any other) will fail? I think this was already settled in the last > > > discussion we had about this several years ago. > > multistrap doesn't take care of this and you can provoke a > > base-files.postinst failure this way. > In such case I would say that as a bootstrapping tool it's not doing its job > properly. > > The first rule of a bootstrapping tool is that it has to work. > (And there are actually no other rules. As far as it does its job, > you are allowed to do all sorts of dirty hacks). > > Bootstrapping tools exist so that we don't have to worry about > dependencies on essential packages. It has always been my opinion that > if we start to do hacks here and there so that bootstrapping tools > work properly, we are already doing it wrong. > > > [,,,] Just because debootstrap encodes a ton of hacks to make things barely > > work (and break every so often) doesn't mean we have to maintain them until > > eternity. > I would say: Just because people writing new bootstrapping tools seem to > forget the lessons learned from previous bootstrapping tools, we have to > learn again what bootstrapping really means: It's not adding hacks to the > normal packages, it's concentrating all the hacks in the bootstrapping tools, > so that we can keep ordinary packages clean of hacks. > > (Or at least that's what I think it was the idea behind the essential > definition).
I'm writing as the author of another "bootstrapping tool", namely mmdebstrap which currently (really accidentally only) does not suffer from this bug but might happen to be affected by it in the future. I would especially like to challenge your view that (if I understood it correctly) all the magic for getting from nothing to a Debian chroot should be encoded in a bootstrapping script or program. You are right that currently this is how it's done but I'd like to convince you that maybe it would be worth reevaluating how we are currently doing this because maybe there is a better way to do it. Let me also point out here, that debootstrap has a place even in a world where all the magic (the things you call hacks twice above) were moved into the individual packages instead of being encoded in a central bootstrapping script. This is because we will still need something that would be considered a hack from the Debian perspective when we create a Debian chroot on systems that are not Debian. My tool mmdebstrap for example will likely only work on Debian or derivatives because it requires apt to be installed. Lets look at the bigger picture: we have an archive with multiple software components we call packages on some remote server and locally we have a fresh empty directory which we somehow want to populate with the functionality that these software components provide. The question is: what would be the best way to do it? Or what would be a good engineering solution to this problem? I claim (and that's where I want to challenge your view) that a better engineering solution than the status quo in Debian would be to let each of these software components (or packages) encode their requirements in a way that lets a bootstrapping tool solve the installation task without involving the other components and thus solve the chicken-and-egg problem of bootstrapping. If this were possible, we can achieve a number of benefits which I'll be getting to in a bit. That this *is* possible is shown by [guillem]'s mail to this bug report. Another approach would be to use functionality of packages installed on the outside of the chroot to fulfill requirements of the packages inside the new chroot [detached chroot]. [guillem] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=924401#35 [detached chroot] https://wiki.debian.org/Teams/Dpkg/Spec/InstallBootstrap So if you can accept that it is technically achievable to create a component based operating system with only very little "magic" in the bootstrapping tools but all required information about interaction of even essential packages in the components itself, then we can achieve multiple benefits: - solving dependencies is a complex matter [sudoku] and implementing a software that can handle NP-complete [np-complete] problem instances is not trivial (see code of apt, aptitude, dose, aspcud...). Existing tools like (c)debootstrap only understand just enough about package dependencies that they happen to be able to install a very minimal system. The status quo is to not add many packages via the --include option but install them via apt inside the chroot after debootstrap is done (#768062, #878961). If we could use existing tools that have a better understanding of Debian dependencies (like apt, aptitude, dose...) for bootstrapping a Debian chroot then we could: * create chroots with multiple architectures, honoring multi-arch * create chroots with packages from multiple mirrors * add all desired packages in the initial program invocation instead of requiring a multi-step process that requires some guess work about which dependency situations might be too hard Especially supporting multiple mirrors would be a great benefit because it would allow to create a chroot of a Debian stable release that includes the updates and security mirrors *and* is up-to-date without requiring extra steps (#543819, #762222) Or how about multiple suites? (#690210) [sudoku] https://lists.gforge.inria.fr/pipermail/dose-devel/2013-December/000059.html [np-complete] https://xavierleroy.org/publi/edos-frcss06.pdf - The idea behind component based software distributions like Debian is, that each package declares how it relates to other packages in the distribution. To solve dependency problems and perform package installations we do not maintain a central registry or script but it is up to each individual package of how it can be installed via the dependencies, conflicts and maintainer scripts it provides. We would generally think it wrong if we would solve a dependency bug by implementing that apt now encodes that to install a specific package XY it has to do some special thing. But so far, this kind of argument about good engineering practice is not applied to the debootstrap level. There, we (currently) accepted that it is okay that debootstrap employs all sorts of hacks to be able to populate a directory with a minimal chroot and we even accept that sometimes individual packages have to change just because debootstrap would otherwise not be able to handle it. This kind of design has numerous downsides: * debootstrap has to carry workarounds even long after they are not anymore required by packages in the current stable release so that old-(old-)stable releases can still be debootstrapped * package maintainers have to go through debootstrap before an issue can be fixed instead of just modifying their package * debootstrap has to implement these workarounds for each derivative and each derivative has to go to the debootstrap maintainer before their distribution is supported * workarounds can be architecture specific, so the set of supported architectures is (again) not limited by what is implemented in the packages but in debootstrap * all these workarounds must be implemented by all debootstrap-like software. So in essence, debootstrap, cdebootstrap, multistrap, and mmdebstrap will all have to implement all workarounds for all architectures and derivatives and try to all keep up to date But what if we would extend the clean design choice that we made for normal package installation to the debootstrap-phase? We would then be able to reap the benefits of this approach to developing a component-based software distribution even at the very early debootstrap-phase: * debootstrapping Debian and derivatives that choose to follow the design that we yet have to decide upon could create a chroot of Debian or a derivative just by supplying one (or more) mirror urls to the debootstrap tool. All distribution-specific magic would be handled by *the packages* and their metadata alone. * workarounds would live in the packages themselves and would only live there for as long as they are required for the distribution they are in and for the architecture they are for * not one single point that de-facto determines how the debootstrap process has to look like and thus no argument like: "debootstrap cannot do this and that, so we have to add this workaround to that package" * process is defined by declarative metadata and not by turing-complete scripts * downstream derivatives do not need to touch debootstrap if they want to change how their system is set up but only change the affected packages -- this keeps the code local to the place that is affected * multiple debootstrap-like tools do not all have to implement the same magic multiple times and all have to be kept up-to-date when anything in any derivative or architecture changes * easier bootstrapping of new architectures from zero as we get rid of all Essential:yes packages have to already exist What I'm listing here are just the benefits of component based software design and we currently apply it with great success to normal package installation for many, many reasons that we all accept and like. My point is: why not extend all these good practices of component based software distributions to the debootstrap phase? It is certainly technically possible. I'm not advocating for doing "hacks here and there so that bootstrapping tools work properly" as you put it but that we think about the question of whether maybe there is a better way to populate an empty directory with software components that does not require as much magic as currently has to be supplied by tools like debootstrap. The result would not be "hacks here and there" but a cleaner and more robust setup procedure. So what I want you to take away from this long mail is: maybe we should re-think the way we are currently creating a Debian chroot because maybe there exists a different approach that would give us benefits that are actually important to our users and make maintaining the universal operating system easier? Thanks! cheers, josch
signature.asc
Description: signature
