Re: Bug#767999: base-files: fails to install with pre-jessie debootstrap
Guillem, KiBi, (It may be worthwhile taking a moment to read the bug logs of #766459 and #767999.) On Fri, Nov 07, 2014 at 8:34:49 +0100, Guillem Jover wrote: > Control: severity -1 serious > Control: retitle -1 dpkg: Correct fix breaks bogus assumptions in old > debootstrap > I'd say the bug title ought to be "Correct fix makes bug in base-passwd surface." > On Fri, 2014-11-07 at 02:33:48 +0100, Cyril Brulebois wrote: [...] > > Tests performed on an amd64 host running wheezy, debootstrapping amd64. > > > > Well I bisected the archive and the last debootstrapable jessie release > > was at 20141102T221202Z; looking at the set of updated packages between > > that one and the next one, it looks like… dpkg got updated from 1.17.13 > > to 1.17.21. And unsurprisingly reverting current jessie to dpkg 1.17.13 > > makes debootstrap work again. > To stick with Santiago's earlier wording, we are now shooting the next messenger (this time dpkg, after first working hard against base-files). > What's making the breakage in debootstrap surface is dpkg 1.17.20, commit > 9ee62ecfc8937f24a82805a424564997042dd984. At least with my testing > using a patched debootstrap, and using --foreign + --second-stage to > inject a dpkg with the reverted commit. > > > A few weeks or even days before the freeze doesn't quite seem to be the > > right time to introduce (not so) subtle changes in dpkg. > > (So we need to actually freeze months in advance of the freeze… right.) > > The above commit is a *correct* fix for a very old regression, to remove > a bogus package queue dependency stage in dpkg. The breakage in > debootstrap in older versions is due to incorrect assumptions, which > where fixed correctly (not worked around, contrary to what is mentioned > on its changelog) in 1.0.56. The change was: > > - x_core_install base-files base-passwd > + x_core_install base-passwd > + x_core_install base-files > > where debootstrap was expecting that dpkg run to process base-passwd > first, even though it was passed last. This, on a call to dpkg using > --force-depends, for essential packages that do not have any kind of > dependency relationship between them, which makes any such assumption > be based on something far beyond undefined behavior. > I do agree with all the dpkg reasoning and in a way I'm grateful that dpkg made this bug surface. But really there shouldn't be any such dependency on the order of configuration of base-files and base-passwd. > > Reassigning it > > to dpkg for now; and cc-ing the release team because of things like the > > #768346 unblock request. > > I'm going to revert the commit above (only in 1.17.x, it will be kept > in 1.18.x), because it is very minimal, just reintroduces again an > unnecessary package queue stage, and such regression is acceptable if > it makes buggy bootstrappers work again. But a fixed debootstrap (and > maybe cdebootstrap if that fails too) should really be pushed to stable. > I think you might want to hold off on this revert. See https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=767999#73 and Adam's kind testing of that patch. Best, Michael pgp2aK8N_1KIc.pgp Description: PGP signature
Re: Bug#767999: base-files: fails to install with pre-jessie debootstrap
On Fri, 07 Nov 2014, Guillem Jover wrote: > I'm going to revert the commit above (only in 1.17.x, it will be kept > in 1.18.x), because it is very minimal, just reintroduces again an > unnecessary package queue stage, and such regression is acceptable if > it makes buggy bootstrappers work again. But a fixed debootstrap (and > maybe cdebootstrap if that fails too) should really be pushed to stable. FWIW, cdebootstrap in stable is affected, it has been fixed in version 0.6.0 (see bug #737939). In Kali we were affected by this and our solution was to make base-files depends on base-passwd. I argued for this in #760568 first. Cheers, -- Raphaël Hertzog ◈ Debian Developer Support Debian LTS: http://www.freexian.com/services/debian-lts.html Learn to master Debian: http://debian-handbook.info/get/ -- To UNSUBSCRIBE, email to debian-dpkg-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20141107091431.gd6...@home.ouaza.com
Re: Bug#767999: base-files: fails to install with pre-jessie debootstrap
Santiago Vila (2014-11-07): > On Fri, 7 Nov 2014, Michael Tautschnig wrote: > > > To stick with Santiago's earlier wording, we are now shooting the next > > messenger (this time dpkg, after first working hard against base-files). > > Indeed. I would not like to see dpkg as the next "victim" of this problem. > > Being able to bootstrap jessie from wheezy is highly desirable, but > IMHO not something that should prevent us from fixing bugs in dpkg > when we could just apply the short and simple fix to the wheezy > version of debootstrap: > > - x_core_install base-files base-passwd > + x_core_install base-passwd > + x_core_install base-files Can we please stop the drama here? This isn't about shooting anyone. I meant to pinpoint the exact change which triggered the failure to debootstrap jessie, and I did. Then Guillem shared his point of view as a dpkg maintainer, which I'm fine with: dpkg's change was right, debootstrap's expectations weren't. Working around the wrong expectations in dpkg for the moment means sid can be fixed in hours or days, ditto for jessie. Fixing debootstrap's expectations in wheezy means we get a longterm fix (for when dpkg switches back to the current, correct behaviour). But this can't happen *immediately* as I already explained in an earlier mail, because of the need for p-u and a point release. As far as I can tell nothing speaks against having both dpkg patched and debootstrap fixed. And that's the current plan. Mraw, KiBi. signature.asc Description: Digital signature
Re: Bug#767999: base-files: fails to install with pre-jessie debootstrap
On Fri, 7 Nov 2014, Michael Tautschnig wrote: > To stick with Santiago's earlier wording, we are now shooting the next > messenger (this time dpkg, after first working hard against base-files). Indeed. I would not like to see dpkg as the next "victim" of this problem. Being able to bootstrap jessie from wheezy is highly desirable, but IMHO not something that should prevent us from fixing bugs in dpkg when we could just apply the short and simple fix to the wheezy version of debootstrap: - x_core_install base-files base-passwd + x_core_install base-passwd + x_core_install base-files Thanks. -- To UNSUBSCRIBE, email to debian-dpkg-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/alpine.deb.2.11.1411071125580.8...@cantor.unex.es
Re: debootstrap/base-passwd: #767999 and #766459 should really be fixed in base-passwd
On Thu, Nov 06, 2014 at 02:06:07PM +, Michael Tautschnig wrote: > At least Santiago's and my opinion diverge on whether base-passwd is presently > in line with policy on 3.8 Essential packages. Therefore the route from here > appears to hinge on interpreting policy in one of two ways: my point is that > base-passwd, at present, is not providing its functionality after just being > unpacked - it does require postinst having been run. Santiago claims, if I > interpret this correctly, that every package has to be configured at least > once > before being useful at all (irrespective of whether it is essential or not). On the one hand, I agree with Santiago's policy interpretation here, and have long maintained this in pretty much the same terms. I have always read that policy directive as being for the purpose of requiring Essential packages to behave safely during upgrades so that the problem of upgrading entire systems is tractable. That said, I don't particularly feel the need to stand on a precise interpretation here, and it's true that this problem can be dealt with easily enough in base-passwd. Michael's patch is textually longer than a debootstrap change, it's true, but it doesn't introduce any particular maintenance burden, and it weakens the constraints on configuring the Essential set correctly, which is something I'm generally in favour of. I've therefore gone ahead and uploaded this base-passwd change (after local testing): base-passwd (3.5.37) unstable; urgency=medium * Debconf translations: - Dutch (thanks, Frans Spiesschaert; closes: #765361). - Danish (thanks, Joe Hansen; closes: #765853). * Copy /etc/passwd and /etc/group from the master files in the preinst on initial install, to be more tolerant of bootstrapping tools that configure the Essential set in slightly different orders (based on a patch from Michael Tautschnig; see #766459 and #767999). -- Colin Watson Fri, 07 Nov 2014 15:44:29 + This doesn't preclude fixing this in depth in other ways (e.g. a debootstrap change in stable), so I'm not closing these bugs. I'll leave it to others to decide when/whether to do that. -- Colin Watson [cjwat...@debian.org] -- To UNSUBSCRIBE, email to debian-dpkg-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20141107154809.gs5...@riva.ucam.org
Re: Bug#767999: base-files: fails to install with pre-jessie debootstrap
Hi! On Fri, 2014-11-07 at 08:30:39 +, Michael Tautschnig wrote: > On Fri, Nov 07, 2014 at 8:34:49 +0100, Guillem Jover wrote: > > Control: severity -1 serious > > Control: retitle -1 dpkg: Correct fix breaks bogus assumptions in old > > debootstrap > > I'd say the bug title ought to be "Correct fix makes bug in base-passwd > surface." No, see below. > [...] > > I do agree with all the dpkg reasoning and in a way I'm grateful that dpkg > made > this bug surface. But really there shouldn't be any such dependency on the > order > of configuration of base-files and base-passwd. There needs to be one, and that's part of the problem of bootstrapping a system. I agree with Santiago that adding an implicit Depends completely defeats the point of Essential, and that's a wrong fix. > > > Reassigning it > > > to dpkg for now; and cc-ing the release team because of things like the > > > #768346 unblock request. > > > > I'm going to revert the commit above (only in 1.17.x, it will be kept > > in 1.18.x), because it is very minimal, just reintroduces again an > > unnecessary package queue stage, and such regression is acceptable if > > it makes buggy bootstrappers work again. But a fixed debootstrap (and > > maybe cdebootstrap if that fails too) should really be pushed to stable. > > I think you might want to hold off on this revert. See > > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=767999#73 > > and Adam's kind testing of that patch. The reasoning there is not correct, Santiago is right except for the part where he might be trying to justify this through policy, because bootstrapping a system is currently outside the realms of policy. Many of the packaging system invariants do not hold there. What debootstrap does is more or less this (with other setup sprinkled in between): Extract .debs - dpkg-deb --extract Install core .debs - dpkg --install --force-depends Unpack required .debs - dpkg --unpack --force-depends Configure required .debs - dpkg --configure --pending --force-configure-any --force-depends Install dependencies of pre-depencies for base system .debs - dpkg --force-overwrite --force-confold --skip-same-version --install Install base system .debs - dpkg --force-overwrite --force-confold --skip-same-version --install Configure base system .debs - dpkg --force-confold --skip-same-version --configure -a There are several stages where for example preinst scripts have never been run for any Essential packages, --force options are used all over the place; this should be a very clear indication debootstrap has to operate in a completely different environment and conditions. For example, dpkg will fail flat out if there is no status file, and that cannot be shipped in the .deb; dpkg will create it if not present in its postinst (but that's really too late when bootstrapping a system), so debootstrap needs to create these instead, and this is a very similar situation to the one with the passwd file, but in that case a better solution is to hardcode an install order and offload the work to base-passwd. Someone needs to know that that file has to be created somewhen or by something. ISTR there was in the past discussions (AFAIR either in d-d or a dpkg bug) about trying to move the bootstrapping information into packages in a bootstrap maintscript or similar. Those would need to be run from outside the chroot, so that we are not back to the problem of implicit assumptions and ordering though. And the expectations on the external environment would need to be specified, for example assuming just POSIX utilities. *That* would be a proper fix to the problem of the implicit ordering, would also be a generic solution independent of the distribution or derivative, or current set of packages, and we might be able to have (possibly) a more generic debootstrap. I can try to draft something up if people are interested in this for jessie+1. Thanks, Guillem -- To UNSUBSCRIBE, email to debian-dpkg-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20141107163806.ga28...@gaara.hadrons.org
Re: Bug#767999: base-files: fails to install with pre-jessie debootstrap
Hello Guillem, All, First of all thanks to everyone for the efforts to fix these problems. It seems we've now got fixes in place both in (c)debootstrap and base-passwd, so hopefully we're fine for the next few releases... :-) On Fri, Nov 07, 2014 at 17:38:06 +0100, Guillem Jover wrote: > On Fri, 2014-11-07 at 08:30:39 +, Michael Tautschnig wrote: [...] > > I do agree with all the dpkg reasoning and in a way I'm grateful that dpkg > > made > > this bug surface. But really there shouldn't be any such dependency on the > > order > > of configuration of base-files and base-passwd. > > There needs to be one, and that's part of the problem of bootstrapping > a system. I agree with Santiago that adding an implicit Depends > completely defeats the point of Essential, and that's a wrong fix. > I don't quite see why we would necessarily need a dependency between the two, and the change to base-passwd seems to prove this. (But I understand that explicitly adding a dependency would not be a good idea.) [...] > ISTR there was in the past discussions (AFAIR either in d-d or a dpkg > bug) about trying to move the bootstrapping information into packages > in a bootstrap maintscript or similar. Those would need to be run from > outside the chroot, so that we are not back to the problem of implicit > assumptions and ordering though. And the expectations on the external > environment would need to be specified, for example assuming just POSIX > utilities. > I suppose it was part of those discussions (I wouldn't recall having followed them) that it is not possible to sort out these problems using preinst scripts. > *That* would be a proper fix to the problem of the implicit ordering, > would also be a generic solution independent of the distribution or > derivative, or current set of packages, and we might be able to have > (possibly) a more generic debootstrap. I can try to draft something > up if people are interested in this for jessie+1. > While obviously implicit constraints are worse than explicit ones, having no ordering constraints would seem even better?! I suppose this is infeasible for certain packages, so for now I'll just enjoy that the count has been reduced by one. Thanks again everyone for the efforts, Michael pgp_n2U7mdFCg.pgp Description: PGP signature
