Hi! On Thu, 2023-11-02 at 15:27:54 +0000, Michael Hudson-Doyle wrote: > On Tue, 31 Oct 2023 at 09:21, Guillem Jover wrote: > > This can be used among other things to set up foreign chroots, by > > running the host tools, so disallowing installation could be > > problematic. Even though I guess there could be a warning about this, > > or maybe it could be controlled through a force option, although both > > seems like they could be disruptive.
> Of course in such cases dpkg knows that something funny is going on and > could suppress the warning itself. Right, also true. > I spent a few minutes trying to think hard about this and I honestly don't > think I can predict whether trying to prevent installation of incompatible > packages is worth it (after all one of the ways users could get into > trouble would be moving an installed system to a different CPU and having > binaries start to fail and obviously dpkg can't help there). > > One result of this thinking was: I had been thinking/assuming the issue of > which variants to consider would be apt configuration, but maybe dpkg > configuration would make more sense (after all, --add-architecture is a > parameter to dpkg). And in this case, dpkg could object when installing a > variant that has not been configured. Yes, the "plan" has been to add support for run-time CPU attributes, so that when adding a new arch, for example you can specify whether that arch is runnable, which could help dpkg decide whether to allow by default to install M-A:foreign packages. I guess this is similar, so such future interface should probably take this into account as something to support too. Will check where this is tracked and add a note to it. And of course that is fine as a guardrail, but if a user hit that out of running a frontend, then that would already be too late, which to me means that frontends need to be aware of this too (and not pass packages that dpkg would/could/might refuse to install), when deciding what to pass to dpkg. But in any case, as you say, this currently would not be worse than configuring a foreign arch, installing some foreign package and trying to run it, but it might make it potentially more common. And as mentioned above the effecting layer this needs to be decided up seems higher anyway (even if dpkg could provide the infra for it). > > If the only change in the package filename format is in the <arch> part > > where we'd use a name which would otherwise be valid as an arch name (so, > > no weird symbols, or «-» separators that are not intended to split <os> > > and <cpu> or similar), then using a name for the variant/ISA would be > > fine. > Right. I think that (when possible pretending e.g. "amd64v3" is a distinct > architecture will generally make things easier. E.g. I think britney > wouldn't need to know about the relationship between "amd64" and "amd64v3". I guess that depends on whether the intention is to create a full optimized archive, or just a partial overlay one. In the latter case then it might need to know to be able to satisfy dependencies. > > That would be one solution yes, which could give automatic bijective > > mappings, although ideally with a machine-readable way to get at it, > > which I'm not sure we have currently. > I think "gcc -Q --help=target | grep -e '^\s*-march'" is about as machine > readable as it gets currently, for better or worse (mostly worse). That does not look very satisfactory, though. And llvm/clang does not support it. :/ > > For example code in dpkg-dev > > already runs «$CC -dumpmachine» to infer the host architecture to use > > during builds. > > > > While using a triplet variation could be a way to do that, that would > > require such triplet support for each variant/ISA, which tends to be > > very painful to introduce if it's not there already, so I'd not > > consider this specific way a viable option. > > I admit I'm not an expert on triplet intricacies but I think a new triplet > is not appropriate here (a bit like a new Debian architecture for a > variant/ISA choice is not the right concept). We have i386 or arm (?) as (bad IMO) examples where the triplet can define the arch baseline. The problem is that this requires updating the GNU config.git upstream, and then getting that to trickle down into every package that might be using autotools and not using autoreconf at build time, or to even update triplet matches in configure scripts and similar, which might be "acceptable" for a new arch, but seems disproportionate for a new ISA, so yes, as mentioned I agree it's not viable. > > On Thu, 2023-09-21 at 14:43:42 +1200, Michael Hudson-Doyle wrote: > > > * Should the ISA influence the toolchain via toolchain defaults or > > > dpkg-buildflags? > > > * How is the default ISA for a buildd chroot selected? > > > > So the clear downsides of either modifying the default toolchain or > > having to provide an additional one is that this seems pretty heavy > > weight. Also because people might want to build optimized variants > > locally w/o having to mess with their already existing toolchains. > > (I'm not sure whether something going along the lines of > > <https://git.hadrons.org/cgit/debian/fakecross.git> could be an > > option, although as mentioned above, if that would imply new triplets, > > then probably not.) > > > > So the easiest way might indeed be by controlling this via an envvar, > > DEB_HOST_ARCH_ISA? Yeah, that works, and follows the current DPKG_*_ARCH_ABI lead for example. > > which dpkg-buildpackage could also setup internally via a new option, > > say --arch-isa=amd64v3 or similar > --host-arch-isa would be more coherent I think. Ah absolutely! For some reason had --arch in mind as a valid option (I only see it now in dpkg-scanpackages :D, or maybe I was thinking about --host-isa :). > I guess one could add support for --target-host-arch-isa to build a > toolchain that defaults to a particular ISA. But well. Yes, the ISA support in dpkg should be extensive enough (so that if this needs to be supported in the toolchain, then it is possible). > So to summarise, here are the generic changes that I think need to be made > to src:dpkg to support variant ISAs as a thing: > > * add get_host_arch_isa() to Dpkg::Arch Yes (perhaps as mentioned below also just get_host_isa()). > * dpkg-gencontrol records DEB_HOST_ARCH_ISA into DEBIAN/control as > ArchitectureIsa Probably better Architecture-Isa, otherwise the current automatic case folding would make it end up as Architectureisa. > * dpkg-architecture emits DEB_HOST_ARCH_ISA and grows --host-arch-isa flag Also DEB_BUILD_ARCH_ISA and DEB_TARGET_ARCH_ISA, and also grows a --target-arch-isa (but I'm thinking whether the shorter --host-isa would be nicer, for example the --match-bits are not spelled --match-arch-bits, which would seem also a bit redundant). > * dpkg-buildpackage passes --host-arch-isa to dpkg-architecture Yes, but only when not the baseline. > * dpkg-genchanges should record the ISA in the changes file somehow I > guess? Yes, also dpkg-genbuildinfo. This could be done either from the envvars, or perhaps through the debian/files attributes support. But given that this is supposedly build global (I think it would be rather weird to end up with a .changes including say _amd64.deb and _amd64v3.deb file references from the same build), then probably using the envvar might be the better way. > * dpkg-deb records the ISA in the file name Yes. > Have I missed anything? Nothing else comes to mind right now (except what I might have already added). > (Hmm does anything need to reject unknown values > found in DEB_HOST_ARCH_ISA / --host-arch-isa? Probably...) I guess it indeed makes sense to define what ISAs are supported, and either error out or warn and ignore such values. So there might be a need to add something like a new data/isatable. > Conceptually slightly separately, it might make sense to add a build > "feature" to Dpkg::Vendor::Debian to allow setting -march (and -mtune?) > > Then when we want to add support to an ISA, we add a little thing to > set_build_features (in either Vendor::Debian or Vendor::Ubuntu or wherever) > that maps get_host_arch_isa() to values for the march-influencing feature. Hmm, right, how to hook this. I'm not sure the current interface is good enough to describe this via build flags features, because such new feature area would expose arch-specific features. I have been thinking through the build flags and will try to send a proposal/RFC to revamp parts of it during the weekend. But I think the ISA stuff is better just handled (at leas for now) directly by injecting whatever flags when the requested ISA is different to the baseline. Thanks, Guillem
