Hi, Quoting Michael Tokarev (2022-05-16 07:46:01) > > I'm not sure what you mean with "fall back". Maybe you mean that mmdebstrap > > reads /proc/sys/fs/binfmt_misc/qemu-$arch and then looks for an F in the > > "flags" field. If it finds that, then it will not copy qemu-user-static > > into the chroot (as was the old way to make this work). As you also pointed > > out, today this is automatic. > > No, I mean it is possible to try to execute a foreign binary directly, and > if that fails, do it again but this time, prepend "qemu-foo[-static]' to the > command line. Checking binfmt_misc flags is ofcourse too much. I'm asking > because of this: > > 00:00:00.266 E: + mmdebstrap --mode=root --variant=apt --architectures=arm64 > unstable /tmp/debian-chroot.tar http://127.0.0.1/debian > 00:00:00.599 E: I: arm64 cannot be executed, falling back to qemu-user > > I don't know what it will do with $options->{qemu} after this. I don't see > any reason to mess with qemu-foo-static in tools like debootstrap, - you can > run foreign binaries with it, but you can't - without binfmt_misc support - > you can't run maintscripts or other stuff because this requires executing > multiple binaries which you don't control.
Ah okay, yes this message wrong and misleading and a leftover from when
mmdebstrap still copied qemu-foo-static into the chroot. It doesn't do that
anymore. I'll fix the wording of this message in the next release.
But we still need to "mess" with some stuff:
- in proot mode, --qemu=qemu-$arch has to be passed to proot
- in fakechroot and chrootless mode we have to set QEMU_LD_PREFIX
In root and unshare mode we do nothing if the F flag is set in
/proc/sys/fs/binfmt_misc/qemu-$arch as explained earlier. The message should
indeed be adjusted to reflect reality. Thanks!
> > This brings me to a slightly related question: is it possible to disable
> > this transparent qemu-user-static support of running foreign architecture
> > binaries without the qemu-user-static inside the chroot? While this is
> > probably very useful in 99% of the cases, it is very annoying when I try to
> > cross build a source package and then I get a false positive when the build
> > (wrongly) tries to execute a foreign architecture binary and succeeds.
> > Thus, on my machine I'm constantly uninstalling and then re-installing
> > qemu-user whenever I want to do cross builds. Is there some config option I
> > can set to enable or disable transparent binfmt-misc emulation by qemu?
>
> Yes this is possible, but only at the binfmt_misc level.
>
> echo 0 > /proc/sys/fs/binfmt_misc/qemu-aarch64 -- disable this particular
> entry
> echo 1 > /proc/sys/fs/binfmt_misc/qemu-aarch64 -- enable it
>
> echo 0 > /proc/sys/fs/binfmt_misc/status -- disable whole binfmt support
> echo 1 > /proc/sys/fs/binfmt_misc/status -- enable it
>
> With systemd you can disable particular formats/architectures entirely
> by masking /usr/lib/binfmt.d/qemu-foo.conf in /etc/binfmt.d/ (if you remove
> binfmt-support).
Aha! I didn't know that one can write to those. I now see that the first line
of the contents of qemu-$foo changes from "enabled" to "disabled" and
vice-versa if I write 1 or 0 to it, respectively. Now this is also something
that I can check in other tools so that we don't try to cross build for
architectures on a platform that has qemu binfmt support enabled for the host
architecture. Thanks!
> > Then I also want to respond to some of the things you said in
> > #debian-devel:
> >
> >> 10:11 < mjt> but now I wonder if - after switching from binfmt-support to
> >> systemd-binfmt (and fixing
> >> this qemu-user-static bug), -- if the whole thing will work
> >> when installed in a chroot
> >> 10:13 < mjt> qemu-user should not register its binfmts when installed in
> >> chroot. binfmt-support didn't
> >> care about this, but I've no idea about systemd - maybe that
> >> one cares enough to actually
> >> fix this
> >> 10:15 < mjt> in other words: it worked in this context by accident not by1
> >> design
> >
> > What do I have to change to make it work again?
>
> This is a very good question. Two questions.
>
> First, I still don't know what happened. I tried to reproduce it locally but
> failed so far. Probably should try kernel from unstable to begin with, -
> I'm on bullseye. And I'm puzzled really, - because nothing had changed in
> this area in 7.0+dfsg-3 compared with the previous version, - as it turned
> out, even the thing which actually did change - the way it is being
> registered -
> does not work and hence does not affect this, and since you install
> binfmt-support
> explicitly, everything is exactly the same as before.
>
> And second, the whole thing of registering binfmts in a chroot is, um, wrong.
> This is because this affects whole system, inside this chroot, outside it, and
> in all other chroots too. And you install it in a chroot. It's okay to
> install
> qemu-user-static and register binfmts on the actual host system, but it should
> not be allowed to do in a chroot, because it affects whole system, and it
> changes execution domain for the actual host system!
I was about to tell you how to reproduce the problem by first creating a chroot
and then turning that into an image bootable with qemu so that you can see the
problem. But then it might've just failed because we started with a chroot and
that's somehow wrong (lets talk about that another time). So instead, here is a
reproducer where we do *not* go via a chroot:
$ qemu-img create -f qcow2 disk.qcow 4G
$ curl --fail
https://d-i.debian.org/daily-images/amd64/daily/netboot/debian-installer/amd64/initrd.gz
> initrd.gz
$ curl --fail
https://d-i.debian.org/daily-images/amd64/daily/netboot/debian-installer/amd64/linux
> linux
$ /usr/bin/time -v qemu-system-x86_64 -nographic -no-reboot -enable-kvm -m
1G -initrd initrd.gz -kernel linux -append 'console=ttyS0,9600,n8
auto-install/enable=true debconf/priority=critical
preseed/url=http://www.debian.org/releases/stable/example-preseed.txt
netcfg/get_hostname=hostname netcfg/get_domain=domain
passwd/root-password=r00tme passwd/root-password-again=r00tme
passwd/user-fullname=user passwd/username=user passwd/user-password=insecure
passwd/user-password-again=insecure pkgsel/run_tasksel=false
popularity-contest:popularity-contest/participate=false
grub-installer/bootdev=/dev/sda' -hda disk.qcow
This will create a bootable disk image using debian-installer with usernames
and passwords given via preseed options. You can boot the whole thing like
this:
$ qemu-system-x86_64 -nographic -enable-kvm -m 1G -hda disk.qcow
Log in with user:insecure, become root with password r00tme, edit
/etc/apt/sources.list replacing bullseye with unstable, run apt-get update,
then apt-get upgrade and then:
$ apt install mmdebstrap arch-test mount uidmap binfmt-support
qemu-user-static
Then reboot the thing just to be sure and then run:
$ mmdebstrap --arch=arm64 --variant=apt unstable /dev/null
...
I: extracting archives...
done
I: installing essential packages...
done
qemu: uncaught target signal 11 (Segmentation fault) - core dumped
E: run_chroot failed: E: env --unset=APT_CONFIG --unset=TMPDIR
/usr/sbin/chrod
W: listening on child socket failed:
I: removing tempdir /tmp/mmdebstrap.nBUnpBBsox...
E: mmdebstrap failed to run
root@hostname:/home/user# dpkg -l | grep qemu
ii qemu-guest-agent 1:7.0+dfsg-7 amd64
t
ii qemu-user-static 1:7.0+dfsg-7 amd64
)
Now you have a reproducer that installed qemu-user-static on a real system and
not a chroot and we observe the same effect.
Thanks!
cheers, josch
signature.asc
Description: signature
