Hi, sbuild is our primary tool for constructing a build environment to build Debian packages. It is used on all buildds and for a long time, the backend used with sbuild has always been schroot. More recently, a number of buildds have been moved away from schroot towards --chroot-mode=unshare thanks to the work of at least Aurelien Jarno and Jochen Sprickerhof and a few more working more behind the scenes for me to spot them directly.
In this work, limitations with --chroot-mode=unshare became apparent and that lead to Johannes, Jochen and me sitting down in Berlin pondering ideas on how to improve the situation. That is a longer story, but eventually Timo Röhling asked the innocuous question of why we cannot just use schroot and make it work with namespaces. That lead me to sit down and write a proof of concept. As a result, we now have a little script called unschroot.py that vaguely can be used as a drop-in replacement for schroot when used with sbuild. In trixie and bookworm-backports it can now be plugged into sbuild by setting $schroot = "path/to/unschroot.py" thanks to Johannes. It's not that long and can be viewed at https://git.subdivi.de/~helmut/python-linuxnamespaces.git/tree/examples/unschroot.py. It is vaguely close to reaching feature-parity with sbuild --chroot-mode=unshare and operates in a very similar way. As it is now, it doesn't bring us any benefits beyond separating the containment aspect from the build aspect into different tools. The split into different tools is important in my view. I argue that it allows easier experimentation and its architecture may enable features that were difficult to implement using sbuild --chroot-mode=unshare as sbuild is significantly becoming a container runtime of its own and there things start to get messy. Is this a path worth pursuing further? Would we actually consider moving back from sbuild --chroot-mode=unshare to sbuild --chroot-mode=schroot with a different schroot implementation? Related to that, what would be compelling features to switch? Let me go a bit further into detail. There are two approaches to managing an ephemeral build container using namespaces. In one approach, we create a directory hierarchy of a container root filesystem and for each command and hook that we invoke there, we create new namespaces on demand. In particular, there are no background processes when nothing is running in that container and all that remains is its directory hierarchy. Such a container session can easily survive a reboot (unless stored on tmpfs). Both sbuild --chroot-mode=unshare and unschroot.py follow this approach. For comparison, schroot sets up mounts (e.g /proc) when it begins a session and cleans them up when it ends. No such persistent mounts exist in either sbuild --chroot-mode=unshare or unschroot.py. The other approach is using one set of namespaces for the entire session. Practically, this implies having a background process keeping this namespace alive for the duration of the session and talking to it via some IPC mechanism. We may still spawn a new pid namespace for each command to get reliable process cleanup, but the use of a persistent mount namespace enables the use of fuse2fs, squashfuse, overlayfs and bindfs to construct the root directory of the container by other means than unpacking a tar into a directory. In particular, the use of bindfs allows sharing e.g. the user's ccache with the build container in principle (with proper id shifting). At the time of this writing, this second approach is wishful thinking and not implemented at all. I merely believe that it is implementable with the schroot API already implemented by unschroot.py above. Another possible extension is a hooking mechanism. Regular schroot has hooks already and I've seen requests for sbuild to use package-specific chroots. For instance, one may have a separate Haskell or Rust container that already has a basic set of ecosystem-specific dependencies to speed up the installation of Build-Depends. On-demand updating chroots also have been requested. However, it's not clear to me what a useful interface e.g. unschroot.py could provide for such hooking yet and I invite you to provide more use cases for such hooking. Also sketching how you imagine interfacing with this would be helpful. For instance, you may explain what kind of configuration files or options you'd like to use and how you imagine them to work. I note that this is not a promise that I am going to implement your wishes. I intend to do more work on this and barring really useful extensions, my next goal would be moving to that other approach. Please allow me to thank Freexian for supporting part of this work financially even though it has been my initiative and is not otherwise influenced by Freexian at the time of this writing. Let me also explain the relation between "unschroot.py" and the containing repository "python-linuxnamespaces". linuxnamespaces is a (probably) distribution-agnostic Python module providing plumbing functions for constructing container runtimes written by myself for lack of better alternatives. As such unschroot.py in large parts uses linuxnamespaces (the Python module) to plug together the various parts needed to arrive at a container useful for building with sbuild. If unschroot takes off, it likely needs to get its own home. linuxnamespaces is supposed to enable constructing a systemd-as-pid-1 container as a regular user, but doesn't do that as of yet. While podman and docker allow running unprivileged application containers, they still require privileged containers when you want to run systemd-as-pid-1. Helmut
