Hi fellow cross/toolchain people,
you know I'm looking into making packages cross build and today I
happened to look into fftw3. What sounds like any other package turns
out to be a very interesting one, because fftw3 uses Fortran. It happens
to declare Build-Depends: mpi-default-dev, which happens to declare
Depends: libopenmpi-dev, which happens to declare Depends: gfortran-14 |
gfortran-mod-15. Since the entire chain is architecture dependent
packages with no Multi-Arch: foreign nor :any annotations, what we end
up requesting here is the host architecture gfortran-14 native compiler
and that of course is bad for cross compilation.
Earlier this year, Matthias merged and fixed (thanks a lot!) my
gcc-for-host patches. What we'd want here is some fortran compiler for
the architecture of the libopenmpi-dev package. Now that we have those
fancy -for-host packages it's just a matter of writing
gfortran-14-for-host, right? Unfortunately, no.
I learned the hard way that the semantics provided by the -for-host
packages do not match the practical requirements. If one installs just
the -for-host package, the provided interface really is just the
triplet-prefixed compiler and you don't get the bare one. However, when
performing a native build, what really is getting called is the bare one
and that goes missing. So what we really really want here is different
semantics. If installing a -for-host package, the expectation from most
build systems really is that we can call the triplet-prefixed tool and
in addition to that, but also if we are doing a native build we want the
bare names to work. However, the bare names are not provided for the
native case by the -for-host package.
This sounds easy enough. A -for-host package simply Depends on some
-<triplet> package and when generating that package, we might just add a
Depends: ...-for-build whenever the compiler is native (i.e. the triplet
matches the package architecture). I think this would technically work
in the first instance, but it breaks another use case.
More and more we're running into the problem that 32bit builds run out
of address space and the only final solution to this problem is using a
64bit compiler executable. For example, we'd like to do an i386 build
using gcc-i686-linux-gnu:amd64. From an autotools (and other build
systems) point of view, this is a native build. Hence, the expectation
is that cc and gcc exist and build for i386, but
gcc-i686-linux-gnu:amd64 would not depend on gcc-for-build in that
scheme and gcc-for-build. In fact, gcc-for-build would depend on gcc,
which would depend on gcc-i686-linux-gnu with a (= ...) version
constraint that is never matched by gcc-i686-linux-gnu:amd64.
How do we want to deal with this?
I see the following options:
1. Remove the assumption that you can run bare toolchains from
build-essential. Expect that all toolchain invocations correctly
spell out their architecture. For most build systems, this amounts to
exporting a suitable CC variable or equivalent:
* dpkg's buildtools.mk would always emit prefixed toolchains.
* The debhelper makefile buildsystem would pass a triplet-prefixed
CC for all builds, not just cross builds.
* The debhelper autotools buildsystem would export a
triplet-prefixed CC for the configure invocation.
* meson env2mfile would always emit a triplet-prefixed compiler.
* The debhelper cmake buildsystem would always pass a
triplet-prefixed CMAKE_C_COMPILER (and friends).
With these changes, I expect that we could make about 80% (wild guess)
of packages just work, the remaining ones are many and often hard to
fix (e.g. we completely break any use of mpicc).
2. Ignore the large address space for 32bit use case and just add the
bare toolchain names to the API for -for-host whenever it happens to
be native.
3. Dynamically manage the bare toolchain names using maintainer scripts.
This is a bit tricky. We currently install the gcc-14 ->
<triplet>-gcc-14 symbolic link in gcc-14. This package is important
for a few rare use cases where the compiler architectures must match
the host architecture (e.g. for using plugins). We'd stop installing
this by default and move the gcc-14 link into gcc-14-<triplet>
package. There it would not be part of the data.tar and instead be
managed using a maintainer script that creates the link if the native
architecture happens to be the triplet of the package.
As a result, dpkg -S /usr/bin/gcc-14 would no longer report an owning
package. Since toolchains are diverted in a number of places, the
script would have to use dpkg-divert --truename for creating the
link.
Once the bare links are created in this way, the -for-build packages
would be updated to no longer depend on the bare packages and instead
depend on the matching -for-host packages. For instance,
gcc-14-for-build would change its dependency on gcc-14 to
gcc-14-for-host. Since gcc-14-for-build is an Arch:all package, it
would require e.g. gcc-14-for-host:i386 when installed on i386. If
the dependencies of it are satisfied using
gcc-14-i686-linux-gnu:amd64, the maintainer script would create the
bare tool links.
To make matters worse, the actual gfortran dependency of libopenmpi-dev
is emitted from dh-fortran-mod. If we were to change dh-fortran-mod to
change fortran:Depends to emit gfortran-for-host right now, a lot of
packages would FTBFS as they'd expect gfortran and only find
<triplet>-gfortran.
So this little adventure of fftw3 turned out to yield quite fundamental
insights (which really is why I am doing this). I now ask readers of
this mail to verify my analysis. Do you concur with the implications
here? Do you see any other options to meet the requirements? Do you have
a preference for moving this forward?
Helmut
