On 26.08.21 11:39, Lucas Nussbaum wrote:
> Additionally, one could imagine a DSL to:
> - make minor changes to the source package before building (adjust
> dependencies, apply an additional patch, etc.)
> - tell sbuild that some build-dependencies must be pulled from backported
> packages
>
> Jelmer, did you already think about that? Is there a way one could help
> you?
I worked on something like this last fall, which I tentatively named
'apt-derive'.
Given a source package in some origin distribution, apt-derive can
automatically create a derivative of the source package, and of any
necessary dependencies, for a target distribution.
The popular derivation would be the backport type, of course. But other
possibilities exist. For example, I was interested in optimized build
flavors for certain target environments.
Here's sample code for a numpy buster backport, with the following
output reflecting the rebuild chain needed on amd64 circa September
2020, when I ran this:
| from apt_derive import RebuildResolver
|
| # Used to identify Releases in python-apt
| bullseye = dict(origin='Debian', codename='bullseye')
| buster_bpo = dict(origin='Debian Backports', codename='buster backports')
| buster = dict(origin='Debian', codename='buster')
|
| # origin: bullseye
| # target: buster-backports
| # Packages from buster can be used to satisfy build dependencies
| rr = RebuildResolver(wanted_dist=bullseye,
| target=buster_bpo,
| added_dists=[buster])
|
| # Figure out what needs to be rebuilt
| tree = rr.resolve('numpy')
| if tree:
| print(tree.pformat())
python3-numpy_1:1.19.2-2
└─cython_0.29.21-1
└─dvisvgm_2.10.1-1
└─freetype_2.10.2+dfsg-3
└─python-hypothesis_5.32.1-1
└─sphinx_3.2.1-1
└─python-lark_0.9.0-1
└─sphinx_3.2.1-1
So back then, 7 unique packages needed to be backported (numpy and 6
others for newer build dependencies).
That's the resolving part of apt-derive. There's more to it of course,
for example there is a "blocked" list of packages where automatic
derivation would be refused (eg: for libc).
The rebuild part of apt-derive would create backported sources (with
changelog entries, and any other configured extensions) for the
packages.
My plan was to provide such a automated backports service under the name
"autobpo", with package suffix ~autobpo11, and suites
* bullseye-autobpo-dangerous for the immediate build results
* bullseye-autobpo for results that passed CI and rebuilt
reproducibly
As to which packages to automatically backport, my plan was two-fold:
* By popular request
* Maintainers indicating so with X-AutoBPO-Policy: <something>
somewhere in debian/control
What eventually stalled my effort was that I was lacking a reasonably
simple build environment for multiple architectures. I wrote sbuild-qemu
to get local access to many architectures, but in the end, I didn't see
a way around wanna-build etc. for a multi-node build farm. I found
wanna-build overly complicated, and then other stuff came up in $LIFE.
Raphael initiated debusine in the meantime, which seems to be ideal
solution to my build problem.
However, If anyone with wanna-build experience is interested in
collaborating to get this launched sooner, please ping me.
I'll try to finalize the first draft of apt-derive as soon as possible
in the hopes that something can come of it.
Best,
Christian
