Re: Build and run-time triplets
Well, here's an update on this old thread... On Thu, Jun 09, 2022 at 01:03:25PM +0500, Andrey Rahmatullin wrote: > On Thu, Jun 09, 2022 at 08:42:28AM +0100, Julian Gilbey wrote: > > > > I'd like to ask for some help. I'm working on packaging pydevd, which > > > > builds a private .so library. Ordinary extensions built using cython > > > > or similar end up being called "foo.cpython-310-x86_64-linux-gnu.so", > > > > but this library, which is not dependent on the Python version, should > > > > presumably be called "bar.x86_64-linux-gnu.so". > > > If it's just a private library and not a Python module it should be called > > > bar.so. > > > > > > > Question 1: How do I determine (within Python) the triplet to use when > > > > building the library? > > > You don't. > > > > > > > Question 2: How do I determine (within Python) the triplet to use when > > > > loading the library at runtime? > > > You don't, but also how are you actually loading it? > > > > Well, the upstream wanted to compile two versions of the library, one > > for 64 bit architectures and one for 32 bit architectures. I don't > > really want to build two different arch libraries in a single build, > > because that seems very contrary to the way the Debian architectures > > work, and would also limit it to the amd64/i386 architectures for no > > obviously good reason. I had imagined that if there is some sort of > > multiarch setup, one might have the amd64 and i386 packages installed > > simultaneously, hence the need for different names. > The normal way for this is putting it into > /usr/lib//pkgname/foo.so, and according to the code below you'll > need the full path at the run time so you indeed need the triplet at both > build and run time. You can get the triplet in d/rules, not sure how > should you pass it to the build system as that depends on the build system > used. For the run time, https://wiki.debian.org/Python/MultiArch suggests > sys.implementation._multiarch (note that you cannot use it during the > build as that would break cross-compilation etc.), not sure if there are > better ways. I got all of the triplets working, but was then stymied when I tried to specify Multi-Arch: same in the control file. I got a lintian warning: multi-arch-same-package-calls-pycompile It seems that since the pybuild system (via dh_python3) adds a py3compile command to the postinst of the package, then I can't safely use Multi-Arch: same. I don't know if this is the case for all python3 Arch: any packages with compiled extensions; are they all effectively Multi-Arch: no? Is this worth thinking about in the longer term? Best wishes, Julian
Re: Build and run-time triplets
On Sun, Jul 24, 2022 at 06:36:57PM +0100, Julian Gilbey wrote: > > > > > I'd like to ask for some help. I'm working on packaging pydevd, which > > > > > builds a private .so library. Ordinary extensions built using cython > > > > > or similar end up being called "foo.cpython-310-x86_64-linux-gnu.so", > > > > > but this library, which is not dependent on the Python version, should > > > > > presumably be called "bar.x86_64-linux-gnu.so". > > > > If it's just a private library and not a Python module it should be > > > > called > > > > bar.so. > > > > > > > > > Question 1: How do I determine (within Python) the triplet to use when > > > > > building the library? > > > > You don't. > > > > > > > > > Question 2: How do I determine (within Python) the triplet to use when > > > > > loading the library at runtime? > > > > You don't, but also how are you actually loading it? > > > > > > Well, the upstream wanted to compile two versions of the library, one > > > for 64 bit architectures and one for 32 bit architectures. I don't > > > really want to build two different arch libraries in a single build, > > > because that seems very contrary to the way the Debian architectures > > > work, and would also limit it to the amd64/i386 architectures for no > > > obviously good reason. I had imagined that if there is some sort of > > > multiarch setup, one might have the amd64 and i386 packages installed > > > simultaneously, hence the need for different names. > > The normal way for this is putting it into > > /usr/lib//pkgname/foo.so, and according to the code below you'll > > need the full path at the run time so you indeed need the triplet at both > > build and run time. You can get the triplet in d/rules, not sure how > > should you pass it to the build system as that depends on the build system > > used. For the run time, https://wiki.debian.org/Python/MultiArch suggests > > sys.implementation._multiarch (note that you cannot use it during the > > build as that would break cross-compilation etc.), not sure if there are > > better ways. > > I got all of the triplets working, but was then stymied when I tried > to specify Multi-Arch: same in the control file. I got a lintian > warning: multi-arch-same-package-calls-pycompile > It seems that since the pybuild system (via dh_python3) adds a > py3compile command to the postinst of the package, then I can't safely > use Multi-Arch: same. > > I don't know if this is the case for all python3 Arch: any packages > with compiled extensions; I think the tag desciption has a good step-by-step explanation why does the tag exists. But your package is not a "package with compiled extensions", is it? > are they all effectively Multi-Arch: no? Is this worth thinking about > in the longer term? What do you propose? -- WBR, wRAR signature.asc Description: PGP signature
Re: Build and run-time triplets
On Sun, Jul 24, 2022 at 06:36:57PM +0100, Julian Gilbey wrote: > I got all of the triplets working, but was then stymied when I tried > to specify Multi-Arch: same in the control file. I got a lintian > warning: multi-arch-same-package-calls-pycompile > It seems that since the pybuild system (via dh_python3) adds a > py3compile command to the postinst of the package, then I can't safely > use Multi-Arch: same. Actually, #812228, mentioned in the tag description, was fixed in 2021 so it's possible that this is no longer a problem. -- WBR, wRAR signature.asc Description: PGP signature
Re: Build and run-time triplets
On Sun, Jul 24, 2022 at 11:46:14PM +0500, Andrey Rahmatullin wrote: > On Sun, Jul 24, 2022 at 06:36:57PM +0100, Julian Gilbey wrote: > > I got all of the triplets working, but was then stymied when I tried > > to specify Multi-Arch: same in the control file. I got a lintian > > warning: multi-arch-same-package-calls-pycompile > > It seems that since the pybuild system (via dh_python3) adds a > > py3compile command to the postinst of the package, then I can't safely > > use Multi-Arch: same. > Actually, #812228, mentioned in the tag description, was fixed in 2021 so > it's possible that this is no longer a problem. Ah, that's exciting thanks! So maybe this lintian tag should be dropped? Best wishes, Julian
Re: Build and run-time triplets
On Sun, Jul 24, 2022 at 11:41:56PM +0500, Andrey Rahmatullin wrote: > [...] > > > > I got all of the triplets working, but was then stymied when I tried > > to specify Multi-Arch: same in the control file. I got a lintian > > warning: multi-arch-same-package-calls-pycompile > > It seems that since the pybuild system (via dh_python3) adds a > > py3compile command to the postinst of the package, then I can't safely > > use Multi-Arch: same. > > > > I don't know if this is the case for all python3 Arch: any packages > > with compiled extensions; > I think the tag desciption has a good step-by-step explanation why does > the tag exists. > But your package is not a "package with compiled extensions", is it? Yes, it's got compiled extensions (Cython) in addition to this non-Python .so library file. > > are they all effectively Multi-Arch: no? Is this worth thinking about > > in the longer term? > What do you propose? I think the fix to bug #812228 might have done the job nicely ;-) Best wishes, Julian
Re: Build and run-time triplets
On Sun, Jul 24, 2022 at 08:30:42PM +0100, Julian Gilbey wrote: > On Sun, Jul 24, 2022 at 11:41:56PM +0500, Andrey Rahmatullin wrote: > > [...] > > > > > > I got all of the triplets working, but was then stymied when I tried > > > to specify Multi-Arch: same in the control file. I got a lintian > > > warning: multi-arch-same-package-calls-pycompile > > > It seems that since the pybuild system (via dh_python3) adds a > > > py3compile command to the postinst of the package, then I can't safely > > > use Multi-Arch: same. > > > > > > I don't know if this is the case for all python3 Arch: any packages > > > with compiled extensions; > > I think the tag desciption has a good step-by-step explanation why does > > the tag exists. > > But your package is not a "package with compiled extensions", is it? > > Yes, it's got compiled extensions (Cython) in addition to this > non-Python .so library file. > > > > are they all effectively Multi-Arch: no? Is this worth thinking about > > > in the longer term? > > What do you propose? > > I think the fix to bug #812228 might have done the job nicely ;-) If it actually ships extensions, the "it should usually get a dependency on the Python interpreter for the same architecture" part should still apply as far as I understand it. -- WBR, wRAR signature.asc Description: PGP signature
Re: Build and run-time triplets
On Mon, Jul 25, 2022 at 12:41:16AM +0500, Andrey Rahmatullin wrote:
> On Sun, Jul 24, 2022 at 08:30:42PM +0100, Julian Gilbey wrote:
> [...]
> > > > are they all effectively Multi-Arch: no? Is this worth thinking about
> > > > in the longer term?
> > > What do you propose?
> >
> > I think the fix to bug #812228 might have done the job nicely ;-)
> If it actually ships extensions, the "it should usually get a dependency
> on the Python interpreter for the same architecture" part should still
> apply as far as I understand it.
Thanks Andrey!
OK. So let's dissect this tag info and see where we're currently at.
Explanation: This Multi-Arch: same package uses
pycompile or
py3compile in the specified maintainer script.
.
py{,3}compile are tools used to byte-compile Python source
files. It is typically run on installation of Debian packages that ship
Python modules. However, they do not support installing several
architectures of the same package and this is not Multi-Arch: safe.
This is now out-of-date: firstly, we can presumably get rid of the
pycompile mention, as there are only a tiny handful of Python 2
packages still around, and we're trying to get rid of them.
Secondly, py3compile now supports installing several architectures of
the same package; see the closing changelog message on bug 812228:
Architecture-qualify py*compile and py*clean calls in maintainer scripts,
for architecture-specific Python packages. This allows co-installation
(and even concurrent unpacking) of different architectures of a package.
So the rest of the paragraph is also out of date.
If the contents of the package is not architecture dependent, it should
usually be made binary-all.
That is still certainly true.
If the contents of the package is architecture dependent, it should
usually get a dependency on the Python interpreter for the same
architecture. This is a dependency in the form of python3, not
an architecture-qualified dependency such as python3:any (which
can be fulfilled by the Python interpreter for any architecture).
This is interesting; dh-python gives the dependency:
python3 (<< 3.11), python3 (>= 3~), python3:any
which has both same-architecture and qualified architecture
dependencies; obviously the same-architecture one "wins". But this
paragraph is probably unnecessary for most dh-python-using packages
(though it doesn't seem to harm).
If a dependency on the Python interpreter for the same architecture
exists (usually generated by dh-python), the
Multi-Arch: same has no effect and should be dropped.
Ah. I see the point. Because python3 and python3-minimal are
Multi-Arch: allowed, the different arches of python3 are not
co-installable, and so there is no point in labelling the
arch-dependent module packages as Multi-Arch: same; they still could
not be co-installed.
See-Also: pycompile(1), py3compile(1), Bug#812228
This list can probably be trimmed down to py3compile.
I hope this reasoning is useful; shall I pass it on to the lintian
folk?
Best wishes,
Julian
