Re: how to properly split up into python3-foo and foo-util package?
On Wed, Jul 12, 2023 at 02:21:48AM +0200, Christoph Anton Mitterer wrote: > When I run debuild -us -uc on that, it generates: > debian/tmp/... > debian/tmp/usr/bin/ > debian/tmp/usr/lib/python3.11/dist-packages/foo > debian/tmp/usr/lib/python3.11/dist-packages/foo-1.0.0.dist-info > debian/tmp/usr/lib/python3.11/dist-packages/foo-1.0.0.dist-info/LICENCE > > But then complains that the files aren't installed by anyone. Yes, you usually need to use dh_install when having several subpackages. This is not Python-specific. > 1) Is there some mechanism in dh_python that would automatically split >(respectively install) the files to the two packages, and I'm just >to dumb to use it? >Like that it knows, the Python package should likely go into the >python3-* Debian package, and usr/bin stuff should likely go in the >other? I don't think there is, and I don't think "usr/bin stuff should likely go in the other". Many Python module packages ship executables, especially now that you no longer have Python 2 subpackages. > 2) I then tried with such package.install files like those: >foo-util.install: > usr/bin > >python3-foo.install: > usr/lib > >a) Why does it work to use just usr/... and not debian/tmp/usr/... ? > Actually, both seems to work, which confuses me even more ^^ You can check the search logic in dh_install(1). >b) What - if any - is the proper way here? Like I did, with one > argument? Yes, because the files are already installed into correct destinations. > Or should one use the two arguments per line version? > Or perhaps (for the 2nd file) rather usr/lib/python* ? > Or rather the debian/tmp/usr/ path versions? > Or using something completely different than dh_install? No. > 3) In debian/tmp, the subdir was /python3.11/ but in the final .deb >file it's actually /python3/ (as I think it should be). >Is it expected, that first it's /python3.11/ or am I doing anything >wrong? Yes, it's expected. > 4) Are there way to have the Dependencies in control even more >autodetected? >a) That foo-util's dependency on python3-foo is somehow auto-filled > by dh_python? No, as there is no data to use here. >b) Or the Build-Deps? I mean dh_python should at least be able to > find out about python3-setuptools, python3-setuptools-scm from my > pyproject.toml, shouldn't it? You cannot autodetect build dependencies at the build time. That would be too late. > 5) Not really 100% Debian related, but in the Python sdist,... should >that contain the debian/*? No, and the upstream source shouldn't contain debian/ anyway, as the life cycles of packaging and upstream sources should be separate even if the person doing both is the same.
Re: how to properly split up into python3-foo and foo-util package?
On Wed, 12 Jul 2023 at 02:21:48 +0200, Christoph Anton Mitterer wrote:
> 2) I then tried with such package.install files like those:
>foo-util.install:
> usr/bin
>
>python3-foo.install:
> usr/lib
>
>a) Why does it work to use just usr/... and not debian/tmp/usr/... ?
> Actually, both seems to work, which confuses me even more ^^
From debhelper compatibility level 7 on, dh_install will fall back to
looking in debian/tmp for files, if it does not find them in the
current directory (or wherever you've told it to look using
--sourcedir).
— dh_install(1)
So what dh_install -pfoo-util does for the usr/bin line is:
- is there a ./usr/bin? - no
- is there a ./debian/tmp/usr/bin? - yes, so package that
I think the short form with just usr/... is the more obvious one in simple
cases like this. Writing it out the long way is only necessary if you're
doing multiple builds (like dbus, which builds and installs the same
source code with different options into debian/tmp and debian/tmp-udeb),
or if you have to disambiguate because your source code contains a
./usr directory.
But if you put a greater value on "explicit is better than implicit"
than I do, then you might prefer to prefix everything with debian/tmp/.
>b) What - if any - is the proper way here? Like I did, with one
> argument?
> Or should one use the two arguments per line version?
If the upstream package installs files into their correct places, then
one argument per line is fine, and provides "don't repeat yourself".
More than one argument per line is for when you want to change upstream's
installation location for whatever reason, for example:
usr/bin/this-is-a-game usr/games
or when you are taking a file from the source tree that is not installed
by the upstream build system, and installing it directly:
contrib/utils/some-extra-utility usr/bin
> Or perhaps (for the 2nd file) rather usr/lib/python* ?
IMO it's often good to be relatively specific in .install files, so that
if your upstream makes an incompatible change, attempting to build an
updated package without acknowledging the change will FTBFS and alert you
that something unusual is happening. So I would personally be inclined
to use something like
usr/lib/python3*/dist-packages/foo
usr/lib/python3*/dist-packages/Foo-*.egg-info
on the basis that if those no longer match, then upstream has made a
significant change that will affect compatibility for third-party code,
in which case I want to know about it (and perhaps do an experimental
upload and check that dependent packages are ready for it before going
to unstable).
> 3) In debian/tmp, the subdir was /python3.11/ but in the final .deb
>file it's actually /python3/ (as I think it should be).
>Is it expected, that first it's /python3.11/ or am I doing anything
>wrong?
I think this is expected, dh_python3 moves it around automatically.
> 4) Are there way to have the Dependencies in control even more
>autodetected?
>a) That foo-util's dependency on python3-foo is somehow auto-filled
> by dh_python?
Even if it was auto-detected, dependencies within a single source
package should usually be relatively strict, because within the same
source package it's common to make use of internal interfaces that are
not considered to be public API - so you probably want to override it
so it will depend on python3-foo (= ${binary:Version}) anyway.
smcv
Re: how to properly split up into python3-foo and foo-util package?
On Wed, 12 Jul 2023 at 11:19:07 +0200, Andrey Rakhmatullin wrote: > I don't think "usr/bin stuff should likely go > in the other". Many Python module packages ship executables, especially > now that you no longer have Python 2 subpackages. I would personally say that if the executables are significant, and particularly if we expect that users will use them without knowing or caring whether they are implemented in Python, then they should be in a package with a name and Section that make it look like an executable program and not a Python library: if nothing else, that will make them a lot more discoverable. So I think Christoph is probably correct to be thinking about shipping them as foo-util or similar. If nothing else, making executables part of the interface of the python3-foo package is going to come back to bite us when Python 4 happens (hopefully not soon, but if there have been 3 major versions then there will probably be a 4th eventually). If the Python library is considered to be a public API, then it should be in a python3-foo library. src:binwalk and src:tap.py are examples of separating out executable + library like this. If the Python library is considered to be a private implementation detail of the executables, then it doesn't need to be packaged separately (for example bmap-tools, dput, meson and offlineimap all contain private Python libraries that are not a stable API), and ideally it would be in a location that is not on the default import search path, like /usr/share/foo or /usr/lib/foo (dput and offlineimap do this, although bmap-tools and meson don't). smcv
Re: how to properly split up into python3-foo and foo-util package?
5) Not really 100% Debian related, but in the Python sdist,... should that contain the debian/*? No, and the upstream source shouldn't contain debian/ anyway, as the life cycles of packaging and upstream sources should be separate even if the person doing both is the same. This would then be a "native" package, and it's not recommended to use this package format in most cases: https://www.debian.org/doc/manuals/debmake-doc/ch05.en.html#native While it can happen that a developer maintains the Debian packages for their own software, they also have to consider other dpkg-based distributions, such as Ubuntu. Maintaining the Debian changelog for both in the same upstream repository isn't a lot of fun.
Re: how to properly split up into python3-foo and foo-util package?
Dear Christoph, I'm sure I can not help you but I'm asking because I want to learn. Do you have a link to your repository? What do you mean by the terms "simple Python package" and "two packages"? These terms do not exists in Python context. Python do differentiate between "Distribution Package" (the name you would find e.g. on PyPI) and "Import Packages" (the name you use with your "import" statement). And there is also a difference with "Debian Package" (a deb-file). Of course all three type of packages can be the same but don't have to. Am 12.07.2023 02:21 schrieb Christoph Anton Mitterer: debian/control (all boring fields removed): Source: foo I assume that "foo" is the "Distribution Package" ? 1) Is there some mechanism in dh_python that would automatically split (respectively install) the files to the two packages, and I'm just to dumb to use it? I don't understand why there are two packages? Why two? I can not find that in your setup. Am 12.07.2023 02:21 schrieb Christoph Anton Mitterer: debian/control (all boring fields removed): Source: foo Package: python3-foo Package: foo-util You build two Debian packages (deb-files) out of one source tarball? Interesting to know that this is possible. It would be great if you could upload your "foo" repository somewhere when it works as an example.
Re: how to properly split up into python3-foo and foo-util package?
On Wed, Jul 12, 2023 at 11:05:57AM +, c.bu...@posteo.jp wrote: > What do you mean by the terms "simple Python package" and "two packages"? > These terms do not exists in Python context. These are Debian terms. > Python do differentiate between > "Distribution Package" (the name you would find e.g. on PyPI) and "Import > Packages" (the name you use with your "import" statement). And there is also > a difference with "Debian Package" (a deb-file). Of course all three type of > packages can be the same but don't have to. > > Am 12.07.2023 02:21 schrieb Christoph Anton Mitterer: > > debian/control (all boring fields removed): > > Source: foo > > I assume that "foo" is the "Distribution Package" ? It's the Debian source package as the context is debian/control. Or what do you mean? > > 1) Is there some mechanism in dh_python that would automatically split > >(respectively install) the files to the two packages, and I'm just > >to dumb to use it? > > I don't understand why there are two packages? Why two? I can not find that > in your setup. Package: python3-foo Package: foo-util > > debian/control (all boring fields removed): > > Source: foo > > Package: python3-foo > > Package: foo-util > > You build two Debian packages (deb-files) out of one source tarball? > Interesting to know that this is possible. It's definitely possible and I would expect any good guide to mention this.
Re: how to properly split up into python3-foo and foo-util package?
On Wed, Jul 12, 2023 at 12:16:51PM +0200, Gregor Riepl wrote: > > > 5) Not really 100% Debian related, but in the Python sdist,... should > > > that contain the debian/*? > > No, and the upstream source shouldn't contain debian/ anyway, as the life > > cycles of packaging and upstream sources should be separate even if the > > person doing both is the same. > > This would then be a "native" package, and it's not recommended to use this > package format in most cases: Not necessarily, some people make non-native packages and store debian/ in the VCS. It's not a good idea, though.
Re: how to properly split up into python3-foo and foo-util package?
On Wed, Jul 12, 2023 at 11:16:28AM +0100, Simon McVittie wrote: > On Wed, 12 Jul 2023 at 11:19:07 +0200, Andrey Rakhmatullin wrote: > > I don't think "usr/bin stuff should likely go > > in the other". Many Python module packages ship executables, especially > > now that you no longer have Python 2 subpackages. > > I would personally say that if the executables are significant, and > particularly if we expect that users will use them without knowing or > caring whether they are implemented in Python, then they should be in > a package with a name and Section that make it look like an executable > program and not a Python library: if nothing else, that will make them > a lot more discoverable. So I think Christoph is probably correct to be > thinking about shipping them as foo-util or similar. Oh yeah, I just tried to say that packaging them separately is not the only option and probably not even the main option. > If nothing else, making executables part of the interface of the > python3-foo package is going to come back to bite us when Python 4 happens > (hopefully not soon, but if there have been 3 major versions then there > will probably be a 4th eventually). > > If the Python library is considered to be a public API, then it should > be in a python3-foo library. src:binwalk and src:tap.py are examples > of separating out executable + library like this. There is a popular concern about having binary packages that consist of a 1 kB file + changelog + copyright. It also somewhat complicates packaging, unfortunately. > If the Python library is considered to be a private implementation detail > of the executables, then it doesn't need to be packaged separately > (for example bmap-tools, dput, meson and offlineimap all contain > private Python libraries that are not a stable API), and ideally it > would be in a location that is not on the default import search path, > like /usr/share/foo or /usr/lib/foo (dput and offlineimap do this, > although bmap-tools and meson don't). Yup.
Re: how to properly split up into python3-foo and foo-util package?
Am 12.07.2023 13:25 schrieb Andrey Rakhmatullin: On Wed, Jul 12, 2023 at 11:05:57AM +, c.bu...@posteo.jp wrote: You build two Debian packages (deb-files) out of one source tarball? Interesting to know that this is possible. It's definitely possible and I would expect any good guide to mention this. I really need to see the full repo to better understand whats going on here. One pyproject.toml indicates one and only "Python Distribution Package". I can not imagine why someone would split it up into two "Debian packages"? Isn't it easier (from Debian Package Maintainers view) to have one "Python Distribution Package" per "Debian package". So when I want two Debian packages I would create to Python Distro packages in my upstream repo.
Re: how to properly split up into python3-foo and foo-util package?
On Wed, Jul 12, 2023 at 11:56:05AM +, c.bu...@posteo.jp wrote: > > > You build two Debian packages (deb-files) out of one source tarball? > > > Interesting to know that this is possible. > > It's definitely possible and I would expect any good guide to mention > > this. > > I really need to see the full repo to better understand whats going on here. If you want to learn about Debian source packages that build several binary packages you can look at any such package in the archive. > One pyproject.toml indicates one and only "Python Distribution Package". I > can not imagine why someone would split it up into two "Debian packages"? This is answered in the original email. One package for the module and another one for the executable. > Isn't it easier (from Debian Package Maintainers view) to have one "Python > Distribution Package" per "Debian package". So when I want two Debian > packages I would create to Python Distro packages in my upstream repo. No, and it makes no sense to do that.
