Here's updated dh_python proposal. Main change since my last mail: pycompile/pyclean will no longer search for files to compile/remove in public directories as Matthias will fix #552595 using triggers so I don't have to care about pycentral bugs anymore and dpkg -L seems to be fast enough to be used in maintainer scripts (BTW: my previous proposal had a typo: "prerm" should be replaced with "postrm", I guess everyone figured that out from the context…).
Again, the main idea is to ship files in official site/dist-packages location
or symlink files from /usr/share/py{,3}shared directory there (if files can be
shared).
This means Architecture:all packages will need sourceful uploads once list of
supported Python versions will change (binNMUs do not touch architecture
independent packages and even if they'd do, that still leaves the
"=${source:Version}" problem).
Advantages:
* dpkg is aware of all .py files (including symlinks) and their locations (it
isn't true for pysupport/pycentral),
* no need for helper in Depends and Build-Depends - I want dh_python and
pycompile/pyclean to be shipped in python packages,
* broken modules that use __file__ incorrectly will work without problems,
* less opportunities to break a system while installing / upgrading Python
packages:
- problems with creating/removing symlinks in pycentral (see f.e. #552595),
- starting/stopping daemons problems (very long downtimes due to byte
compilation via triggers) in pysupport,
* no more compile errors at install time in public directories, all Python
versions tested at build time,
* Python modules available out of the box (think about daemons and upgrades)
Short overview of how things will work with new dh_python:
==========================================================
build time:
^^^^^^^^^^^
* files installed into the standard location[1] or into private directory.
It's up to maintainer which files will be installed for which Python version,
by default every package providing public module should build/install for all
`pyversions -vs` versions.
* dh_python will:
- move files to /usr/lib/pythonX.Y/*-packages if /usr/local or
/usr/lib/python2.X/site-packages (for X >= 6) is used
(i.e. --install-layout or --prefix can be skipped in most cases),
- copy all files that can be shared to /usr/share/py{,3}shared and replace
the ones in *-packages with a symlink (if package supports only one Python
version, there's no need to use pyshared),
- create simple maintainer scripts with pycompile and pyclean
commands and (if needed) rtupdate script (for private directories that
use default Python version and will most probably work with next default
version). Private modules that cannot be used with default Python version
will get additional pycompile command with all needed arguments, like
minimum required Python version or hardcoded version in both, maintainer
and rtupdate scripts,
- add substvar for ${python:Versions} (used in XB-Python-Version, i.e. the
"make release managers happy" field (this field will be used by RMs to
track transitions)
- add substvar for ${python:Provides}
[1] SL = standard distutils/setuptools/distribute location:
/usr/{,local}/lib/pythonX.Y/{site,dist}-packages/
installation time:
^^^^^^^^^^^^^^^^^^
* maintainer script will byte compile .pyc files for all provided
symlinks / private directories if given Python version is installed
(dpkg -L output will be used to detect which files need byte compilation,
directories without __init__.py file will be skipped),
* user installs new pythonX.Y package:
- bytecompile related symlinks (pycompile -V X.Y) - no need for a list of
files to compile at this point - all .py files in
/usr/lib/pythonX.Y/{site,dist}-packages will have to be byte-compiled,
byte compilation will not fail as it was already tested at build time,
* user removes pythonX.Y package:
- remove all pythonX.Y's .pyc files (pyclean -V X.Y)
- all packages with private directories that use this Python version will be
removed by dpkg (Depends field will do its job) so prerm will remove all
remaining .pyc files in private directories
* default Python version changes:
- rtupdate scripts for packages with private modules that support it (i.e.
the ones without hardcoded Python version and without private extensions)
will be invoked
examples:
=========
example 1 - public modules/extensions only
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
$ python2.4 ./setup.py install --root=debian/python-foo/
debian/python-foo/usr/lib/python2.4/site-packages/foo/__init__.py
debian/python-foo/usr/lib/python2.4/site-packages/foo/_foo.so
debian/python-foo/usr/lib/python2.4/site-packages/foo/bar.py
debian/python-foo/usr/lib/python2.4/site-packages/spam.py
$ python2.5 ./setup.py install --root=debian/python-foo/
debian/python-foo/usr/lib/python2.5/site-packages/foo/__init__.py
debian/python-foo/usr/lib/python2.5/site-packages/foo/_foo.so
debian/python-foo/usr/lib/python2.5/site-packages/foo/bar.py
debian/python-foo/usr/lib/python2.5/site-packages/spam.py
$ python2.6 ./setup.py install --root=debian/python-foo/
debian/python-foo/usr/local/lib/python2.6/dist-packages/foo/__init__.py
debian/python-foo/usr/local/lib/python2.6/dist-packages/foo/_foo.so
debian/python-foo/usr/local/lib/python2.6/dist-packages/foo/bar.py
debian/python-foo/usr/local/lib/python2.6/dist-packages/spam.py
$ python3.1 ./3.x/setup.py install --root=debian/python3-foo/
debian/python3-foo/usr/local/lib/python3.1/dist-packages/foo/__init__.py
debian/python3-foo/usr/local/lib/python3.1/dist-packages/foo/_foo.so
debian/python3-foo/usr/local/lib/python3.1/dist-packages/foo/bar.py
$ dh_python
# doesn't have to use debian/pyversions or XS-Python-Version as Debian
# packages now have to build/test files for all supported Python versions;
# ignores private dirs
debian/python-foo/usr/lib/python2.4/site-packages/foo/_foo.so
debian/python-foo/usr/lib/python2.4/site-packages/foo/__init__.py ->
/usr/share/pyshared/foo/__init__.py
debian/python-foo/usr/lib/python2.4/site-packages/foo/bar.py ->
/usr/share/pyshared/foo/bar.py
debian/python-foo/usr/lib/python2.4/site-packages/foo/spam.py ->
/usr/share/pyshared/foo/spam.py
debian/python-foo/usr/lib/python2.5/site-packages/foo/_foo.so
debian/python-foo/usr/lib/python2.5/site-packages/foo/__init__.py ->
/usr/share/pyshared/foo/__init__.py
debian/python-foo/usr/lib/python2.5/site-packages/foo/bar.py ->
/usr/share/pyshared/foo/bar.py
debian/python-foo/usr/lib/python2.5/site-packages/foo/spam.py ->
/usr/share/pyshared/foo/spam.py
debian/python-foo/usr/lib/python2.6/dist-packages/foo/_foo.so
debian/python-foo/usr/lib/python2.6/dist-packages/foo/__init__.py ->
/usr/share/pyshared/foo/__init__.py
debian/python-foo/usr/lib/python2.6/dist-packages/foo/bar.py ->
/usr/share/pyshared/foo/bar.py
debian/python-foo/usr/lib/python2.6/dist-packages/foo/spam.py # not a symlink
as it differs from Python2.4 and 2.5's version
debian/python-foo/usr/share/pyshared/foo/__init__.py
debian/python-foo/usr/share/pyshared/foo/bar.py
debian/python-foo/usr/share/pyshared/spam.py
debian/python3-foo/usr/lib/python3.1/dist-packages/foo/_foo.so
debian/python3-foo/usr/lib/python3.1/dist-packages/foo/__init__.py
debian/python3-foo/usr/lib/python3.1/dist-packages/foo/bar.py
debian/python3-foo/usr/lib/python3.1/dist-packages/foo/spam.py
# note that debian/python3-foo/usr/share/py3shared/ is not used here as
# there's only one 3.X version supported by this package
$ grep foo debian/python-foo.postinst
dpkg -L python-foo | pycompile
$ grep foo debian/python3-foo.postinst
dpkg -L python3-foo | pycompile
$ grep foo debian/python-foo.prerm
dpkg -L python-foo | pyclean
$ grep foo debian/python3-foo.prerm
dpkg -L python3-foo | pyclean
example 2 - private modules/scripts
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
# bar uses default Python version
$ python ./setup.py install --root=debian/mypackage/
--install-lib=/usr/share/privatedir1/ --install-scripts=/usr/share/privatedir1
debian/mypackage/usr/share/privatedir1/bar/__init__.py
debian/mypackage/usr/share/privatedir1/run.py
# spam uses Python 2.4 only
$ python ./setup.py install --root=debian/mypackage/
--install-lib=/usr/share/privatedir2/ --install-scripts=/usr/share/privatedir2
debian/mypackage/usr/share/privatedir2/spam/__init__.py
debian/mypackage/usr/share/privatedir2/run.py
# egg uses Python >= 2.6
$ python ./setup.py install --root=debian/mypackage/
--install-lib=/usr/share/privatedir3/ --install-scripts=/usr/share/privatedir3
debian/mypackage/usr/lib/privatedir3/egg/__init__.py
debian/mypackage/usr/lib/privatedir3/run.py
# baz uses Python >= 2.5 and has private extension
$ python ./setup.py install --root=debian/mypackage/
--install-lib=/usr/lib/privatedir4/ --install-scripts=/usr/lib/privatedir4
debian/mypackage/usr/lib/privatedir4/baz/__init__.py
debian/mypackage/usr/lib/privatedir4/baz/baz.so
debian/mypackage/usr/lib/privatedir4/run.py
$ dh_python /usr/share/privatedir1 # ignores SL
debian/mypackage/usr/share/privatedir1/bar/__init__.py
debian/mypackage/usr/share/privatedir1/run.py
# Python not hardcoded in shebang and -V not used so recompile once default
# Python version will change, minimum required version hardcoded in the script
debian/mypackage/usr/share/python/runtime.d/mypackage.rtupdate
$ grep python:Depends debian/mypackage.substvars
python:Depends=python
$ dh_python /usr/share/privatedir2 -V 2.4
debian/mypackage/usr/share/privatedir2/bar/__init__.py
debian/mypackage/usr/share/privatedir2/run.py
# note that rtupdate script is not touched, python2.4 is added to the Depends
# instead and maintainer script contains hardcoded python2.4
debian/mypackage/usr/share/python/data/mypackage
$ grep python:Depends debian/mypackage.substvars
python:Depends=python, python2.4
$ grep privatedir debian/mypackage.postinst
dpkg -L mypackage | pycompile /usr/share/privatedir1
dpkg -L mypackage | pycompile /usr/share/privatedir2 -V 2.4
$ grep privatedir debian/mypackage.prerm
dpkg -L mypackage | pyclean /usr/share/privatedir1
dpkg -L mypackage | pyclean /usr/share/privatedir2
$ dh_python /usr/share/privatedir3 -V 2.6+
debian/mypackage/usr/share/privatedir3/egg/__init__.py
debian/mypackage/usr/share/privatedir3/run.py
debian/mypackage/usr/share/python/runtime.d/mypackage.rtupdate
$ grep python:Depends debian/mypackage.substvars
python:Depends=python, python2.4, python (>=2.6) | python2.6
$ grep privatedir debian/mypackage.postinst
dpkg -L mypackage | pycompile /usr/share/privatedir1
dpkg -L mypackage | pycompile /usr/share/privatedir2 -V 2.4
dpkg -L mypackage | pycompile /usr/share/privatedir3 -V 2.6+
$ grep privatedir debian/mypackage.prerm
dpkg -L mypackage | pyclean /usr/share/privatedir1
dpkg -L mypackage | pyclean /usr/share/privatedir2
dpkg -L mypackage | pyclean /usr/share/privatedir3
$ dh_python /usr/share/privatedir4 -V 2.5+ # note the "+" here and lack of it
later
debian/mypackage/usr/lib/privatedir4/baz/__init__.py
debian/mypackage/usr/lib/privatedir4/baz/baz.so
debian/mypackage/usr/lib/privatedir4/run.py
$ grep python:Depends debian/mypackage.substvars
python:Depends=python, python2.4, python (>=2.6) | python2.6, python2.5
$ grep privatedir debian/mypackage.postinst
dpkg -L mypackage | pycompile /usr/share/privatedir1
dpkg -L mypackage | pycompile /usr/share/privatedir2 -V 2.4
dpkg -L mypackage | pycompile /usr/share/privatedir3 -V 2.6+
dpkg -L mypackage | pycompile /usr/lib/privatedir4 -V 2.5 # note that there's
no "+" here (due to private extension)
$ grep privatedir debian/mypackage.prerm
dpkg -L mypackage | pyclean /usr/share/privatedir1
dpkg -L mypackage | pyclean /usr/share/privatedir2
dpkg -L mypackage | pyclean /usr/share/privatedir3
dpkg -L mypackage | pyclean /usr/lib/privatedir4
--
Piotr Ożarowski Debian GNU/Linux Developer
www.ozarowski.pl www.griffith.cc www.debian.org
GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645
signature.asc
Description: Digital signature
