Here's updated dh_python proposal. Main change since my last mail: pycompile/pyclean will no longer search for files to compile/remove in public directories as Matthias will fix #552595 using triggers so I don't have to care about pycentral bugs anymore and dpkg -L seems to be fast enough to be used in maintainer scripts (BTW: my previous proposal had a typo: "prerm" should be replaced with "postrm", I guess everyone figured that out from the context…).
Again, the main idea is to ship files in official site/dist-packages location or symlink files from /usr/share/py{,3}shared directory there (if files can be shared). This means Architecture:all packages will need sourceful uploads once list of supported Python versions will change (binNMUs do not touch architecture independent packages and even if they'd do, that still leaves the "=${source:Version}" problem). Advantages: * dpkg is aware of all .py files (including symlinks) and their locations (it isn't true for pysupport/pycentral), * no need for helper in Depends and Build-Depends - I want dh_python and pycompile/pyclean to be shipped in python packages, * broken modules that use __file__ incorrectly will work without problems, * less opportunities to break a system while installing / upgrading Python packages: - problems with creating/removing symlinks in pycentral (see f.e. #552595), - starting/stopping daemons problems (very long downtimes due to byte compilation via triggers) in pysupport, * no more compile errors at install time in public directories, all Python versions tested at build time, * Python modules available out of the box (think about daemons and upgrades) Short overview of how things will work with new dh_python: ========================================================== build time: ^^^^^^^^^^^ * files installed into the standard location[1] or into private directory. It's up to maintainer which files will be installed for which Python version, by default every package providing public module should build/install for all `pyversions -vs` versions. * dh_python will: - move files to /usr/lib/pythonX.Y/*-packages if /usr/local or /usr/lib/python2.X/site-packages (for X >= 6) is used (i.e. --install-layout or --prefix can be skipped in most cases), - copy all files that can be shared to /usr/share/py{,3}shared and replace the ones in *-packages with a symlink (if package supports only one Python version, there's no need to use pyshared), - create simple maintainer scripts with pycompile and pyclean commands and (if needed) rtupdate script (for private directories that use default Python version and will most probably work with next default version). Private modules that cannot be used with default Python version will get additional pycompile command with all needed arguments, like minimum required Python version or hardcoded version in both, maintainer and rtupdate scripts, - add substvar for ${python:Versions} (used in XB-Python-Version, i.e. the "make release managers happy" field (this field will be used by RMs to track transitions) - add substvar for ${python:Provides} [1] SL = standard distutils/setuptools/distribute location: /usr/{,local}/lib/pythonX.Y/{site,dist}-packages/ installation time: ^^^^^^^^^^^^^^^^^^ * maintainer script will byte compile .pyc files for all provided symlinks / private directories if given Python version is installed (dpkg -L output will be used to detect which files need byte compilation, directories without __init__.py file will be skipped), * user installs new pythonX.Y package: - bytecompile related symlinks (pycompile -V X.Y) - no need for a list of files to compile at this point - all .py files in /usr/lib/pythonX.Y/{site,dist}-packages will have to be byte-compiled, byte compilation will not fail as it was already tested at build time, * user removes pythonX.Y package: - remove all pythonX.Y's .pyc files (pyclean -V X.Y) - all packages with private directories that use this Python version will be removed by dpkg (Depends field will do its job) so prerm will remove all remaining .pyc files in private directories * default Python version changes: - rtupdate scripts for packages with private modules that support it (i.e. the ones without hardcoded Python version and without private extensions) will be invoked examples: ========= example 1 - public modules/extensions only ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ $ python2.4 ./setup.py install --root=debian/python-foo/ debian/python-foo/usr/lib/python2.4/site-packages/foo/__init__.py debian/python-foo/usr/lib/python2.4/site-packages/foo/_foo.so debian/python-foo/usr/lib/python2.4/site-packages/foo/bar.py debian/python-foo/usr/lib/python2.4/site-packages/spam.py $ python2.5 ./setup.py install --root=debian/python-foo/ debian/python-foo/usr/lib/python2.5/site-packages/foo/__init__.py debian/python-foo/usr/lib/python2.5/site-packages/foo/_foo.so debian/python-foo/usr/lib/python2.5/site-packages/foo/bar.py debian/python-foo/usr/lib/python2.5/site-packages/spam.py $ python2.6 ./setup.py install --root=debian/python-foo/ debian/python-foo/usr/local/lib/python2.6/dist-packages/foo/__init__.py debian/python-foo/usr/local/lib/python2.6/dist-packages/foo/_foo.so debian/python-foo/usr/local/lib/python2.6/dist-packages/foo/bar.py debian/python-foo/usr/local/lib/python2.6/dist-packages/spam.py $ python3.1 ./3.x/setup.py install --root=debian/python3-foo/ debian/python3-foo/usr/local/lib/python3.1/dist-packages/foo/__init__.py debian/python3-foo/usr/local/lib/python3.1/dist-packages/foo/_foo.so debian/python3-foo/usr/local/lib/python3.1/dist-packages/foo/bar.py $ dh_python # doesn't have to use debian/pyversions or XS-Python-Version as Debian # packages now have to build/test files for all supported Python versions; # ignores private dirs debian/python-foo/usr/lib/python2.4/site-packages/foo/_foo.so debian/python-foo/usr/lib/python2.4/site-packages/foo/__init__.py -> /usr/share/pyshared/foo/__init__.py debian/python-foo/usr/lib/python2.4/site-packages/foo/bar.py -> /usr/share/pyshared/foo/bar.py debian/python-foo/usr/lib/python2.4/site-packages/foo/spam.py -> /usr/share/pyshared/foo/spam.py debian/python-foo/usr/lib/python2.5/site-packages/foo/_foo.so debian/python-foo/usr/lib/python2.5/site-packages/foo/__init__.py -> /usr/share/pyshared/foo/__init__.py debian/python-foo/usr/lib/python2.5/site-packages/foo/bar.py -> /usr/share/pyshared/foo/bar.py debian/python-foo/usr/lib/python2.5/site-packages/foo/spam.py -> /usr/share/pyshared/foo/spam.py debian/python-foo/usr/lib/python2.6/dist-packages/foo/_foo.so debian/python-foo/usr/lib/python2.6/dist-packages/foo/__init__.py -> /usr/share/pyshared/foo/__init__.py debian/python-foo/usr/lib/python2.6/dist-packages/foo/bar.py -> /usr/share/pyshared/foo/bar.py debian/python-foo/usr/lib/python2.6/dist-packages/foo/spam.py # not a symlink as it differs from Python2.4 and 2.5's version debian/python-foo/usr/share/pyshared/foo/__init__.py debian/python-foo/usr/share/pyshared/foo/bar.py debian/python-foo/usr/share/pyshared/spam.py debian/python3-foo/usr/lib/python3.1/dist-packages/foo/_foo.so debian/python3-foo/usr/lib/python3.1/dist-packages/foo/__init__.py debian/python3-foo/usr/lib/python3.1/dist-packages/foo/bar.py debian/python3-foo/usr/lib/python3.1/dist-packages/foo/spam.py # note that debian/python3-foo/usr/share/py3shared/ is not used here as # there's only one 3.X version supported by this package $ grep foo debian/python-foo.postinst dpkg -L python-foo | pycompile $ grep foo debian/python3-foo.postinst dpkg -L python3-foo | pycompile $ grep foo debian/python-foo.prerm dpkg -L python-foo | pyclean $ grep foo debian/python3-foo.prerm dpkg -L python3-foo | pyclean example 2 - private modules/scripts ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ # bar uses default Python version $ python ./setup.py install --root=debian/mypackage/ --install-lib=/usr/share/privatedir1/ --install-scripts=/usr/share/privatedir1 debian/mypackage/usr/share/privatedir1/bar/__init__.py debian/mypackage/usr/share/privatedir1/run.py # spam uses Python 2.4 only $ python ./setup.py install --root=debian/mypackage/ --install-lib=/usr/share/privatedir2/ --install-scripts=/usr/share/privatedir2 debian/mypackage/usr/share/privatedir2/spam/__init__.py debian/mypackage/usr/share/privatedir2/run.py # egg uses Python >= 2.6 $ python ./setup.py install --root=debian/mypackage/ --install-lib=/usr/share/privatedir3/ --install-scripts=/usr/share/privatedir3 debian/mypackage/usr/lib/privatedir3/egg/__init__.py debian/mypackage/usr/lib/privatedir3/run.py # baz uses Python >= 2.5 and has private extension $ python ./setup.py install --root=debian/mypackage/ --install-lib=/usr/lib/privatedir4/ --install-scripts=/usr/lib/privatedir4 debian/mypackage/usr/lib/privatedir4/baz/__init__.py debian/mypackage/usr/lib/privatedir4/baz/baz.so debian/mypackage/usr/lib/privatedir4/run.py $ dh_python /usr/share/privatedir1 # ignores SL debian/mypackage/usr/share/privatedir1/bar/__init__.py debian/mypackage/usr/share/privatedir1/run.py # Python not hardcoded in shebang and -V not used so recompile once default # Python version will change, minimum required version hardcoded in the script debian/mypackage/usr/share/python/runtime.d/mypackage.rtupdate $ grep python:Depends debian/mypackage.substvars python:Depends=python $ dh_python /usr/share/privatedir2 -V 2.4 debian/mypackage/usr/share/privatedir2/bar/__init__.py debian/mypackage/usr/share/privatedir2/run.py # note that rtupdate script is not touched, python2.4 is added to the Depends # instead and maintainer script contains hardcoded python2.4 debian/mypackage/usr/share/python/data/mypackage $ grep python:Depends debian/mypackage.substvars python:Depends=python, python2.4 $ grep privatedir debian/mypackage.postinst dpkg -L mypackage | pycompile /usr/share/privatedir1 dpkg -L mypackage | pycompile /usr/share/privatedir2 -V 2.4 $ grep privatedir debian/mypackage.prerm dpkg -L mypackage | pyclean /usr/share/privatedir1 dpkg -L mypackage | pyclean /usr/share/privatedir2 $ dh_python /usr/share/privatedir3 -V 2.6+ debian/mypackage/usr/share/privatedir3/egg/__init__.py debian/mypackage/usr/share/privatedir3/run.py debian/mypackage/usr/share/python/runtime.d/mypackage.rtupdate $ grep python:Depends debian/mypackage.substvars python:Depends=python, python2.4, python (>=2.6) | python2.6 $ grep privatedir debian/mypackage.postinst dpkg -L mypackage | pycompile /usr/share/privatedir1 dpkg -L mypackage | pycompile /usr/share/privatedir2 -V 2.4 dpkg -L mypackage | pycompile /usr/share/privatedir3 -V 2.6+ $ grep privatedir debian/mypackage.prerm dpkg -L mypackage | pyclean /usr/share/privatedir1 dpkg -L mypackage | pyclean /usr/share/privatedir2 dpkg -L mypackage | pyclean /usr/share/privatedir3 $ dh_python /usr/share/privatedir4 -V 2.5+ # note the "+" here and lack of it later debian/mypackage/usr/lib/privatedir4/baz/__init__.py debian/mypackage/usr/lib/privatedir4/baz/baz.so debian/mypackage/usr/lib/privatedir4/run.py $ grep python:Depends debian/mypackage.substvars python:Depends=python, python2.4, python (>=2.6) | python2.6, python2.5 $ grep privatedir debian/mypackage.postinst dpkg -L mypackage | pycompile /usr/share/privatedir1 dpkg -L mypackage | pycompile /usr/share/privatedir2 -V 2.4 dpkg -L mypackage | pycompile /usr/share/privatedir3 -V 2.6+ dpkg -L mypackage | pycompile /usr/lib/privatedir4 -V 2.5 # note that there's no "+" here (due to private extension) $ grep privatedir debian/mypackage.prerm dpkg -L mypackage | pyclean /usr/share/privatedir1 dpkg -L mypackage | pyclean /usr/share/privatedir2 dpkg -L mypackage | pyclean /usr/share/privatedir3 dpkg -L mypackage | pyclean /usr/lib/privatedir4 -- Piotr Ożarowski Debian GNU/Linux Developer www.ozarowski.pl www.griffith.cc www.debian.org GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645
signature.asc
Description: Digital signature