Source: python3-defaults
Version: 3.5.3-3
Severity: important
As reported in https://bugs.debian.org/870820 and getting very positive
feedback at DEBCONF17 after migrating Debian Policy to DocBookXML,
debiandoc-sgml will be removed from the archive at buster+{0,1}. I am sending
reminder to packages which still uses this package.
The offending SGML source is converted to XML using "debiandoc2dbk -1" command.
Also "pandoc -f doocbook -t rst" can convert docbookxml into pandoc.
If you wish to move this to policy, please talk to them.
Also, since this is Python, why not RST?
So please replace it and use the actively maintained XML/RST tool chain.
Auto converted file attached. (You may need to do manual fix.)
Osamu
-- System Information:
Debian Release: 9.1
APT prefers stable
APT policy: (500, 'stable')
Architecture: amd64 (x86_64)
Kernel: Linux 4.9.0-3-amd64 (SMP w/2 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8),
LANGUAGE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
<?xml version='1.0' encoding='utf-8'?>
<!-- -*- DocBook -*- -->
<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
"http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd"; [
<!-- Include entity definition file by uncommenting the following -->
<!-- <!ENTITY % versiondata SYSTEM "version.ent"> %versiondata; -->
]>
<book lang="en">
<title>Debian Python Policy</title>
<bookinfo>
<authorgroup>
<author><personname>Neil
Schemenauer</personname><email>n...@debian.org</email></author>
<author><personname>Matthias
Klose</personname><email>d...@debian.org</email></author>
<author><personname>Gregor
Hoffleit</personname><email>fli...@debian.org</email></author>
<author><personname>Josselin
Mouette</personname><email>j...@debian.org</email></author>
<author><personname>Joe
Wreschnig</personname><email>pi...@debian.org</email></author>
</authorgroup>
<releaseinfo>version 0.4.1.0</releaseinfo>
<pubdate><!-- put date --></pubdate>
<abstract>
<para>
This document describes the packaging of Python within the Debian GNU/Linux
distribution and the policy requirements for packaged Python programs and
modules.
</para>
</abstract>
<copyright><year>1999, 2001, 2003, 2006</year><holder>Software in the Public
Interest</holder></copyright>
<legalnotice>
<para>
This manual is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
Foundation; either version 2 of the License, or (at your option) any later
version.
</para>
<para>
This is distributed in the hope that it will be useful, but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE. See the GNU General Public License for more details.
</para>
<para>
A copy of the GNU General Public License is available as
<literal>/usr/share/common-licences/GPL</literal> in the Debian GNU/Linux
distribution or on the World Wide Web at <ulink
url="http://www.gnu.org/copyleft/gpl.html";>The GNU Public Licence</ulink>.
</para>
<para>
You can also obtain it by writing to the Free Software Foundation, Inc., 59
Temple Place - Suite 330, Boston, MA 02111-1307, USA.
</para>
</legalnotice>
</bookinfo>
<chapter id="python"><title>Python Packaging</title>
<section id="versions"><title>Versions</title>
<para>
At any given time, the package <systemitem role="package">python</systemitem>
will represent the current default Debian Python version.
</para>
<para>
The default Debian Python version should alway be the latest stable upstream
release that can be integrated in the distribution.
</para>
<para>
Apart from the default version, legacy versions of Python or beta versions of
future releases may be included as well in the distribution, as long as they
are needed by other packages, or as long as it seems reasonable to provide
them. (Note: For the scope of this document, Python versions are synonymous to
feature releases, i.e. Python 2.0 and 2.0.1 are subminor versions of the same
Python version 2.0, but Python 2.1 and 2.2 are indeed different versions.)
</para>
<para>
For any version, the main package must be called <systemitem
role="package">python<replaceable>X</replaceable>.<replaceable>Y</replaceable></systemitem>.
</para>
<para>
The set of currently supported python versions can be found in
<filename>/usr/share/python/debian_defaults</filename>.
</para>
</section>
<section id="base"><title>Main package</title>
<para>
For every Python version provided in the distribution, the package <systemitem
role="package">python<replaceable>X</replaceable>.<replaceable>Y</replaceable></systemitem>
shall comprise a complete distribution for <emphasis>deployment</emphasis> of
Python scripts and applications. The package includes the binary
<filename>/usr/bin/python<replaceable>X</replaceable>.<replaceable>Y</replaceable></filename>
and all modules of the upstream Python distribution.
</para>
<para>
Excluded are any modules that depend on non-<emphasis>required</emphasis>
packages, they will be provided in separate packages. Some tools and files for
the <emphasis>development</emphasis> of Python modules are split off in a
separate package <systemitem
role="package">python<replaceable>X</replaceable>.<replaceable>Y</replaceable>-dev</systemitem>.
Documentation will be provided separately as well.
</para>
<para>
At any time, the <systemitem role="package">python</systemitem> package must
contain a symlink <filename>/usr/bin/python</filename> to the the appropriate
binary
<filename>/usr/bin/python<replaceable>X</replaceable>.<replaceable>Y</replaceable></filename>.
The <systemitem role="package">python</systemitem> package must also depend on
the appropriate <systemitem
role="package">python<replaceable>X</replaceable>.<replaceable>Y</replaceable></systemitem>
to ensure this binary is installed. The version of the <systemitem
role="package">python</systemitem> package must be greater than or equal to
<replaceable>X</replaceable>.<replaceable>Y</replaceable> and smaller than
<replaceable>X</replaceable>.<replaceable>Y+1</replaceable>.
</para>
</section>
<section id="interpreter"><title>Python Interpreter</title>
<section id="interpreter-name"><title>Interpreter Name</title>
<para>
Python scripts depending on the default Python version (see <xref
linkend="base"/>) or not depending on a specific Python version should use
<filename>python</filename> (unversioned) as the interpreter name.
</para>
<para>
Python scripts that only work with a specific Python version must explicitly
use the versioned interpreter name
(<filename>python<replaceable>X</replaceable>.<replaceable>Y</replaceable></filename>).
</para>
</section>
<section id="interpreter-loc"><title>Interpreter Location</title>
<para>
The preferred specification for the Python interpreter is
<filename>/usr/bin/python</filename> or
<filename>/usr/bin/python<replaceable>X</replaceable>.<replaceable>Y</replaceable></filename>.
This ensures that a Debian installation of python is used and all dependencies
on additional python modules are met.
</para>
<para>
If a maintainer would like to provide the user with the possibility to override
the Debian Python interpreter, he may want to use <filename>/usr/bin/env
python</filename> or <filename>/usr/bin/env
python<replaceable>X</replaceable>.<replaceable>Y</replaceable></filename>.
However this is not advisable as it bypasses Debian's dependency checking and
makes the package vulnerable to incomplete local installations of python.
</para>
</section>
</section>
<section id="paths"><title>Module Path</title>
<para>
The module search path for Debian has been amended to include a directory tree
in /usr/local at the beginning of the path. By default, sys.path is searched
in the following order:
</para>
<screen>
/usr/lib/python<replaceable>XY</replaceable>.zip
/usr/lib/python<replaceable>X</replaceable>.<replaceable>Y</replaceable>
/usr/lib/python<replaceable>X</replaceable>.<replaceable>Y</replaceable>/plat-linux2
/usr/lib/python<replaceable>X</replaceable>.<replaceable>Y</replaceable>/lib-tk
/usr/lib/python<replaceable>X</replaceable>.<replaceable>Y</replaceable>/lib-dynload
/usr/local/lib/python<replaceable>X</replaceable>.<replaceable>Y</replaceable>/site-packages
/usr/lib/python<replaceable>X</replaceable>.<replaceable>Y</replaceable>/site-packages
/var/lib/python-support/python<replaceable>X</replaceable>.<replaceable>Y</replaceable>
/usr/lib/python<replaceable>X</replaceable>.<replaceable>Y</replaceable>/site-packages/<replaceable>module-dir</replaceable>
/usr/lib/site-python
</screen>
<para>
The use of the <filename>/usr/lib/site-python</filename> directory is
deprecated. The directory may be dropped from the path in a future version.
The /usr/lib/python<replaceable>XY</replaceable>.zip archive appeared in
python2.3; it is not currently used in Debian. Modules should not install
directly to the <filename>/var/lib/python-support</filename> directory; it is
for use by <xref linkend="pysupport"/>.
</para>
</section>
<section id="docs"><title>Documentation</title>
<para>
Python documentation is split out in separate packages <systemitem
role="package">python<replaceable>X</replaceable>.<replaceable>Y</replaceable>-doc</systemitem>.
The package <systemitem role="package">python-doc</systemitem> will always
provide the documentation for the default Debian Python version.
</para>
<para>
TODO: Policy for documentation of third party packages.
</para>
</section>
</chapter>
<chapter id="module-packages"><title>Packaged Modules</title>
<para>
The goal of these policies is to reduce the work necessary for Python
transitions. Python modules are internally very dependent on a specific Python
version. However, we want to automate recompiling modules when possible,
either during the upgrade itself (re-bytecompiling pyc and pyo files) or
shortly thereafter with automated rebuilds (to handle C extensions). These
policies encourage automated dependency generation and loose version bounds
whenever possible.
</para>
<section id="s2.1"><title>Types of Python Modules</title>
<para>
There are two kinds of Python modules, "pure" Python modules, and extension
modules. Pure Python modules are Python source code that works across many
versions of Python. Extensions are C code compiled and linked against a
specific version of the libpython library, and so can only be used by one
version of Python.
</para>
<para>
Python packages are directories containing at least a
<filename>__init__.py</filename>, other modules, extensions and packages (A
package in the Python sense is unrelated to a Debian package). Python packages
must be packaged into the same directory (as done by upstream). Splitting
components of a package across directories changes the import order and may
confuse documentation tools and IDEs.
</para>
<para>
There are two ways to distribute Python modules. Public modules are installed
in one of the directories listed in <xref linkend="paths"/>. They are
accessible to any program. Private modules are installed in a directory such
as <filename>/usr/share/<replaceable>packagename</replaceable></filename> or
<filename>/usr/lib/<replaceable>packagename</replaceable></filename>. They are
generally only accessible to a specific program or suite of programs included
in the same package.
</para>
</section>
<section id="package-names"><title>Module Package Names</title>
<para>
Public modules should be packaged with a name of <systemitem
role="package">python-<replaceable>foo</replaceable></systemitem>, where
<replaceable>foo</replaceable> is the name of the module. Such a package
should support the current Debian Python version, and more if possible (there
are several tools to help implement this, see <xref
linkend="packaging-tools"/>). For example, if Python 2.3, 2.4, and 2.5 are
supported, the Python command
</para>
<screen>
import foo
</screen>
<para>
should import the module when the user is running any of
<command>/usr/bin/python2.3</command>, <command>/usr/bin/python2.4</command>,
and <command>/usr/bin/python2.5</command>. This requirement also applies to
extension modules; binaries for all the supported Python versions should be
included in a single package.
</para>
</section>
<section id="specifying-versions"><title>Specifying Supported Versions</title>
<para>
The <literal>XS-Python-Version</literal> field in
<filename>debian/control</filename> specifies the versions of Python supported
by the package. This is used to track packages during Python transitions, and
is also used by some packaging scripts to automatically generate appropriate
Depends and Provides lines. The format of the field may be one of the
following:
</para>
<screen>
XS-Python-Version: all
XS-Python-Version: current
XS-Python-Version: current, >= X.Y
XS-Python-Version: >= X.Y
XS-Python-Version: >= A.B, << X.Y
XS-Python-Version: A.B, X.Y
</screen>
<para>
Where "all" means the package supports any Python version available, and
"current" means it supports Debian's current Python version. Explicit Versions
or version ranges can also be used.
</para>
<para>
Your control file should also have a line:
</para>
<screen>
XB-Python-Version: ${python:Versions}
</screen>
<para>
The python:Versions is substituted by the supported Python versions of the
binary package, based on <literal>XS-Python-Version</literal>. (If you are not
using <command>dh_python</command> you will need to handle this substitution
yourself.) The format of the field <literal>XB-Python-Version</literal> is the
same as the <literal>XS-Python-Version</literal> field for packages not
containing extensions. Packages with extensions must list the versions
explicitely.
</para>
<para>
If your package is used by another module or application that requires a
specific Python version, it should also <literal>Provide:
python<replaceable>X</replaceable>.<replaceable>Y</replaceable>-foo</literal>
for each version it supports.
</para>
</section>
<section id="dependencies"><title>Dependencies</title>
<para>
Packaged modules available for the default Python version (or many versions
including the default) as described in <xref linkend="package-names"/> must
depend on "<systemitem role="package">python
(>= <replaceable>X</replaceable>.<replaceable>Y</replaceable></systemitem>)".
If they require other modules to work, they must depend on the corresponding
<systemitem role="package">python-foo</systemitem>. They must not depend on
any <systemitem
role="package">python<replaceable>X</replaceable>.<replaceable>Y</replaceable>-foo</systemitem>.
</para>
<para>
Packaged modules available for one particular version of Python must depend on
the corresponding <systemitem
role="package">python<replaceable>X</replaceable>.<replaceable>Y</replaceable></systemitem>
package instead. If they need other modules, they must depend on the
corresponding <systemitem
role="package">python<replaceable>X</replaceable>.<replaceable>Y</replaceable>-foo</systemitem>
packages, and must not depend on any <systemitem
role="package">python-foo</systemitem>.
</para>
</section>
<section id="provides"><title>Provides</title>
<para>
Provides in packages of the form <systemitem
role="package">python-<replaceable>foo</replaceable></systemitem> must be
specified, if the package contains an extension for more than one python
version. Provides should also be added on request of maintainers who depend on
a non-default python version.
</para>
<para>
Packaged modules available for one particular version of Python must depend on
the corresponding <systemitem
role="package">python<replaceable>X</replaceable>.<replaceable>Y</replaceable></systemitem>
package instead. If they need other modules, they must depend on the
corresponding <systemitem
role="package">python<replaceable>X</replaceable>.<replaceable>Y</replaceable>-foo</systemitem>
packages, and must not depend on any <systemitem
role="package">python-foo</systemitem>.
</para>
</section>
<section id="bytecompilation"><title>Modules Bytecompilation</title>
<para>
If a package provides any binary-independent modules
(<filename>foo.py</filename> files), the corresponding bytecompiled modules
(<filename>foo.pyc</filename> files) and optimized modules
(<filename>foo.pyo</filename> files) must not ship in the package. Instead,
they should be generated in the package's postinst, and removed in the
package's prerm. The package's prerm has to make sure that both
<filename>foo.pyc</filename> and <filename>foo.pyo</filename> are removed.
</para>
<para>
A package should only byte-compile the files which belong to the package.
</para>
<para>
The file <filename>/etc/python/debian_config</filename> allows configuration
how modules should be byte-compiled. The postinst scripts should respect these
settings.
</para>
<para>
Modules in private installation directories and in
<filename>/usr/lib/site-python</filename> should be byte-compiled, when the
default python version changes.
</para>
</section>
</chapter>
<chapter id="programs"><title>Python Programs</title>
<section id="version-indep-progs"><title>Programs using the default
python</title>
<para>
Programs that can run with any version of Python must begin with
<literal>#!/usr/bin/python</literal> or <literal>#!/usr/bin/env
python</literal> (the former is preferred). They must also specify a
dependency on <systemitem role="package">python</systemitem>, with a versioned
dependency if necessary.
</para>
<para>
If the program needs the python module <literal>foo</literal>, it must depend
on <systemitem role="package">python-foo</systemitem>.
</para>
<section id="current-version-progs"><title>Programs Shipping Private
Modules</title>
<para>
A program using <filename>/usr/bin/python</filename> as interpreter can come up
with private Python modules. These modules should be installed in
<literal>/usr/share/<replaceable>module</replaceable></literal>, or
<literal>/usr/lib/<replaceable>module</replaceable></literal> if the modules
are architecture-dependent (e.g. extensions).
</para>
<para>
<filename>/usr/lib/site-python</filename> is deprecated and should no longer be
used for this purpose.
</para>
<para>
The rules explained in <xref linkend="bytecompilation"/> apply to those private
modules: the bytecompiled modules must not be shipped with the package, they
should be generated in the package's postinst, using the current default Python
version, and removed in the prerm. Modules should be bytecompiled using the
current default Python version.
</para>
<para>
Programs that have private compiled extensions must either handle multiple
version support themselves, or declare a tight dependency on the current Python
version (e.g. <literal>Depends: python (>= 2.4), python (<= 2.5)</literal>.
No tools currently exist to alleviate this situation.
</para>
</section>
</section>
<section id="version-dep-progs"><title>Programs Using a Particular Python
Version</title>
<para>
A program which requires a specific version of Python must begin with
<literal>#!/usr/bin/python<replaceable>X</replaceable>.<replaceable>Y</replaceable></literal>
(or <literal>#!/usr/bin/env
python<replaceable>X</replaceable>.<replaceable>Y</replaceable></literal>). It
must also specify a dependency on <systemitem
role="package">python<replaceable>X</replaceable>.<replaceable>Y</replaceable></systemitem>
and on any <systemitem
role="package">python<replaceable>X</replaceable>.<replaceable>Y</replaceable>-foo</systemitem>
package providing necessary modules. It should not depend on any <systemitem
role="package">python-foo</systemitem> package, unless it requires a specific
version of the package (since virtual packages cannot be versioned). If this
is the case, it should depend on both the virtual package and the main package
(e.g. <literal>Depends: python2.4-foo, python-foo (>= 1.0)</literal>).
</para>
<para>
The notes on installation directories and bytecompilation for programs that
support any version of Python also apply to programs supporting only a single
Python version. Modules to be bytecompiled should use the same Python version
as the package itself.
</para>
</section>
</chapter>
<chapter id="embed"><title>Programs Embedding Python</title>
<section id="build-embedded"><title>Building Embedded Programs</title>
<para>
Programs which embed a Python interpreter must declare a
<literal>Build-Depends</literal> on <systemitem
role="package">python<replaceable>X</replaceable>.<replaceable>Y</replaceable>-dev</systemitem>,
where python<replaceable>X</replaceable>.<replaceable>Y</replaceable> is the
python version the program builds against. It should be the current default
python version unless the program doesn't work correctly with this version.
</para>
</section>
<section id="embedded-deps"><title>Embedded Python Dependencies</title>
<para>
Dependencies for programs linking against the shared Python library will be
automatically created by <command>dpkg-shlibdeps</command>. The
<literal>libpython<replaceable>X</replaceable>.<replaceable>Y</replaceable>.so.<replaceable>Z</replaceable></literal>
library the program is built against is provided by the <systemitem
role="package">python<replaceable>X</replaceable>.<replaceable>Y</replaceable></systemitem>
package.
</para>
</section>
</chapter>
<chapter id="other"><title>Interaction with Locally Installed Python
Versions</title>
<para>
As long as you don't install other versions of Python in your path, Debian's
Python versions won't be affected by a new version.
</para>
<para>
If you install a different subrelease of the version of python you've got
installed, you'll need to be careful to install all the modules you use for
that version of python too.
</para>
</chapter>
<appendix id="build-dependencies"><title>Build Dependencies</title>
<para>
Build dependencies for Python dependent packages must be declared for every
Python version that the package is built for. The <systemitem
role="package">python-all-dev</systemitem> should be used when building modules
for any or all Python versions. To build for a specific version or versions,
Build-Depend on <systemitem
role="package">python<replaceable>X</replaceable>.<replaceable>Y</replaceable>-dev</systemitem>.
</para>
<para>
Some applications and pure Python modules may be able to depend only on
<systemitem role="package">python</systemitem> or <systemitem
role="package">python-all</systemitem> and not require the -dev packages.
</para>
<para>
Build-Depend on at least:
</para>
<screen>
Build-Depends: python2.3 (>= 2.3-1)
Build-Depends: python2.4 (>= 2.4-1)
Build-Depends: python (>= 2.3.5-7)
Build-Depends: python-all
Build-Depends: python2.3-dev (>= 2.3-1)
Build-Depends: python2.4-dev (>= 2.4-1)
Build-Depends: python-dev (>= 2.3.5-7)
Build-Depends: python-all-dev
</screen>
<para>
If you use either <systemitem role="package">python-support</systemitem> or
<systemitem role="package">python-central</systemitem> you must additionally
Build-Depend on those. If you are using <command>dh_python</command> at all,
you must Build-Depend on <systemitem role="package">python</systemitem>, as
<systemitem role="package">debhelper</systemitem> does not depend on it.
</para>
</appendix>
<appendix id="packaging-tools"><title>Packaging Tools</title>
<para>
This section describes the various tools to help package Python programs and
modules for Debian. Although none of these tools are mandatory, their use is
strongly encouraged, as the above policy has been designed with them in mind
(and vice versa). This appendix is just an overview. If you use these tools,
you should read their full documentation.
</para>
<section id="pysupport"><title>python-support</title>
<para>
The python-support system provides a simple way to bytecompile pure Python
modules and manage dependencies. It integrates with <systemitem
role="package">debhelper</systemitem>. When using python-support, you should
install your modules to
<filename>/usr/share/python-support/<replaceable>package</replaceable></filename>
rather than the standard Python directories. python-support will then handle
compiling the modules and making appropriate symbolic links for installed
Python versions to find them, substitute <literal>${python:Depends}</literal>,
<literal>${python:Versions}</literal>, and
<literal>${python:Provides}</literal> in your control file, and manage
bytecompilation in your postinst/prerm.
</para>
<para>
To use it, call <command>dh_pysupport</command> before
<command>dh_python</command>, and make sure you've installed the modules in the
right place:
</para>
<screen>
PREFIX := debian/python-package/usr
...
install:
...
./setup.py install --no-compile \
--install-lib=$(PREFIX)/share/python-support/python-package
binary-indep: build install
...
dh_pysupport
dh_python
...
</screen>
<para>
python-support can also manage private modules. To use this feature, pass a
list of directories to be managed by python-support to
<command>dh_pysupport</command> and <command>dh_python</command>.
python-support cannot handle compiled extensions.
</para>
</section>
<section id="pycentral"><title>python-central</title>
<para>
python-central provides another way to manage Python modules. It integrates
with <systemitem role="package">debhelper</systemitem>, but can also be used
without it. When using python-central, you should install your modules
normally. It will then move them to its private directory, and manage the same
things python-support does.
</para>
<para>
To use it, call <command>dh_pycentral</command> before
<command>dh_python</command>:
</para>
<screen>
install:
...
./setup.py install
binary-indep: build install
...
dh_pycentral
dh_python
...
</screen>
<para>
python-central can handle compiled extensions for multiple Python versions. If
you want python-central to handle private modules, you must pass the list of
directories containing them to <command>dh_python</command> (but not
<command>dh_pycentral</command>).
</para>
<para>
If python-central should not move the files to its private directory,
use<command>DH_PYCENTRAL=nomove dh_pycentral</command> instead.
</para>
<para>
Examples for source packages using python-central are pyenchant, python-imaging
(modules and extensions), pyparallel (modules only).
</para>
</section>
<section id="cdbs"><title>CDBS</title>
<para>
FIXME: Someone familiar with CDBS should write this part.
</para>
</section>
</appendix>
<appendix id="upgrade"><title>Upgrade Procedure</title>
<para>
This section describes the procedure for the upgrade when the default python
version is changed in the <literal>unstable</literal> distribution, requiring
recompilation of many python-related packages.
</para>
<orderedlist numeration="arabic">
<listitem>
<para>
Have a long and heated discussion.
</para>
</listitem>
<listitem>
<para>
The Debian Python maintainer decides for the new default Debian Python version
and announces the upgrade.
</para>
</listitem>
<listitem>
<para>
Upload of the python core metapackages <systemitem
role="package">python</systemitem>, <systemitem
role="package">python-dev</systemitem>, <systemitem
role="package">python-doc</systemitem> and several <systemitem
role="package">python-<replaceable>module</replaceable></systemitem>, depending
on the new <systemitem
role="package">python<replaceable>X</replaceable>.<replaceable>Y</replaceable></systemitem>,
<systemitem
role="package">python<replaceable>X</replaceable>.<replaceable>Y</replaceable>-dev</systemitem>
and so on.
</para>
</listitem>
<listitem>
<para>
The release team schedules rebuilds for packages that may need it. Packages
that require manual work get updated and uploaded.
</para>
</listitem>
</orderedlist>
</appendix>
</book>
Python Packaging
================
Versions
--------
At any given time, the package python will represent the current default
Debian Python version.
The default Debian Python version should alway be the latest stable
upstream release that can be integrated in the distribution.
Apart from the default version, legacy versions of Python or beta
versions of future releases may be included as well in the distribution,
as long as they are needed by other packages, or as long as it seems
reasonable to provide them. (Note: For the scope of this document,
Python versions are synonymous to feature releases, i.e. Python 2.0 and
2.0.1 are subminor versions of the same Python version 2.0, but Python
2.1 and 2.2 are indeed different versions.)
For any version, the main package must be called pythonX.Y.
The set of currently supported python versions can be found in
``/usr/share/python/debian_defaults``.
Main package
------------
For every Python version provided in the distribution, the package
pythonX.Y shall comprise a complete distribution for *deployment* of
Python scripts and applications. The package includes the binary
``/usr/bin/pythonX.Y`` and all modules of the upstream Python
distribution.
Excluded are any modules that depend on non-\ *required* packages, they
will be provided in separate packages. Some tools and files for the
*development* of Python modules are split off in a separate package
pythonX.Y-dev. Documentation will be provided separately as well.
At any time, the python package must contain a symlink
``/usr/bin/python`` to the the appropriate binary
``/usr/bin/pythonX.Y``. The python package must also depend on the
appropriate pythonX.Y to ensure this binary is installed. The version of
the python package must be greater than or equal to X.Y and smaller than
X.Y+1.
Python Interpreter
------------------
Interpreter Name
~~~~~~~~~~~~~~~~
Python scripts depending on the default Python version (see
`section\_title <#base>`__) or not depending on a specific Python
version should use ``python`` (unversioned) as the interpreter name.
Python scripts that only work with a specific Python version must
explicitly use the versioned interpreter name (``pythonX.Y``).
Interpreter Location
~~~~~~~~~~~~~~~~~~~~
The preferred specification for the Python interpreter is
``/usr/bin/python`` or ``/usr/bin/pythonX.Y``. This ensures that a
Debian installation of python is used and all dependencies on additional
python modules are met.
If a maintainer would like to provide the user with the possibility to
override the Debian Python interpreter, he may want to use
``/usr/bin/env
python`` or ``/usr/bin/env
pythonX.Y``. However this is not advisable as it bypasses Debian's
dependency checking and makes the package vulnerable to incomplete local
installations of python.
Module Path
-----------
The module search path for Debian has been amended to include a
directory tree in /usr/local at the beginning of the path. By default,
sys.path is searched in the following order:
::
/usr/lib/pythonXY.zip
/usr/lib/pythonX.Y
/usr/lib/pythonX.Y/plat-linux2
/usr/lib/pythonX.Y/lib-tk
/usr/lib/pythonX.Y/lib-dynload
/usr/local/lib/pythonX.Y/site-packages
/usr/lib/pythonX.Y/site-packages
/var/lib/python-support/pythonX.Y
/usr/lib/pythonX.Y/site-packages/module-dir
/usr/lib/site-python
The use of the ``/usr/lib/site-python`` directory is deprecated. The
directory may be dropped from the path in a future version. The
/usr/lib/pythonXY.zip archive appeared in python2.3; it is not currently
used in Debian. Modules should not install directly to the
``/var/lib/python-support`` directory; it is for use by
`section\_title <#pysupport>`__.
Documentation
-------------
Python documentation is split out in separate packages pythonX.Y-doc.
The package python-doc will always provide the documentation for the
default Debian Python version.
TODO: Policy for documentation of third party packages.
Packaged Modules
================
The goal of these policies is to reduce the work necessary for Python
transitions. Python modules are internally very dependent on a specific
Python version. However, we want to automate recompiling modules when
possible, either during the upgrade itself (re-bytecompiling pyc and pyo
files) or shortly thereafter with automated rebuilds (to handle C
extensions). These policies encourage automated dependency generation
and loose version bounds whenever possible.
Types of Python Modules
-----------------------
There are two kinds of Python modules, "pure" Python modules, and
extension modules. Pure Python modules are Python source code that works
across many versions of Python. Extensions are C code compiled and
linked against a specific version of the libpython library, and so can
only be used by one version of Python.
Python packages are directories containing at least a ``__init__.py``,
other modules, extensions and packages (A package in the Python sense is
unrelated to a Debian package). Python packages must be packaged into
the same directory (as done by upstream). Splitting components of a
package across directories changes the import order and may confuse
documentation tools and IDEs.
There are two ways to distribute Python modules. Public modules are
installed in one of the directories listed in
`section\_title <#paths>`__. They are accessible to any program. Private
modules are installed in a directory such as ``/usr/share/packagename``
or ``/usr/lib/packagename``. They are generally only accessible to a
specific program or suite of programs included in the same package.
Module Package Names
--------------------
Public modules should be packaged with a name of python-foo, where foo
is the name of the module. Such a package should support the current
Debian Python version, and more if possible (there are several tools to
help implement this, see `appendix\_title <#packaging-tools>`__). For
example, if Python 2.3, 2.4, and 2.5 are supported, the Python command
::
import foo
should import the module when the user is running any of
``/usr/bin/python2.3``, ``/usr/bin/python2.4``, and
``/usr/bin/python2.5``. This requirement also applies to extension
modules; binaries for all the supported Python versions should be
included in a single package.
Specifying Supported Versions
-----------------------------
The ``XS-Python-Version`` field in ``debian/control`` specifies the
versions of Python supported by the package. This is used to track
packages during Python transitions, and is also used by some packaging
scripts to automatically generate appropriate Depends and Provides
lines. The format of the field may be one of the following:
::
XS-Python-Version: all
XS-Python-Version: current
XS-Python-Version: current, >= X.Y
XS-Python-Version: >= X.Y
XS-Python-Version: >= A.B, screen>
Where "all" means the package supports any Python version available, and
"current" means it supports Debian's current Python version. Explicit
Versions
or version ranges can also be used.
Your control file should also have a line:
XB-Python-Version: ${python:Versions}
The python:Versions is substituted by the supported Python versions of the
binary package, based on XS-Python-Version. (If you are not
using dh_python you will need to handle this substitution
yourself.) The format of the field XB-Python-Version is the
same as the XS-Python-Version field for packages not
containing extensions. Packages with extensions must list the versions
explicitely.
If your package is used by another module or application that requires a
specific Python version, it should also Provide:
pythonX.Y-foo
for each version it supports.
Dependencies
------------
Packaged modules available for the default Python version (or many
versions including the default) as described in
`section\_title <#package-names>`__ must depend on "python (>= X.Y)". If
they require other modules to work, they must depend on the
corresponding python-foo. They must not depend on any pythonX.Y-foo.
Packaged modules available for one particular version of Python must
depend on the corresponding pythonX.Y package instead. If they need
other modules, they must depend on the corresponding pythonX.Y-foo
packages, and must not depend on any python-foo.
Provides
--------
Provides in packages of the form python-foo must be specified, if the
package contains an extension for more than one python version. Provides
should also be added on request of maintainers who depend on a
non-default python version.
Packaged modules available for one particular version of Python must
depend on the corresponding pythonX.Y package instead. If they need
other modules, they must depend on the corresponding pythonX.Y-foo
packages, and must not depend on any python-foo.
Modules Bytecompilation
-----------------------
If a package provides any binary-independent modules (``foo.py`` files),
the corresponding bytecompiled modules (``foo.pyc`` files) and optimized
modules (``foo.pyo`` files) must not ship in the package. Instead, they
should be generated in the package's postinst, and removed in the
package's prerm. The package's prerm has to make sure that both
``foo.pyc`` and ``foo.pyo`` are removed.
A package should only byte-compile the files which belong to the
package.
The file ``/etc/python/debian_config`` allows configuration how modules
should be byte-compiled. The postinst scripts should respect these
settings.
Modules in private installation directories and in
``/usr/lib/site-python`` should be byte-compiled, when the default
python version changes.
Python Programs
===============
Programs using the default python
---------------------------------
Programs that can run with any version of Python must begin with
``#!/usr/bin/python`` or ``#!/usr/bin/env
python`` (the former is preferred). They must also specify a dependency
on python, with a versioned dependency if necessary.
If the program needs the python module ``foo``, it must depend on
python-foo.
Programs Shipping Private Modules
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
A program using ``/usr/bin/python`` as interpreter can come up with
private Python modules. These modules should be installed in
``/usr/share/module``, or ``/usr/lib/module`` if the modules are
architecture-dependent (e.g. extensions).
``/usr/lib/site-python`` is deprecated and should no longer be used for
this purpose.
The rules explained in `section\_title <#bytecompilation>`__ apply to
those private modules: the bytecompiled modules must not be shipped with
the package, they should be generated in the package's postinst, using
the current default Python version, and removed in the prerm. Modules
should be bytecompiled using the current default Python version.
Programs that have private compiled extensions must either handle
multiple version support themselves, or declare a tight dependency on
the current Python version (e.g.
``Depends: python (>= 2.4), python (literal>.
No tools currently exist to alleviate this situation.
``
Programs Using a Particular Python Version
------------------------------------------
A program which requires a specific version of Python must begin with
``#!/usr/bin/pythonX.Y`` (or ``#!/usr/bin/env
pythonX.Y``). It must also specify a dependency on pythonX.Y and on any
pythonX.Y-foo package providing necessary modules. It should not depend
on any python-foo package, unless it requires a specific version of the
package (since virtual packages cannot be versioned). If this is the
case, it should depend on both the virtual package and the main package
(e.g. ``Depends: python2.4-foo, python-foo (>= 1.0)``).
The notes on installation directories and bytecompilation for programs
that support any version of Python also apply to programs supporting
only a single Python version. Modules to be bytecompiled should use the
same Python version as the package itself.
Programs Embedding Python
=========================
Building Embedded Programs
--------------------------
Programs which embed a Python interpreter must declare a
``Build-Depends`` on pythonX.Y-dev, where pythonX.Y is the python
version the program builds against. It should be the current default
python version unless the program doesn't work correctly with this
version.
Embedded Python Dependencies
----------------------------
Dependencies for programs linking against the shared Python library will
be automatically created by ``dpkg-shlibdeps``. The
``libpythonX.Y.so.Z`` library the program is built against is provided
by the pythonX.Y package.
Interaction with Locally Installed Python Versions
==================================================
As long as you don't install other versions of Python in your path,
Debian's Python versions won't be affected by a new version.
If you install a different subrelease of the version of python you've
got installed, you'll need to be careful to install all the modules you
use for that version of python too.
Build Dependencies
==================
Build dependencies for Python dependent packages must be declared for
every Python version that the package is built for. The python-all-dev
should be used when building modules for any or all Python versions. To
build for a specific version or versions, Build-Depend on pythonX.Y-dev.
Some applications and pure Python modules may be able to depend only on
python or python-all and not require the -dev packages.
Build-Depend on at least:
::
Build-Depends: python2.3 (>= 2.3-1)
Build-Depends: python2.4 (>= 2.4-1)
Build-Depends: python (>= 2.3.5-7)
Build-Depends: python-all
Build-Depends: python2.3-dev (>= 2.3-1)
Build-Depends: python2.4-dev (>= 2.4-1)
Build-Depends: python-dev (>= 2.3.5-7)
Build-Depends: python-all-dev
If you use either python-support or python-central you must additionally
Build-Depend on those. If you are using ``dh_python`` at all, you must
Build-Depend on python, as debhelper does not depend on it.
Packaging Tools
===============
This section describes the various tools to help package Python programs
and modules for Debian. Although none of these tools are mandatory,
their use is strongly encouraged, as the above policy has been designed
with them in mind (and vice versa). This appendix is just an overview.
If you use these tools, you should read their full documentation.
python-support
--------------
The python-support system provides a simple way to bytecompile pure
Python modules and manage dependencies. It integrates with debhelper.
When using python-support, you should install your modules to
``/usr/share/python-support/package`` rather than the standard Python
directories. python-support will then handle compiling the modules and
making appropriate symbolic links for installed Python versions to find
them, substitute ``${python:Depends}``, ``${python:Versions}``, and
``${python:Provides}`` in your control file, and manage bytecompilation
in your postinst/prerm.
To use it, call ``dh_pysupport`` before ``dh_python``, and make sure
you've installed the modules in the right place:
::
PREFIX := debian/python-package/usr
...
install:
...
./setup.py install --no-compile \
--install-lib=$(PREFIX)/share/python-support/python-package
binary-indep: build install
...
dh_pysupport
dh_python
...
python-support can also manage private modules. To use this feature,
pass a list of directories to be managed by python-support to
``dh_pysupport`` and ``dh_python``. python-support cannot handle
compiled extensions.
python-central
--------------
python-central provides another way to manage Python modules. It
integrates with debhelper, but can also be used without it. When using
python-central, you should install your modules normally. It will then
move them to its private directory, and manage the same things
python-support does.
To use it, call ``dh_pycentral`` before ``dh_python``:
::
install:
...
./setup.py install
binary-indep: build install
...
dh_pycentral
dh_python
...
python-central can handle compiled extensions for multiple Python
versions. If you want python-central to handle private modules, you must
pass the list of directories containing them to ``dh_python`` (but not
``dh_pycentral``).
If python-central should not move the files to its private directory,
use\ ``DH_PYCENTRAL=nomove dh_pycentral`` instead.
Examples for source packages using python-central are pyenchant,
python-imaging (modules and extensions), pyparallel (modules only).
CDBS
----
FIXME: Someone familiar with CDBS should write this part.
Upgrade Procedure
=================
This section describes the procedure for the upgrade when the default
python version is changed in the ``unstable`` distribution, requiring
recompilation of many python-related packages.
1. Have a long and heated discussion.
2. The Debian Python maintainer decides for the new default Debian
Python version and announces the upgrade.
3. Upload of the python core metapackages python, python-dev, python-doc
and several python-module, depending on the new pythonX.Y,
pythonX.Y-dev and so on.
4. The release team schedules rebuilds for packages that may need it.
Packages that require manual work get updated and uploaded.
