Re: dh_python and python policy analysis
Hi, Another day, another draft. Here is the latest update for my take on the new Python policy document. The current version, and future updates, are to be found at http://www.golden-gryphon.com/software/manoj-policy/ I am including a text version below. manoj Packaging with the new Python policy A package developers view Manoj Srivastava Copyright (c) 2006 Manoj Srivastava Revision History Revision 1.0.4 25 Jul 2006 Obstacles to conformance with the new python policy. While the new Python policy, specifically the [1]"Packaged Modules" chapter, contains the elements that must be present in the debian/control filename, it is not very explicit about how the values are to be substituted. The Debian Wiki falls back on calling dh_python, which is not helpful in understanding the actual logic to be followed. This article is an attempt to correct this gap in documentation. -- Table of Contents 1. [2]Introduction 1.1. [3]Categorization of Python software 2. [4]Goals of the new Python policy 3. [5]Recipe for developers 3.1. [6]General Notes 3.1.1. [7]Python versions supported by the source 3.1.2. [8]Depends: 3.1.3. [9]Provides 3.1.4. [10]Build-Depends: 3.2. [11]Deprecating "current" in versions supported 3.3. [12]Script 3.3.1. [13]Supported versions 3.4. [14]Private Pure Python Modules 3.4.1. [15]Byte compilation 3.4.2. [16]Supported versions 3.5. [17]Private Extension 3.5.1. [18]Supported versions 3.6. [19]Public pure Python Module 3.6.1. [20]Byte compiling 3.6.2. [21]Supported versions 3.6.3. [22]Provides: 3.7. [23]Public Extension 3.7.1. [24]Supported versions 3.7.2. [25]Provides 4. [26]Changing the default Python version 4.1. [27]Python rtupdate scripts 4.1.1. [28]rtupdate script invocation 1. Introduction While trying to update SELinux packages, I ran across problems in trying to determine if my packages were complying with the new python policy: any practical tips for packaging generally devolved to the statement "Oh, just run dh_python". This is my attempt to offer more concrete tips for packaging. While this document started by reverse engineering dh_python, it has, thanks to help from various people more knowledgeable about Python than I, moved beyond that, and is closer to being a specification unfettered by the idiosyncrasies of current tools and implementations. -- 1.1. Categorization of Python software Program/script This consists of software directly called by an end user of external program, and is independently interpreted by the Python interpreter. Usually starts with the magic bytes #!, with the interpreter being /usr/bin/python* or /usr/bin/env python*. Modules This is code included in python "programs/scripts", and not invoked directly (serving as library modules in compiled languages). Modules can be categorized under two orthogonal criteria: firstly, based on the whether or not they are implemented purely in python, like so: Pure Python Module These are python source code, to be interpreted by the Python interpreter just like program/script code is, and may work across many versions of Python. Extension Module Extensions are C code compiled and linked against a specific version of the libpython library, and so can only be used by one version of Python. Another way of categorizing modules is based on whether or not they are available for use by third party scripts/modules. Public Public modules are available for use in other Python scripts or modules using the import directive. They are installed in one of the directories: /usr/lib/pythonX.Y This directory is reserved for official python modules. No other package apart from upstream official Python modules should install modules in this directory.
Re: dh_python and python policy analysis
On Sat, 12 Aug 2006 17:34:06 +0200, Matthias Klose <[EMAIL PROTECTED]> said: > Manoj Srivastava writes: >> policy document. The current version, and future updates, are to be >> found at http://www.golden-gryphon.com/software/manoj-policy/ Unfortunately, you are commenting on an old version of the document, so some comments refer to issue that have already been addressed. >> Extension Module >> >> Extensions are C code compiled and linked against a specific >> version of the libpython library, and so can only be used by one >> version of Python. > There should be no reason to link the extension against the python > library. Usually many extensions which are developed upstream on > Windows do link by default to libpython. Other extensions linking > against libpython are those with build infrastructure maybe > predating distutils. python-semanage is an example (and should not > link using -z defs). I'll add these in general notes, or perhaps in the extension specific sections (since compilation rules do not belong in the classification section). > Another thing to mention here is a "Python package", a directory > containing an __init__.py file plus modules and extensions. This is mentioned in general notes; I did not think it was a useful fifth classification, since we do not actually treat it very differently. >> Public >> >> Public modules are available for use in other Python scripts or >> modules using the import directive. They are installed in one of >> the directories: >> > maybe shorten that to "all directories in sys.path"; not sure if an > explicit list of directories is needed. Not all paths in sys.path are oermitted in packages (like the path ending in .zip). >> Packages may install public Python modules in directories specific >> to Python packaging utilities -- which specify the directories >> under which such modules should be droppped, and the the structure >> of these directories is defined by the utilities themselves. Please >> note that these directories are not in the path for Python, and are >> not available for modules to be imported from. At the time of >> writing, such uility specific directories include: >^^ >> >> /usr/share/pycentral /usr/share/python-support > These location are tool specific and should not be referenced > explicitely in the packaging scripts (debian/rules) You have just made all my selinux python packages violate this line :). I use update-python-modules in postinst and prerm, but I do not use any debhelper modules, and I should not be required to do so. I don't think that implicitly mandating the use of the helper scripts dh_(pysupport|pycentral|python) should belong in policy. >> 2. Goals of the new Python policy >> > No, not the whole design goal. Although the document is titled > "developer's view", the other goals should be mentioned as well. > These are meant to work around processes in debian which are > currently suboptimal, but unlikely to change. [SNIP lots of good stuff] I'll add these to the document. >> New python version introduced >> >> * Most pure Python modules with no restrictions on the >> version of Python supported, and those pure Python modules that >> only have a lower bound on the versions of python supported (for >> example, "2.3-", or "all"), would require no > "2.3-" -> ">= 2.3", not "all". the range notation cannot express > things like "2.2, >= 2.4". The use case for the latter is the jython > package (now removed from testing) still at an implementation level > of the corresponding cpython version, with i.e. 2.3 not a supported > python version anymore. So in the following text "set of versions", > instead of "range of versions" should be used. OK, done. >> The new policy also reduces the numbers of packages in the archive, >> by supporting multiple versions of Python in the same binary >> package (at the cost of increased size of that one package, but it >> should still result in space saving.) > Maybe mention the two cases, where the package size increases: > - extension modules > - pure modules where different versions of the upstream package are >shipped and are directly installed into /usr/lib/pythonX.Y/ OK. >> 3. Recipe for developers >> 3.1.1. Python versions supported by the source >> >> The XS-Python-Version field in debian/control specifies the >> versions of Python supported by
Re: dh_python and python policy analysis
On Sun, 13 Aug 2006 00:01:37 -0700, Steve Langasek <[EMAIL PROTECTED]> said: > On Sat, Aug 12, 2006 at 12:10:07PM -0500, Manoj Srivastava wrote: >> >> 3.1.3. Provides >> >> >> >> Packages with public modules and extensions should be named, or >> >> should provide, python-foo. Pure Python public modules that >> >> support all Python versions need not have a Provides field. >> > ... unless there is an application using a non-default python >> > version using this module. or else you require the application >> > depending on any indirect dependency of python-foo. >> Hmm. Two things: if application X requires my pure python public >> module (called, say, python-foo), and uses some specific version of >> python, why can't it depend on just python-foo Why do I have to >> provide pythonX.Y-foo? > Because a dependency on "python-foo" expresses the request "give me > the foo module for the current version of python". No, it does not. When a package bar depend on package baz, it means just that-- that it requires the package baz to work. With the policy draft that I have proposed, if it is a pure python module, with no intrinsic restrictions on the versions of python supported, other packages can just depend on the package, knowing that a policy compliant package would support all available versions. > There is no guarantee that the python-foo package installed is > compatible with, or provides support for, the pythonX.Y you're > using, except if this package declares a Provides: pythonX.Y-foo; so > the depends/provides: Rubissh. You are just making up these rules, and since it hurts, just don't make them up. If you want a pure python module that complies with the new policy, and does not provide pythonX.Y-foo; you know trhat you can just depend on python-foo, and things shall work. > pythonX.Y-foo needs to be there to ensure that the app and the > modules it needs aren't allowed to get out of sync on a user's > system (or in testing). And why would they get out of sync? If they are compliant, then when a new python version is installed the module is compiled for it -- so no matter what version of python you use, there is a compiled .pyc file there. >> Also, as a maintainer of python-foo, I can't know when such an >> application would be created, and we are trying to minimize >> reuploads of packages -- so either one provides all such >> pythonX.Y-foo at the get go, and reupload at every new python >> version or dropping of the old version -- or we upload every time >> some app is uploaded that may require yet abother X.y, and when we >> drop a version of Python. > Such apps would ideally be few and far between, but after thinking > about it for a while, I wasn't actually able to come up with a > concrete case where having the Provides: declared ahead of time > complicates transitions more than not having them would. For pure > python modules, this still means inconvenient sourceful reuploads > when new python implementations become available, since the > Provides: can't be declared for pythonX.Y that we don't yet know > about, but fortunately those reuploads would only need to be done on > demand for modules that are actually used from scripts invoking a > non-default python interpreter. And if you just follow the new policy, no uploads are needed at all. The new policy wins. manoj -- America: born free and taxed to death. Manoj Srivastava <[EMAIL PROTECTED]> <http://www.debian.org/%7Esrivasta/> 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
My take on the Python policy: draft
Hi, Here comes draft 1.0.4, with suggestions from Matthias Klose incorporated (well, most of them). The current and future updates, are to be found at http://www.golden-gryphon.com/software/manoj-policy/. The sources are at: http://www.golden-gryphon.com/software/manoj-policy/python-policy.xml, and are now licensed under th GPL (for some reason, docbook article mode suppresses LegalNotice). I am including a text version below. manoj Packaging with the new Python policy A package developers view Manoj Srivastava Developer The Debian Project Copyright (c) 2006 Manoj Srivastava Revision History Revision 1.0.4 12 Aug 2006 Revision 1.0.3 10 Aug 2006 Revision 1.0.2 8 Aug 2006 Revision 1.0.1 07 Aug 2006 Revision 1.0.0 31 Jul 2006 Specification of the new Python policy. This article grew as an attempt to correct a gap in the concrete specification of the new Python policy, and has grown to be close to a formal specification of the proposed new policy. -- Table of Contents 1. [1]Introduction 1.1. [2]Acknowledgements 2. [3]Goals of the new Python policy 3. [4]Nomenclature and definitions 3.1. [5]Python versions 3.1.1. [6]The Default Python version 3.2. [7]Categorization of Python software 4. [8]General Notes 4.1. [9]Naming python module packages 4.2. [10]Python versions supported by the source 4.3. [11]Byte compilation 4.4. [12]Linking extention modules 4.5. [13]Depends: 4.6. [14]Provides 4.7. [15]Build-Depends: 4.8. [16]Deprecating "current" in versions supported 5. [17]Recipe for developers 5.1. [18]Script 5.1.1. [19]Supported versions 5.2. [20]Private Pure Python Modules 5.2.1. [21]Byte compilation 5.2.2. [22]Supported versions 5.3. [23]Private Extension 5.3.1. [24]Supported versions 5.4. [25]Public pure Python Module 5.4.1. [26]Byte compiling 5.4.2. [27]Supported versions 5.4.3. [28]Depends: 5.5. [29]Public Extension 5.5.1. [30]Supported versions 5.5.2. [31]Provides 6. [32]Changing the default Python version 6.1. [33]Python rtupdate scripts 6.1.1. [34]rtupdate script invocation 1. Introduction While trying to update SELinux packages, I ran across problems in trying to determine if my packages were complying with the new python policy: any practical tips for packaging generally devolved to the statement "Oh, just run dh_python". This is my attempt to offer more concrete tips for packaging. While this document started by reverse engineering dh_python, it has, thanks to help from various people more knowledgeable about Python than I, moved beyond that, and is closer to being a specification unfettered by the idiosyncrasies of current tools and implementations. So now this document is a draft formal specification of the proposed new Python packaging policy. While it draws upon earlier documents, notable [35]Debian Python Policy , and the [36]new policy Wiki, the [37]Debian Python FAQ, and the source code for dh_python, and debhelper scripts, it has essentially been written from scratch, with material reworded, reorganized, and rearranged, to the extent that it bears little resemblance to the original sources. Compiled Python modules are very dependent on the specific Python version they were compiled against, and may otherwise have restrictions on the versions of Python they are compatible with. Unless care is taken, introducing, or dropping, new versions of Python, or changing the default version, trigger long and often painful transitions where the archive is inconsistent, and the systems is ill-integrated for the duration. This new Python policy seeks to address these potential messy transitions, and minimize the pain. -- 1.1. Acknowledgements While this document draws upon the expertise of multiple people and various documents, it has benefited especially from the patience and gentle corrections of people on the Debian-devel mailing list, and specifically from Josselin Mouette, Loíc Minier, Pierre Habouzit, a
Re: dh_python and python policy analysis
On Sun, 13 Aug 2006 10:28:43 -0700, Steve Langasek <[EMAIL PROTECTED]> said: > On Sun, Aug 13, 2006 at 03:32:27AM -0500, Manoj Srivastava wrote: >> On Sun, 13 Aug 2006 00:01:37 -0700, Steve Langasek >> <[EMAIL PROTECTED]> said: >> > On Sat, Aug 12, 2006 at 12:10:07PM -0500, Manoj Srivastava wrote: >> >> >> 3.1.3. Provides >> >> >> Packages with public modules and extensions should be named, >> >> >> or should provide, python-foo. Pure Python public modules >> >> >> that support all Python versions need not have a Provides >> >> >> field. >> >> > ... unless there is an application using a non-default python >> >> > version using this module. or else you require the application >> >> > depending on any indirect dependency of python-foo. >> >> Hmm. Two things: if application X requires my pure python >> >> public module (called, say, python-foo), and uses some specific >> >> version of python, why can't it depend on just python-foo Why do >> >> I have to provide pythonX.Y-foo? >> > Because a dependency on "python-foo" expresses the request "give >> > me the foo module for the current version of python". >> No, it does not. When a package bar depend on package baz, it means >> just that-- that it requires the package baz to work. With the >> policy draft that I have proposed, if it is a pure python module, >> with no intrinsic restrictions on the versions of python supported, >> other packages can just depend on the package, knowing that a >> policy compliant package would support all available versions. > Then what do you name a package that is a pure python module that > *DOES* have intrinsic restrictions on the version of python > supported? WShy do you need a special name? Had you read the actual proposed draft, you'd realize that we do cater to that possibility. > You are wrong to assume that a pure python module will automatically > support all available versions of python -- if all python versions > were completely backwards- and forwards-compatible with one another, > there would be no reason in the first place to *have* multiple > versions of them in the archive. > The reality is that there *are* language differences with each > implementation of python, and a pure python module may *not* work > with any given implementation of python available in the archive, > and we need a way to express such dependencies that guards users > against package relationships being satisfied by broken > combinations. In these two paragraphs you are doing three things. a) Belabouring the obvious b) jumping to conclusions about what I assume, c) demonstrating that you have not read the proposed draft. >> > There is no guarantee that the python-foo package installed is >> > compatible with, or provides support for, the pythonX.Y you're >> > using, except if this package declares a Provides: pythonX.Y-foo; >> > so the depends/provides: >> Rubissh. You are just making up these rules, and since it hurts, >> just don't make them up. > What hurts is your ignorance of design requirements that were > discussed at length at the DebConf python BoF. A little knowledge is dangerous. Go and educate yourself on what is being discussed in this thread -- or, if you are too almighty busy to familiarize yourself with the things you weigh in on, let meattempt to summarize, eliding any "oooh shiny" sections: If you are packaging a public pure python module (not extension or private modules), there are two cases: either you have intrinsic versions dependencies in your module, or not. If you have version dependencies, your Depends and Provides rules are differennt, and you need to read the draft to see what they are. If there are no such internal dependencies, then you need not provide pythonX.Y-foo virtual packages, since you should arrange for your packages to be compiled for any new version that is uploaded. As to the BOF thing, I'll bite: Why one earth did the bof come up with design decisiosn that require every single goldarned python module package to be reuploaded every time a new version of python is added or removed? Why did the BOF choose to ignore the experience of the emacs listp community, that also has byte compilation issues, and subtle differences in byte compiler? Were the people in the bof aware that the python interpreter, like emacs itself, can optionally byte compile source on the fly, and lal that is lost is some speed? Frankly, if the BOF came up with a design that requires every s
Re: dh_python and python policy analysis
Hi, OK, I see I have to dot the i's and cross the t's for this case here. So, here is the scenario: package python-foo packages a public pure python module. Package bar imports the module foo. Package baz is a package not yet written that would be written for Python2.6 that would also need module foo, but only when we actually get python2.6. Let us also have a package bar-baz that is written for python2.5. Also, ket us assume the module foo would work for versions 2.3, 2.4, 2.5 -- but in the yet unreleased version 2.6, stuff changes, and module foo would not be compatible as written. OK? State of Python at the start of the thought experiment: current is 2.3, available is also 2.4, and let us pretend no one has heard of 2.5 or 2.6 yet. With me so far? Now we have two policy proposals, A, and B. A decrees that python-foo depends on python, ad has no provides. Policy B requires that python-foo also provide python2.3-foo and python2.4-foo. The following transition events occur. 1) Python 2.5 is added. policy Apolicy B no upload. python-foo recompiledUpload python-foo, adding for 2.5 provides for 2.5 No transition for package bar package bar-baz must wait or bar-baz the upload. 2) Default python version changes to 2,4 3) Python2.3 is dropped. policy Apolicy B no upload. Upload python-foo, removing provides for 2.3 4) Python 2.6 is added. Here there are two cases. Either module foo can't be written at all for version 2.6, or it the same functionality can be provided with a code change, perhaps hidden behind a version conditional. How often have people seen a regression in Python that something that was doable for version N can't be done at all in version N + 1? 4a) foo can be coded for version 2.6 Policy A policy B Package uploaded, with the changed Package uploaded, with the source. package baz has to wait changed source, and with the provides. Package baz has to wait. 4b) foo can't be written for version 2,6, or will take time, and support for 2,6 is dropped (at least for the moment) Policy A policy B Package uploaded, with provides, Package uploaded, with provides Packages bar, bar-baz, and baz Package baz has to wait. (rdepends python-foo) informed of the provides. Need to upload the rdepends Now, most pure python packages will never see option 4 at all; and those that do, a number will be case 4a. Even for the case of 4b, there is time to do the transition for packages bar and bar-baz -- until 2.6 becomes the default, there is no critical bug in bar or bar-baz. Now take this to every single pure python module package in Debian, multiply with the upload by default for every single addition or removal of python packages, and you can see that adding more work in the corner case 4b is worth not having to upload packages multiple times by default. manoj -- I used to be Snow White, but I drifted. Mae West Manoj Srivastava <[EMAIL PROTECTED]> <http://www.debian.org/%7Esrivasta/> 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: dh_python and python policy analysis
On Sun, 13 Aug 2006 23:37:15 +0200, Pierre Habouzit <[EMAIL PROTECTED]> said: > Le dim 13 août 2006 22:17, Manoj Srivastava a écrit : >> As to the BOF thing, I'll bite: Why one earth did the bof >> come up with design decisiosn that require every single goldarned >> python module package to be reuploaded every time a new version of >> python is added or removed? > actually,it's not truly needed, it's needed iff a package do needs > the new provides or not. > and those reuploads are kind of binNMUs, the real problem here is > that tehre is no arch:all binNMU and maybe that's here the problem > that need fixing. They are sourceful NMU's: since the packaged source comprises of the .dsc, and that includes the full provides line, as well as the information in the source Packages file, etc, also needs to changes. The draft does not require any more uploads in the default case than absolutely required. I think the corner case that something that was feasible in the old version but not in th new one is a low probability event, really, so making that case a trifle inconvenient to ease the routine transitions of adding/removing python versions is a good tradeoff. manoj -- Novinson's Revolutionary Discovery: When comes the revolution, things will be different -- not better, just different. Manoj Srivastava <[EMAIL PROTECTED]> <http://www.debian.org/%7Esrivasta/> 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: dh_python and python policy analysis
On Mon, 14 Aug 2006 11:21:07 +1000, Robert Collins <[EMAIL PROTECTED]> said: > On Sun, 2006-08-13 at 19:56 -0500, Manoj Srivastava wrote: >> Here there are two cases. Either module foo can't be written at all >> for version 2.6, or it the same functionality can be provided with > This is a little simplistic. > The parser changes fairly routinely in python versions. This means > that a version conditional is not sufficient to provide > compatability with older pythons - the module will not parse. > The usual thing done for cross version support is to write in the > older version of python, or in extreme cases (i.e. where performance > really hurts) have two separate modules _foo_2_5 and _foo_2_6 and > conditionally do 'from _foo_2_5 import *' etc. Either conditional code or conditional inclusion are variations on the common theme I alluded to in my admittedly, and designedly, simplified example. manoj -- There's too much beauty upon this earth for lonely men to bear. Richard Le Gallienne Manoj Srivastava <[EMAIL PROTECTED]> <http://www.debian.org/%7Esrivasta/> 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: dh_python and python policy analysis
On Sun, 13 Aug 2006 23:03:13 -0700, Steve Langasek <[EMAIL PROTECTED]> said: > On Sun, Aug 13, 2006 at 07:56:09PM -0500, Manoj Srivastava wrote: >> OK, I see I have to dot the i's and cross the t's for this case >> here. So, here is the scenario: package python-foo packages a >> public pure python module. Package bar imports the module >> foo. Package baz is a package not yet written that would be written >> for Python2.6 that would also need module foo, but only when we >> actually get python2.6. Let us also have a package bar-baz that is >> written for python2.5. > AFAICS you've correctly described the requirements for this > scenario, but this is not the scenario that I've been trying to > point out to you. > Now, introduce version 2.0 of python-foo. Because upstream > considers python 2.3 obsolete, they have begun using language > features in their module (internally, not as part of the module > interface which remains unchanged) that are specific to python 2.4 > and above. This is similar to the case 4b in my scenario: a package changes from supporting all known versions of python to supporting a subset of that range. As in my case 4B, you don't take this action lightly, while there is python2.3 in the archive, you do a conditional import of the old version of the module in order to retain the compatibility, or you talk to your rdependsand tell them about the change. > Now, looking at your example: >> The following transition events occur. > No disagreements on 1) and 2). >> 3) Python2.3 is dropped. >> policy A policy B no upload. Upload python-foo, removing provides >> for 2.3 > Why does python-foo need to drop the provides for 2.3? In what way > does python2.3's removal from the archive mean that python-foo has > stopped supporting python2.3? Because when python2.3 is removed, the site-packages directory would also be purged of all automatically compiled versions of the pure python modules. Indeed, I think this is already done; and minimizes the house cleaning packages have to perform. > Also, >> 4b) foo can't be written for version 2,6, or will take time, and >> support for 2,6 is dropped (at least for the moment) Policy A >> policy B Package uploaded, with provides, Package uploaded, with >> provides Packages bar, bar-baz, and baz Package baz has to wait. >> (rdepends python-foo) informed of the provides. Need to upload the >> rdepends > This is a case where I don't see any need to reupload foo under > policy B: it already has all the provides it's going to have, since > it didn't yet know about python2.6 and therefore couldn't have > declared any provides for it, no? So python-foo Provides: > python2.5-foo because bar-baz is written for python2.5 and needs it; > bar is AFAICS written to use "python", so only depends on > python-foo; and baz has to wait. Well, if you are not going to upload, you can't be using python-support or python-central as the tools to automatically compile your module -- and unless you have changed the .versions file or XS-Python-Version variable, They'll try to compile, and may leave a miscompiled version around, depending on how subtle the 2.6 incompatibility is. === 1) Python 2.5 is added | no upload. python-foo | Upload python-foo, | recompiled for 2.5| adding provides for | | 2.5 | No transition for | package bar-baz must | package bar or| wait the upload. | bar-baz | -- 2) Default python | | version changes to | | 2.4| | -- 3) Python2.3 is dropped| | Upload python-foo, | | removing provides for | | 2.3 (since presumably | | the compiled files | | are gone -- 4) Python 2.6 is added.| | A) Source change fix | Package uploaded, with| Package uploaded, with | changed source. | changed source, and | | the provides. | package baz has to| package baz has t | wait
Re: dh_python and python policy analysis
Hi, I have some more thoughts to offer on the example Steve presented: in essence, he is talking about a package that has become incompatible with the version that was in stable. On Sun, 13 Aug 2006 23:03:13 -0700, Steve Langasek <[EMAIL PROTECTED]> said: > On Sun, Aug 13, 2006 at 07:56:09PM -0500, Manoj Srivastava wrote: > Now, introduce version 2.0 of python-foo. Because upstream > considers python 2.3 obsolete, they have begun using language > features in their module (internally, not as part of the module > interface which remains unchanged) that are specific to python 2.4 > and above. This could happen to _any_ package written in any other language. Policy does not go out of its way to protect the reverse dependencies of a package that breaks compatibility between stable and testing -- were the package written in any other language. Why should Python be treated differently, if it makes for more uploads and makes Python transitions slower? What _do_ we do in other cases? There is precedence for creating a brand new package, called foo2 (X100/11, f;ex, apache, fvwm, and a load of others) so that new users can use the new functionality, while users of older package have time to transition. Maintainers can modify packages for Debian to retain compatibility. Or, worst case, we shrug and tell people that sorry, not in all cases can one maintain compatibility, so people should change the dependencies. I see no reason to go to heroic measures just for the sake of packages programmed in Python as opposed to, say, C -- given the drawbacks I have already mentioned in previous posts. manoj -- A nuclear war can ruin your whole day. Manoj Srivastava <[EMAIL PROTECTED]> <http://www.debian.org/%7Esrivasta/> 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: dh_python and python policy analysis
Hi, This is version 1.05 of the draft, now with typo fixes, and some initial policy support for partial upgrades for pure python public modules that are trying to drop support for older versions of python. The idea is that error cases are minimized if we do not drop a version of python in a new stable that was the default version in oldstable; at the same time insisting that every pre python public module that says "all" in the versions supported field must support all shipping versions of python in stable (even if this takes extra effort in the form of splitting off private modules and including them conditionally based on python version from a wrapper module that does little but conditional imports. The current version, and future updates, are to be found at http://www.golden-gryphon.com/software/manoj-policy/ manoj Packaging with the new Python policy A package developers view Manoj Srivastava Developer The Debian Project Copyright (c) 2006 Manoj Srivastava Revision History Revision 1.0.5 4^th November 2006 Revision 1.0.4 12 Aug 2006 Revision 1.0.3 10 Aug 2006 Revision 1.0.2 8 Aug 2006 Revision 1.0.1 07 Aug 2006 Revision 1.0.0 31 Jul 2006 Specification of the new Python policy. This article grew as an attempt to correct a gap in the concrete specification of the new Python policy, and has grown to be close to a formal specification of the proposed new policy. -- Table of Contents 1. [1]Introduction 1.1. [2]Acknowledgements 2. [3]Goals of the new Python policy 3. [4]Nomenclature and definitions 3.1. [5]Python versions 3.1.1. [6]The Default Python version 3.2. [7]Categorization of Python software 4. [8]General Notes 4.1. [9]Naming python module packages 4.2. [10]Python versions supported by the source 4.3. [11]Byte compilation 4.4. [12]Linking extention modules 4.5. [13]Depends: 4.6. [14]Provides 4.7. [15]Build-Depends: 4.8. [16]Deprecating "current" in versions supported 5. [17]Recipe for developers 5.1. [18]Script 5.1.1. [19]Supported versions 5.2. [20]Private Pure Python Modules 5.2.1. [21]Byte compilation 5.2.2. [22]Supported versions 5.3. [23]Private Extension 5.3.1. [24]Supported versions 5.4. [25]Public pure Python Module 5.4.1. [26]Byte compiling 5.4.2. [27]Supported versions 5.4.3. [28]Depends: 5.5. [29]Public Extension 5.5.1. [30]Supported versions 5.5.2. [31]Provides 6. [32]Changing the default Python version 6.1. [33]Python rtupdate scripts 6.1.1. [34]rtupdate script invocation 1. Introduction While trying to update SELinux packages, I ran across problems in trying to determine if my packages were complying with the new python policy: any practical tips for packaging generally devolved to the statement "Oh, just run dh_python". This is my attempt to offer more concrete tips for packaging. While this document started by reverse engineering dh_python, it has, thanks to help from various people more knowledgeable about Python than I, moved beyond that, and is closer to being a specification unfettered by the idiosyncrasies of current tools and implementations. So now this document is a draft formal specification of the proposed new Python packaging policy. While it draws upon earlier documents, notable [35]Debian Python Policy , and the [36]new policy Wiki, the [37]Debian Python FAQ, and the source code for dh_python, and debhelper scripts, it has essentially been written from scratch, with material reworded, reorganized, and rearranged, to the extent that it bears little resemblance to the original sources. Compiled Python modules are very dependent on the specific Python version they were compiled against, and may otherwise have restrictions on the versions of Python they are compatible with. Unless care is taken, introducing, or dropping, new versions of Python, or changing the default version, trigger long and often painful transitions where the archive is inconsistent, and the systems is ill-integrated for the duration. This new Python policy seeks to address these potential messy tr
The new python policy and packaging
Hi, [Please keep the CC to debian-devel, or please CC me, if yu are replying to this on debian-python@lists.debian.org, since I am not subscribed to that list] I have been trying to ensure that my SELinux related packages which have a python component are compliant with the new Python policy[1]. The wiki page for the policy[2] seems to be mostly a dh_python usage manual, and defers to dh_python to set up the variable substitutions correctly, so I have been trying to write up my own manual based on the policy and reverse engineering dh_python. In the course of this, there were points which I found very confusing in the Python policy which I think need to be cleared up. Section 2.2: Firstly, public modules are supposed to be packaged in a package called python-foo. What about public extensions? section 2.2 talks about naming, and also talks about supporting current python version. The last sentence in section 2.2 says: "This requirement also applies to extension modules; binaries for all the supported Python versions should be included in a single package." It is not clear whether "This requirement" refers to the naming, or the support. Perhaps policy should explicitly say "Public modules _and extensions_ should be packaged with a name of python-foo," (assuming that I now have the correct interpretation). (BTW, the wiki page seems to differentiate more strongly between modules and extensions: python extension: a .so file for use by python applications python module: a .py file for use by python applications) section 2.4: Can anyone explain why if my python packages depend on Python version 2.4, and selinux.so for python 2.4, it should depend on python2.4 and python2.4-selinux; but when the default changes to 2,4, my package dependencies would beed to change to python (>= 2.4) and pythong-selinux? (This according to section 2.4 of [1]) It is not clear why a package that depends strictly on python 2.3, which is the current default, should fall into paragraph one of the section, and not into paragraph two. ,[ section 2.4, paragraph 1 ] | Packaged modules available for the default Python version (or many | versions including the default) as described in Module Package Names, | Section 2.2 must depend on "python (>= X.Y)". If they require other | modules to work, they must depend on the corresponding | python-foo. They must not depend on any pythonX.Y-foo. ` ,[ section 2.4, paragraph 2 ] | Packaged modules available for one particular version of Python must | depend on the corresponding pythonX.Y package instead. If they need | other modules, they must depend on the corresponding pythonX.Y-foo | packages, and must not depend on any python-foo ` I assumed it fell into paragraph one, since it is explicitly stated so in policy. Logically, though, only paragraph two should apply. In any case, this must be disambiguated. section 2.5: The second paragraph of section 2,5 is a duplicate of the second paragraph of section 2,4. This seems like a cut-and-paste error, and directives appear to be missing from section 2.5. There should be something about providing pythonX.Y-foo if your package provides an extension for version X.Y of python. I'll add to this list as I get further along in my reading. manoj [1]http://www.debian.org/doc/packaging-manuals/python-policy/ [2]http://wiki.debian.org/DebianPython/NewPolicy ps: I think policy should state clearly how it expects the various fields in the control file to be initialized, rather than leaving it mostly unstated and deferring to an external helper program, but that is another thread -- Without fools there would be no wisdom. Manoj Srivastava <[EMAIL PROTECTED]> <http://www.debian.org/%7Esrivasta/> 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: The new python policy and packaging
On Thu, 27 Jul 2006 14:15:47 -0500, Joe Wreschnig <[EMAIL PROTECTED]> said: > On Thu, 2006-07-27 at 11:56 -0500, Manoj Srivastava wrote: >> Hi, >> >> [Please keep the CC to debian-devel, or please CC me, if yu are >> replying to this on debian-python@lists.debian.org, since I am not >> subscribed to that list] >> >> I have been trying to ensure that my SELinux related packages which >> have a python component are compliant with the new Python >> policy[1]. The wiki page for the policy[2] seems to be mostly a >> dh_python usage manual, and defers to dh_python to set up the >> variable substitutions correctly, so I have been trying to write up >> my own manual based on the policy and reverse engineering >> dh_python. >> >> In the course of this, there were points which I found very >> confusing in the Python policy which I think need to be cleared up. >> >> Section 2.2: Firstly, public modules are supposed to be packaged in >> a package called python-foo. What about public extensions? section >> 2.2 talks about naming, and also talks about supporting current >> python version. The last sentence in section 2.2 says: "This >> requirement also applies to extension modules; binaries for all the >> supported Python versions should be included in a single package." >> It is not clear whether "This requirement" refers to the naming, or >> the support. >> >> Perhaps policy should explicitly say "Public modules _and >> extensions_ should be packaged with a name of python-foo," >> (assuming that I now have the correct interpretation). >> >> (BTW, the wiki page seems to differentiate more strongly between >> modules and extensions: python extension: a .so file for use by >> python applications python module: a .py file for use by python >> applications) > Your confusion is due to an incorrect distinction. .py and .so files > are both modules; .so files are "extension modules" and .py files > are "pure Python modules". When the word "module" is used alone it > always refers to both. Then please explain section 2.2: "Public modules should be packaged with a name of python-foo, where foo is the name of the module. This requirement also applies to extension modules; binaries for all the supported Python versions should be included in a single package. " The wording implies that Public modules are distinct from extension modules (since something stated for public modules also applies to extension modules. Policy otherwise states public vs private and pure modules vs extension modules are orthogonal, so there is an inconsistency here. > This is made clear in the policy; > http://www.debian.org/doc/packaging-manuals/python-policy/ch-module_packages.html#s2.1 Inconsistently so, as noted above. > If the wiki uses "modules" to refer to only pure Python modules, it > is incorrect. Then I suggest someone who knows about Python policy please update the Wiki page to correct it. >> section 2.4: >> Can anyone explain why if my python packages depend on Python >> version 2.4, and selinux.so for python 2.4, it should depend on >> python2.4 and python2.4-selinux; but when the default changes to >> 2,4, my package dependencies would beed to change to python (>= >> 2.4) and pythong-selinux? > Because then applications using the default python (/usr/bin/python) > can access it, and that will be true for versions beyond 2.4. The > versioned names are not appropriate for packages using the default > version. Why do my packages need to change the dependencies, then? Because when the default changes to 2.5, the dependency would have to change yet again, for no changes in the package itself. > If you always need Python 2.4, even when Python 2.5 is the default, > then the python2.4-foo deps are still appropriate. Right. But that is an answer to a different question. >> It is not clear why a package that depends strictly on python 2.3, >> which is the current default, should fall into paragraph one of the >> section, and not into paragraph two. >> >> ,[ section 2.4, paragraph 1 ] | Packaged modules available for >> the default Python version (or many | versions including the >> default) as described in Module Package Names, | Section 2.2 must >> depend on "python (>= X.Y)". If they require other | modules to >> work, they must depend on the corresponding | python-foo. They must >> not depend on any pythonX.Y-foo. ` >> >> ,[ section 2.4, paragraph 2 ] | Packaged modules available for >>
More dh_python questions
); 483 } 484 } 485 } -- Hmm, python_header is initialized from XS-Python-Version, which, if it were mandatory, would mean that we would get X.Y in either case. So, we get a XB-Python-Version, and we get a Provides, in either case. But if XS-Python-Version is missing from python-foo, we would get a XB-Python-Version: all and no provides, even though we told dh_python that only one version of python can run our .py files. manoj -- Trying to break out of the Tempter's control, one's mind writhes to and fro, like a fish pulled from its watery home onto dry ground. 34 Manoj Srivastava <[EMAIL PROTECTED]> <http://www.debian.org/%7Esrivasta/> 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: More dh_python questions
Hi,
I have a growing suspicion that dh_python does not do the
right thing for private pure Python modules in the presence of
XS-Python-version
This is how dh_python behaves:
*** PRIVATE PURE MODULE:
If there is a .py file, and it is in a private dir. only one
version is supported, even though the private module can be
recompiled for any new version.
Add dir to $private_dirs_list{"$dir"} (for recompilation)
add to the deps var if no -V option is given.
* If versions_field is unset (so precedence goes to extensions),
** if $min_version and min_version=max_version (1 version supported)
$versions_field = $min_version
** else $versions_field = current ?
or set $verdeps{X.Y} if only version X.Y is supported (-V)
* Each entry is in the verdeps hash gets added to depends
No pyversions_found.
**Set Versions to $versions_field
So, if I have a package with a private pure python module, and
I have set XS-Python-Version to suggest that a ubset of python
versions are supported, and I have not used -V (since more than one
python version is to be supported).
What happens?
Case A:
I have only one version in XS-Python-Version (say, 2.4)
,
| Package: python-foo
| XB-Python-Version: X.Y
| Provides: pythonX.Y-foo
`
But the package does not depend on pythonX.Y, which I think
it should (since we have stated that only X.Y is supported).
Case B:
There is no lower bound, but there is an upper bound
,
| Package: python-foo
| XB-Python-Version: 2.3
| Provides: python2.3-foo
`
Hmm. I suppose this is OK, since if we do not support the
current version, we should not be uploading the package.
Case C:
I have a range in the XS-Python-Version
Case C1: The current version is not supported (say, I support 2.4
and 2.5)
,
| Package: python-foo
| XB-Python-Version: 2.3
| Provides: python2.3-foo
`
This is wrong.
Case C2: The current version is supported
,
| Package: python-foo
| XB-Python-Version: 2.3
| Provides: python2.3-foo
`
This is OK as well.
So, I posit that dh_python, as currently coded, does not
handle case A and C1 correctly.
manoj
--
Sex is like air. It's only a big deal if you can't get any.
Manoj Srivastava <[EMAIL PROTECTED]> <http://www.debian.org/%7Esrivasta/>
1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
dh_python and python policy analysis
Hi, I have finished my initial analysis of Python policy and dh_python, and created a rough specification of what the python policy is supposed to be (based on current dh_python behaviour). The current analysis, and future updates, are to be found at http://www.golden-gryphon.com/software/manoj-policy/ The document is a draft, since I have not been involved in Python development, it may have flaws, and I am hoping that people more conversant with Python development would point them out to me. The document could also stand some polishing; and since it was written piecemeal, continuity leaves much to be desired as yet. I am including a text version below. manoj Packaging with the new Python policy A package developers view Manoj Srivastava Copyright (c) 2006 Manoj Srivastava Revision History Revision 1.0 25 Jul 2006 Obstacles to conformance with the new python policy. While the new Python policy, specifically the [1]"Packaged Modules" chapter, contains the elements that must be present in the debian/control filename, it is not very explicit about how the values are to be substituted. The Debian Wiki falls back on calling dh_python, which is not helpful in understanding the actual logic to be followed. This article is an attempt to correct this gap in documentation. -- Table of Contents 1. [2]Introduction 1.1. [3]Categorization of Python software 2. [4]Requirements for packages (new policy) 2.1. [5]XS-Python-Version: 2.2. [6]XB-Python-Version: 2.3. [7]Depends: 2.4. [8]Provides 3. [9]Recipe for developers 3.1. [10]Based on type of python modules being packaged 3.1.1. [11]Script 3.1.2. [12]Private Pure Python Modules 3.1.3. [13]Private Extension 3.1.4. [14]Public pure Python Module 3.1.5. [15]Public Extension 1. Introduction While trying to update SELinux packages, I ran across problems in trying to determine if my packages were complying with the new python policy: any practical tips for packaging generally devolved to the statement "Oh, just run dh_python". This is my attempt to offer more concrete tips for packaging, by reverse engineering dh_python for the specifications and tips. -- 1.1. Categorization of Python software Program/script This consists of software directly called by an end user of external program, and is independently interpreted by the Python interpreter. Usually starts with the magic bytes #!, with the interpreter being /usr/bin/python* or /usr/bin/env python*. Modules This is code included in python "programs/scripts", and not invoked directly (serving as library modules in compiled languages). Modules can be categorized under two orthogonal criteria: firstly, based on the whether or not they are implemented purely in python, like so: Pure Python Module These are python source code, to be interpreted by the Python interpreter just like program/script code is, and may work across many versions of Python. Extension Module Extensions are C code compiled and linked against a specific version of the libpython library, and so can only be used by one version of Python. Another way of categorizing modules is based on whether or not they are available for use by third party scripts/modules. Public Public modules are available for use in other Python scripts or modules using the import directive. They are installed in one of the directories /usr/lib/pythonX.Y/site-packages /usr/lib/pythonX.Y /var/lib/python-support/pythonX.Y /usr/share/pycentral /usr/share/python-support Private Private modules are generally only accessible to a specific program or suite of programs included in the same package. They are installed in special directories, for example: /usr/lib/ /usr/share/ /usr/lib/games/ /usr/share/games/ -- 2. Requirements for packages (new policy) The new python policy places certain requirements for packages that contain Python bits. -- 2.1. XS-Python-Version:
Re: dh_python and python policy analysis
On Tue, 01 Aug 2006 09:55:39 +0200, Josselin Mouette <[EMAIL PROTECTED]> said:
> Le lundi 31 juillet 2006 à 21:10 -0500, Manoj Srivastava a écrit :
>> Public modules are available for use in other Python scripts or
>> modules using the import directive. They are installed in one of
>> the directories
>>
>> /var/lib/python-support/pythonX.Y /usr/share/python-support
> Note that these two contain the same modules. The /usr/share
> directory isn't read by python, only the generated tree in /var is.
Thanks, I'll explicitly make a note of that in that section.
>> The new python policy places certain requirements for packages that
>> contain Python bits.
>> 2.1. XS-Python-Version: 2.2. XB-Python-Version:
> These two are by no means requirements. XS-Python-Version is only a
> way to tell the packaging tools which versions to use, but you can
> also use debian/pyversions which is the recommended way as it
> doesn't pollute control files.
Hmm. I'll remove mention of it, then. How exactly one
invokes packaging tools should not be in this document; which is
trying to be a specification more than a packaging guide.
> XB-Python-Version is a way to generate metadata but it isn't
> necessary either. The same applies to all you've written about these
> fields.
The draft Python policy states:
,[ Section 2.3 ]
| Your control file should also have a line:
| XB-Python-Version: ${python:Versions}
| The python:Versions is substituted by the supported Python versions of
| the binary package, based on XS-Python-Version. (If you are not using
| dh_python you will need to handle this substitution yourself.) The
| format of the field XB-Python-Version is the same as the
| XS-Python-Version field for packages not containing
| extensions. Packages with extensions must list the versions
| explicitely.
`
http://www.debian.org/doc/packaging-manuals/python-policy/ch-module_packages.html
Is the draft policy incorrect in this case? (The directive is
a should.)
>> 2.3. Depends:
>>
>> Packaged modules available for the default Python version (or many
>> versions including the default) must depend on python (>= X.Y). If
>> they require other modules to work, they must depend on the
>> corresponding python-foo. They must not depend on any
>> pythonX.Y-foo.
> For the packages to be consistent, the package should depend on all
> pythonX.Y-foo for the versions listed in Provides:.
I'll add this note, thanks.
> However, no packaging tool is currently able to generate this
> information.
Well, that's all right. First we decide what is the right
thing to do, then we provide tools. Packaging tools are there to
assist us, after all, not to limit us.
>> 2.4. Provides
>>
>> Packages with public modules and extensions should be named, or
>> should provide, python-foo, if the package contains an extension
>> for more than one python version. Also, for every version of python
>> supported the package should provide pythonX.Y-foo packages.
> In fact, it should not provide this unless it has correct
> dependencies on all pythonX.Y-bar - but everyone is doing this
> wrong.
Thanks, I'll try an go through to document to ensure I have
the specifications done correctly.
>> 3.1.5. Public Extension
>>
>> Public extensions should be packaged with a name of python-foo,
>> where foo is the name of the module. Such a package should support
>> the current Debian Python version, and more if possible.
> Maybe a word on how public extensions and public python modules
> interact would be nice.
I'd appreciate any suggestions.
> Generally speaking, I don't find this document useful to the package
> maintainer. It focuses mostly on python-central's internals, which
> don't need to be fully understood by the maintainer, and which
> aren't useful if you don't use python-central.
It is curious that you say that, since I have not yet looked
at pycentral, what you see here is derived ojnly from reading the
draft policy in detail and then reverse engineering what dh_python
does.
The motivation for this exercise was for me to be able to
understand what the desired requirements of the new python policy are
well enough to comply with them -- I prefer not using packaging tools
if I do not understand what they do and can't do it independently.
At this point, could you please point out the salient points
of divergence between python-central and python-support? From what I
am given to understand, these take public pure Python modules and
byte compile them for every avaialable version on the taarget
machine, and recompile as need
Re: dh_python and python policy analysis
On Tue, 1 Aug 2006 09:35:56 +0200, Loïc Minier <[EMAIL PROTECTED]> said: > On Mon, Jul 31, 2006, Manoj Srivastava wrote: >> 2.1. [5]XS-Python-Version: 2.2. [6]XB-Python-Version: > Your document keeps mentionning these, even as "requirements", but > XB- isn't required for packages using python-support, and XS can be > replaced by debian/pyversions. I guess I do not understand enough about python-support, in that case. I was basing it off section 2.3 of the draft policy[1], which has inclusions of XB-Python-Version as a should directive. Could you point me to documentation on python-support, what it does, how to use it, and how it differs from python-central? > Is your document targetted at inclusion in the dev-ref? Targeted for dev-ref, no. If python policy is ever going to be made into official Debian technical policy, then some kind of detailed specification > PS: I really don't see the point in cross-posting to debian-devel@, > please either post to one or the other, thanks. I think that this is of general enough interest to involve -devel; but I do not want to miss out Python folks who have given up on -devel, and this is of interest to Python packaging. So, in my opinion, cross posting for this is jusified as long as the discussion stays on topic. manoj [1] http://www.debian.org/doc/packaging-manuals/python-policy/ch-module_packages.html -- "Who cares if it doesn't do anything? It was made with our new Triple-Iso-Bifurcated-Krypton-Gate-MOS process ..." Manoj Srivastava <[EMAIL PROTECTED]> <http://www.debian.org/%7Esrivasta/> 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: dh_python and python policy analysis
Hi, Here is round two of my take on python policy. I have incorporate the correction offered by various people, and read the documents for python-central and python-support, and incorporated my understanding of those into this document. So, this is my take on the new python policy, based on the analysis of the new Python policy draft and dh_python, and is supposed to be a rough specification of what the python policy is supposed to be (based on current dh_python behaviour). The current analysis, and future updates, are to be found at http://www.golden-gryphon.com/software/manoj-policy/ This can still use a lot of polishing. I am including a text version below. manoj Packaging with the new Python policy A package developers view Manoj Srivastava Copyright (c) 2006 Manoj Srivastava Revision History Revision 1.0 25 Jul 2006 Obstacles to conformance with the new python policy. While the new Python policy, specifically the [1]"Packaged Modules" chapter, contains the elements that must be present in the debian/control filename, it is not very explicit about how the values are to be substituted. The Debian Wiki falls back on calling dh_python, which is not helpful in understanding the actual logic to be followed. This article is an attempt to correct this gap in documentation. -- Table of Contents 1. [2]Introduction 1.1. [3]Categorization of Python software 2. [4]Recipe for developers 2.1. [5]General Notes 2.1.1. [6]Python versions supported by the source 2.1.2. [7]Supported versions in the binary package 2.1.3. [8]Depends: 2.1.4. [9]Provides 2.1.5. [10]Build-Depends: 2.2. [11]Script 2.2.1. [12]Supported versions 2.2.2. [13]Depends: 2.3. [14]Private Pure Python Modules 2.3.1. [15]Byte compilation 2.3.2. [16]Supported versions 2.3.3. [17]Depends: 2.4. [18]Private Extension 2.4.1. [19]Supported versions 2.4.2. [20]Depends: 2.5. [21]Public pure Python Module 2.5.1. [22]Byte compiling 2.5.2. [23]Supported versions 2.5.3. [24]Depends: 2.5.4. [25]Provides: 2.6. [26]Public Extension 2.6.1. [27]Supported versions 2.6.2. [28]Depends: 2.6.3. [29]Provides 1. Introduction While trying to update SELinux packages, I ran across problems in trying to determine if my packages were complying with the new python policy: any practical tips for packaging generally devolved to the statement "Oh, just run dh_python". This is my attempt to offer more concrete tips for packaging, by reverse engineering dh_python for the specifications and tips. -- 1.1. Categorization of Python software Program/script This consists of software directly called by an end user of external program, and is independently interpreted by the Python interpreter. Usually starts with the magic bytes #!, with the interpreter being /usr/bin/python* or /usr/bin/env python*. Modules This is code included in python "programs/scripts", and not invoked directly (serving as library modules in compiled languages). Modules can be categorized under two orthogonal criteria: firstly, based on the whether or not they are implemented purely in python, like so: Pure Python Module These are python source code, to be interpreted by the Python interpreter just like program/script code is, and may work across many versions of Python. Extension Module Extensions are C code compiled and linked against a specific version of the libpython library, and so can only be used by one version of Python. Another way of categorizing modules is based on whether or not they are available for use by third party scripts/modules. Public Public modules are available for use in other Python scripts or modules using the import directive. They are installed in one of the directories /u
