The new python policy and packaging

[Manoj Srivastava]

Hi,

[Please keep the CC to debian-devel, or please CC me, if yu
are replying to this on debian-python@lists.debian.org, since
I am not subscribed to that list]

I have been trying to ensure that my SELinux related packages
 which have a python component are compliant with the new Python
 policy[1].  The wiki page for the policy[2] seems to be mostly a dh_python
 usage manual, and defers to dh_python to set up the variable
 substitutions correctly, so I have been trying to write up my own
 manual based on the policy and reverse engineering dh_python.

In the course of this, there were points which I found very
 confusing in the Python policy which I think need to be cleared up.

 Section 2.2:
Firstly, public modules are supposed to be packaged in a
 package called python-foo. What about public extensions? section 2.2
 talks about naming, and also talks about supporting current python
 version. The last sentence in section 2.2 says:
 "This requirement also applies to extension modules; binaries
  for all the supported Python versions should be included in
  a single package."
 It is not clear whether "This requirement" refers to the naming, or
 the support.

Perhaps policy should explicitly say "Public modules _and
 extensions_ should be packaged with a name of python-foo,"  (assuming
 that I now have the correct interpretation).

(BTW, the wiki page seems to differentiate more strongly
 between modules and extensions:
 python extension: a .so file for use by python applications
 python module: a .py file for use by python applications)

section 2.4:
Can anyone explain why if my python packages depend on Python
 version 2.4, and selinux.so for python 2.4, it should depend on
 python2.4 and python2.4-selinux; but when the default changes to 2,4,
 my package dependencies would beed to change to python (>= 2.4) and
 pythong-selinux? (This according to section 2.4 of [1])

It is not clear why a package that depends strictly on python
 2.3, which is the current default, should fall into paragraph one of
 the section, and not into paragraph two.

,[ section 2.4, paragraph 1 ]
|  Packaged modules available for the default Python version (or many
|  versions including the default) as described in Module Package Names,
|  Section 2.2 must depend on "python (>= X.Y)". If they require other
|  modules to work, they must depend on the corresponding
|  python-foo. They must not depend on any pythonX.Y-foo. 
`

,[ section 2.4, paragraph 2 ]
|  Packaged modules available for one particular version of Python must
|  depend on the corresponding pythonX.Y package instead. If they need
|  other modules, they must depend on the corresponding pythonX.Y-foo
|  packages, and must not depend on any python-foo 
`

I assumed it fell into paragraph one, since it is explicitly
 stated so in policy. Logically, though, only paragraph two should
 apply.  In any case, this must be disambiguated.

section 2.5:

The second paragraph of section 2,5 is a duplicate of the
 second paragraph of section 2,4.  This seems like a cut-and-paste
 error, and directives appear to be missing from section 2.5. There
 should be something about providing pythonX.Y-foo if your package
 provides an extension for version X.Y of python.

I'll add to this list as I get further along in my reading. 

manoj

[1]http://www.debian.org/doc/packaging-manuals/python-policy/
[2]http://wiki.debian.org/DebianPython/NewPolicy

ps: I think policy should state clearly how it expects the various
fields in the control file to be initialized, rather than leaving it
mostly unstated and deferring to an external helper program, but that
is another thread
-- 
Without fools there would be no wisdom.
Manoj Srivastava   <[EMAIL PROTECTED]>  
1024D/BF24424C print 4966 F272 D093 B493 410B  924B 21BA DABB BF24 424C


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: The new python policy and packaging

[Joe Wreschnig]

On Thu, 2006-07-27 at 11:56 -0500, Manoj Srivastava wrote:
> Hi,
> 
> [Please keep the CC to debian-devel, or please CC me, if yu
> are replying to this on debian-python@lists.debian.org, since
> I am not subscribed to that list]
> 
> I have been trying to ensure that my SELinux related packages
>  which have a python component are compliant with the new Python
>  policy[1].  The wiki page for the policy[2] seems to be mostly a dh_python
>  usage manual, and defers to dh_python to set up the variable
>  substitutions correctly, so I have been trying to write up my own
>  manual based on the policy and reverse engineering dh_python.
> 
> In the course of this, there were points which I found very
>  confusing in the Python policy which I think need to be cleared up.
> 
>  Section 2.2:
> Firstly, public modules are supposed to be packaged in a
>  package called python-foo. What about public extensions? section 2.2
>  talks about naming, and also talks about supporting current python
>  version. The last sentence in section 2.2 says:
>  "This requirement also applies to extension modules; binaries
>   for all the supported Python versions should be included in
>   a single package."
>  It is not clear whether "This requirement" refers to the naming, or
>  the support.
> 
> Perhaps policy should explicitly say "Public modules _and
>  extensions_ should be packaged with a name of python-foo,"  (assuming
>  that I now have the correct interpretation).
> 
> (BTW, the wiki page seems to differentiate more strongly
>  between modules and extensions:
>  python extension: a .so file for use by python applications
>  python module: a .py file for use by python applications)

Your confusion is due to an incorrect distinction. .py and .so files are
both modules; .so files are "extension modules" and .py files are "pure
Python modules". When the word "module" is used alone it always refers
to both.

This is made clear in the policy;
http://www.debian.org/doc/packaging-manuals/python-policy/ch-module_packages.html#s2.1

If the wiki uses "modules" to refer to only pure Python modules, it is
incorrect.

> section 2.4:
> Can anyone explain why if my python packages depend on Python
>  version 2.4, and selinux.so for python 2.4, it should depend on
>  python2.4 and python2.4-selinux; but when the default changes to 2,4,
>  my package dependencies would beed to change to python (>= 2.4) and
>  pythong-selinux?

Because then applications using the default python (/usr/bin/python) can
access it, and that will be true for versions beyond 2.4. The versioned
names are not appropriate for packages using the default version.

If you always need Python 2.4, even when Python 2.5 is the default, then
the python2.4-foo deps are still appropriate.

> It is not clear why a package that depends strictly on python
>  2.3, which is the current default, should fall into paragraph one of
>  the section, and not into paragraph two.
> 
> ,[ section 2.4, paragraph 1 ]
> |  Packaged modules available for the default Python version (or many
> |  versions including the default) as described in Module Package Names,
> |  Section 2.2 must depend on "python (>= X.Y)". If they require other
> |  modules to work, they must depend on the corresponding
> |  python-foo. They must not depend on any pythonX.Y-foo. 
> `
> 
> ,[ section 2.4, paragraph 2 ]
> |  Packaged modules available for one particular version of Python must
> |  depend on the corresponding pythonX.Y package instead. If they need
> |  other modules, they must depend on the corresponding pythonX.Y-foo
> |  packages, and must not depend on any python-foo 
> `
> 
> I assumed it fell into paragraph one, since it is explicitly
>  stated so in policy. Logically, though, only paragraph two should
>  apply.  In any case, this must be disambiguated.

I would not consider modules for only 2.3 to be available "for the
default Python version", unless a simple rebuild would make them for
only 2.4 after 2.4 becomes the default (XS-Python-Version 2.3 versus
XS-Python-Version current). And if your module supports more than one
version, you should be packaging it for more than one version, meaning
it unambiguously falls under 2.4.1.

The language could perhaps be clearer, but you're the first person to
have problems with it (and you figured it out as well), so I think it's
adequate.

> section 2.5:
> 
> The second paragraph of section 2,5 is a duplicate of the
>  second paragraph of section 2,4.  This seems like a cut-and-paste
>  error, and directives appear to be missing from section 2.5. There
>  should be something about providing pythonX.Y-foo if your package
>  provides an extension for version X.Y of python.

Yes, this is a real mistake, probably my fault. The second paragraph can
probably just be deleted.

The Provides: has nothing to do with extensions. It's solely t

Re: The new python policy and packaging

[Pierre Habouzit]

Le jeu 27 juillet 2006 23:20, Manoj Srivastava a écrit :
> On Thu, 27 Jul 2006 14:15:47 -0500, Joe Wreschnig 
<[EMAIL PROTECTED]> said:

> > Your confusion is due to an incorrect distinction. .py and .so
> > files are both modules; .so files are "extension modules" and .py
> > files are "pure Python modules". When the word "module" is used
> > alone it always refers to both.
>
> Then please explain section 2.2:
>  "Public modules should be packaged with a name of python-foo, where
>   foo is the name of the module.  This requirement also applies
>   to extension modules; binaries for all the supported Python
> versions should be included in a single package. "
>
> The wording implies that Public modules are distinct from
>  extension modules (since something stated for public modules
>  also applies to extension modules. Policy otherwise states public vs
>  private and pure modules vs extension modules are orthogonal, so
>  there is an inconsistency here.

public modules are modules that live 
in /usr/{lib,share}/python$VERSION/site-packages/ and that anybody 
can 'import' in a python program

whereas python private modules/extensions are modules that are used 
internally by a program, but that it does not intend to share with 
others. it's completely orthogonal to beeing pure python or a C 
extension.

wording in the python folks is often module (like in pure python module) 
vs. extension (like C extension). I suppose it's sometimes hard to 
write a policy with a wording a bit different of what is often used.

*I* personnaly prefer module/extension terminology, as it's nearer to 
what it is. a module is implicitely written in the language, whereas an 
extension cannot, hence is written as an C library. but this is 
nitpicking.

> > Because then applications using the default python
> > (/usr/bin/python) can access it, and that will be true for versions
> > beyond 2.4. The versioned names are not appropriate for packages
> > using the default version.
>
> Why do my packages need to change the dependencies, then?
>  Because when the default changes to 2.5, the dependency would have
>  to change yet again, for no changes in the package itself.

yes but the package would only need a rebuild (aka binNMU) which is the 
whole point of the policy: make the python transition be a big run of 
binNMU for arch:any packages (as per definition, arch:all packages are 
not binNMUable)

and if you use dh_* thingies, it achieves that. I suppose it's also 
doable without them though.

> Oh, I see. Say, there is an extension that implements a very
>  standard API for framework bar, which is packaged in package
>  bar. If this extension modules is used by a non-version-specific
>  script, the script can just import module foo, and not care that it
>  lives in a version specific dir.
>
> Expanding on that thought, suppose I have python-foo version
>  1.0 that has been compiled with python 2.3. If the script uses
>  /usr/bin/python2.4, it won't be able to import my module --- unless
>  it asks for and gets a python2.4-foo. So, if the script uses a non
>  default version of python, or the extension module was compiled for
> a non-default version of python, I must provide pythonX.Y-fooin case
> someone somewhere has a script calling a specific version of python.
>
> There is no actual harm in always providing pythonX.Y-foo for
>  all extentsion module packages (for all x.y supported and shipped
>  with the package).

that's it.


-- 
·O·  Pierre Habouzit
··O[EMAIL PROTECTED]
OOOhttp://www.madism.org


pgprjP1Gwjxl5.pgp
Description: PGP signature

Re: The new python policy and packaging

[Manoj Srivastava]

On Thu, 27 Jul 2006 14:15:47 -0500, Joe Wreschnig <[EMAIL PROTECTED]> said: 

> On Thu, 2006-07-27 at 11:56 -0500, Manoj Srivastava wrote:
>> Hi,
>> 
>> [Please keep the CC to debian-devel, or please CC me, if yu are
>> replying to this on debian-python@lists.debian.org, since I am not
>> subscribed to that list]
>> 
>> I have been trying to ensure that my SELinux related packages which
>> have a python component are compliant with the new Python
>> policy[1].  The wiki page for the policy[2] seems to be mostly a
>> dh_python usage manual, and defers to dh_python to set up the
>> variable substitutions correctly, so I have been trying to write up
>> my own manual based on the policy and reverse engineering
>> dh_python.
>> 
>> In the course of this, there were points which I found very
>> confusing in the Python policy which I think need to be cleared up.
>> 
>> Section 2.2: Firstly, public modules are supposed to be packaged in
>> a package called python-foo. What about public extensions? section
>> 2.2 talks about naming, and also talks about supporting current
>> python version. The last sentence in section 2.2 says: "This
>> requirement also applies to extension modules; binaries for all the
>> supported Python versions should be included in a single package."
>> It is not clear whether "This requirement" refers to the naming, or
>> the support.
>> 
>> Perhaps policy should explicitly say "Public modules _and
>> extensions_ should be packaged with a name of python-foo,"
>> (assuming that I now have the correct interpretation).
>> 
>> (BTW, the wiki page seems to differentiate more strongly between
>> modules and extensions: python extension: a .so file for use by
>> python applications python module: a .py file for use by python
>> applications)

> Your confusion is due to an incorrect distinction. .py and .so files
> are both modules; .so files are "extension modules" and .py files
> are "pure Python modules". When the word "module" is used alone it
> always refers to both.

Then please explain section 2.2:
 "Public modules should be packaged with a name of python-foo, where
  foo is the name of the module.  This requirement also applies
  to extension modules; binaries for all the supported Python versions
  should be included in a single package. "

The wording implies that Public modules are distinct from
 extension modules (since something stated for public modules
 also applies to extension modules. Policy otherwise states public vs
 private and pure modules vs extension modules are orthogonal, so
 there is an inconsistency here.


> This is made clear in the policy;
> http://www.debian.org/doc/packaging-manuals/python-policy/ch-module_packages.html#s2.1

Inconsistently so, as noted above.

> If the wiki uses "modules" to refer to only pure Python modules, it
> is incorrect.

Then I suggest someone who knows about Python policy please
 update the Wiki page to correct it.

>> section 2.4:

>> Can anyone explain why if my python packages depend on Python
>> version 2.4, and selinux.so for python 2.4, it should depend on
>> python2.4 and python2.4-selinux; but when the default changes to
>> 2,4, my package dependencies would beed to change to python (>=
>> 2.4) and pythong-selinux?

> Because then applications using the default python (/usr/bin/python)
> can access it, and that will be true for versions beyond 2.4. The
> versioned names are not appropriate for packages using the default
> version.

Why do my packages need to change the dependencies, then?
 Because when the default changes to 2.5, the dependency would have
 to change yet again, for no changes in the package itself.

> If you always need Python 2.4, even when Python 2.5 is the default,
> then the python2.4-foo deps are still appropriate.

Right. But that is an answer to a different question.

>> It is not clear why a package that depends strictly on python 2.3,
>> which is the current default, should fall into paragraph one of the
>> section, and not into paragraph two.
>> 
>> ,[ section 2.4, paragraph 1 ] | Packaged modules available for
>> the default Python version (or many | versions including the
>> default) as described in Module Package Names, | Section 2.2 must
>> depend on "python (>= X.Y)". If they require other | modules to
>> work, they must depend on the corresponding | python-foo. They must
>> not depend on any pythonX.Y-foo.  `
>> 
>> ,[ section 2.4, paragraph 2 ] | Packaged modules available for
>> one particular version of Python must | depend on the corresponding
>> pythonX.Y package instead. If they need | other modules, they must
>> depend on the corresponding pythonX.Y-foo | packages, and must not
>> depend on any python-foo `
>> 
>> I assumed it fell into paragraph one, since it is explicitly stated
>> so in policy. Logically, though, only paragraph two should apply.
>> In any case, this must be disambiguated.

> I would not consider modules for only