Re: version independent pythin packages: ?
On Thu, Aug 07, 2003 at 12:58:25PM +1000, Donovan Baarda wrote: > The only problem is when someone with write access to > /usr/lib/site-python uses a non-default python... the pyc's will be > "updated" for the non-default python. > > After testing, it seems that there is no way to prevent root from > updating the pyc's when using a non-default python. However, even if Yes, even chmoding the .pyc file won't work. > this does occur, everything will still work, it will just be slightly > less optimal when using the default python. The only solution is "root > should never use the non-default python when importing modules from > /usr/lib/site-python"... which may or may not be OK. For most programs this should not really be a big issue. Most of the time, people tend to use mostly one python version on production machines. The worst case I've seen with missing/inapropriate .pyc files was on a cgi based web application, where the performance increase we got when fixing the problem was noticeable by end users. For long running processes, the import is done only once, so the loss of time is evened out in the long run. > Given the complexity of the alternatives, this is a simple solution that > fixes the problem with only minor issues. I'd be tempted to make this > the solution and put it into the Python policy. +1 > Anyone else agree? > > Other things to think about; > > Python applications using the default Python with their own modules not > in /usr/lib/site-python... not an issue? They are expected to configure their own PYTHONPATH, are they not? > Making sure all the pyc's are re-compiled when the default python is > updated. Yes. Should not be too difficult. > A brute force approach is to have the python packages post-inst run > "dpkg-reconfigure" over every package that depends on "python", and > require that packages recompile their pyc files on a "dpkg-reconfigure". > This has the advantage of "notifying" all these packages when the > default python has changed so they can do other stuff if they need to. This would be nice, but packages will take some time to catch up. Another thing we should take care of is packages which used to be pure python in version N, and include C extensions in version N+1. The packages for N+1 should be versioned, but how can we provide a nice upgrade path? In order to be sure not to break anything, we would have to make the new python-foobar package depend on python2.X-foobar for X in (1,2,3). The last point is how to write dependencies on packages which are ot available on all python versions. For instance python-docutils needs python-xmlbase and python-difflib. python-xmlbase exists for python2.1 and 2.2 but not 2.3, and difflib exists for 2.1, but not 2.2 or 2.3. How should the dependencies be written ? -- Alexandre Fayolle LOGILAB, Paris (France). http://www.logilab.com http://www.logilab.fr http://www.logilab.org Développement logiciel avancé - Intelligence Artificielle - Formations
Re: version independent pythin packages: ?
On Thu, Aug 07, 2003 at 12:58:25PM +1000, Donovan Baarda wrote:
> Anyone else agree?
I see no problem, aside from performance for those users not using the
default version, but since this is only a "load time" problem, it
shouldn't be more than a little annoyance. I suppose that really picky
users could add some local repository to their PYTHONPATH and copy there
the load-time-critical modules if really in a need, or even ask the
sysadmin to set a version-specific optimized repository.
We can't do better with current Python behaviour, and I think most pure
Python packages would benefit from this change to Policy.
Thinking on this problem a bit further (not feasible with current
implementation), wouldn't it be nice if the user could enable Python
(via environment or command line switch) to use some local repository
(~/.python/X.Y/) to store .py{c,o} files if (s)he hasn't write access to
the system files?
Re: version independent pythin packages: ?
Hi, Alexandre Fayolle wrote:
> The last point is how to write dependencies on packages which are ot
> available on all python versions. For instance python-docutils needs
> python-xmlbase and python-difflib. python-xmlbase exists for python2.1
> and 2.2 but not 2.3, and difflib exists for 2.1, but not 2.2 or 2.3. How
> should the dependencies be written ?
The same way we currently handle non-pure Python packages -- except that
the nonversioned package contains the actual Python code and the versioned
packages are empty, instead of the other way round.
--
Matthias Urlichs | {M:U} IT Design @ m-u-it.de | [EMAIL PROTECTED]
Disclaimer: The quote was selected randomly. Really. | http://smurf.noris.de
--
taniwha: Quote material :)
Endy: :)
Endy: I already snipped it
/usr/include/python
Hi, why isn't there a default /usr/include/python (possibly accomplished as a symlink in the package python-dev, like /usr/bin/python in the package python)? (I'm sure there is a reason for that, I just didn't find it documented somewhere.) Thanks! bye, Roland signature.asc Description: This is a digitally signed message part
Re: /usr/include/python
Roland Stigge writes: > Hi, > > why isn't there a default /usr/include/python (possibly accomplished as > a symlink in the package python-dev, like /usr/bin/python in the package > python)? (I'm sure there is a reason for that, I just didn't find it > documented somewhere.) it's not needed. distutils selects the correct include path for you.
Re: version independent pythin packages: ?
On Thu, 2003-08-07 at 18:44, Alexandre Fayolle wrote: > On Thu, Aug 07, 2003 at 12:58:25PM +1000, Donovan Baarda wrote: [...] > > Python applications using the default Python with their own modules not > > in /usr/lib/site-python... not an issue? Actually... I think I prefer /usr/lib/python/site-packages rather than /usr/lib/site-python because that way the directory structure mirrors that for the /usr/lib/pythonX.Y "specific version" tree. > They are expected to configure their own PYTHONPATH, are they not? Yeah, the issues with the current policy and this is that there is no way to re-compile these modules when the default python changes. > > Making sure all the pyc's are re-compiled when the default python is > > updated. > > Yes. Should not be too difficult. There are tricks here that make it a little harder than you would expect. Because deb's don't support "triggers", the python package will have to find all the *.py's that need compiling itself. There are two ways I can see to handle this; 1) have all packages put .py's in a suitable place (/usr/lib/python) so they can be re-compiled. But this breaks Python programs that have their own modules in some other place. 2) Have the python package identify all the packages with py's that need to be recompiled, then "tell them" to recompile themselves by executing something that triggers the package's recompile scripts (dpkg-reconfigure or the .post-inst, or something). Identifying the .py's that need recompiling is actually pretty tricky too.. you need to identify all packages that depend on "python" with "*.py" files. > > A brute force approach is to have the python packages post-inst run > > "dpkg-reconfigure" over every package that depends on "python", and > > require that packages recompile their pyc files on a "dpkg-reconfigure". > > This has the advantage of "notifying" all these packages when the > > default python has changed so they can do other stuff if they need to. > > This would be nice, but packages will take some time to catch up. I think a dpkg-reconfigure re-runs the post-inst scripts which should already compile stuff. We could call the post-inst scripts directly, but dpkg-reconfigure might do other stuff that is more package friendly. I don't think any packages apart from "python" will need changing to support this (hence why I suggested it, despite it being a bit of overkill) > Another thing we should take care of is packages which used to be pure > python in version N, and include C extensions in version N+1. The > packages for N+1 should be versioned, but how can we provide a nice > upgrade path? In order to be sure not to break anything, we would have > to make the new python-foobar package depend on python2.X-foobar for X > in (1,2,3). Example use case; Up to and including python (2.1) we had; python-foo (1.1) with files in /usr/lib/python Depends: python Now we have python (2.2) and a new upstream release of "foo" python2.2-foo (2.1) has a C exension in /usr/lib/python2.2 Depends: python2.2 We need to make the following packages; python2.1-foo (1.1) with files in /usr/lib/python Depends: python1.5 | python1.6 | python2.0 | python2.1 Provides: python1.5-foo, python1.6-foo, python2.0-foo Replaces: python-foo (<=1.1) python-foo (2.1) as a wrapper for python2.2-foo Depends: python (>=2.2), python (<<2.3), python2.2-foo the "almost version idependent" python2.1-foo provides support for python1.5, python1.6, python2.0, and python2.1. The python2.2-foo provides support for python2.2. The new python-foo wrapper provides support for the default python (2.2). Because the python path for python2.2 lists /usr/lib/python2.2 before /usr/lib/python, python2.2 will grab the correct C extension version, but other versions of python will use the old pure python packages in /usr/lib/python. Hmmm... this has implications that affect what version the .pyc's for python2.1-foo in /usr/lib/python should be compiled for. They should _not_ be compiled against the default python, so we can't just compile-all in /usr/lib/python when the default python package changes. It seems each individual package should be responsible for compiling it's own *.py's with an appropriate version of python, even in /usr/lib/python. We can't have the "python" package handle it directly. Here's a suggestion; Each package that depends on any version of python or pythonX.Y must compile it's own *.py's in its postinst and remove its own *.pyc's in its postrm using a suitable version of python. Packages that support the default python should compile using /usr/bin/python (provided the version is OK). Packages that support multiple pythonX.Y's should compile using that highest installed version of pythonX.Y that they support. The "python" package's postinst and postrm scripts should find all the installed packages that depend on "python" and get them to re-run their postinst scripts, either by calling the postinst scripts directly
Re: python 2.2 -> python 2.3 transition
i agree, we have a great support for Python. thanks to those who make it possible. cavok On Thu, Aug 07, 2003 at 11:47:48AM +1000, Donovan Baarda wrote: ... > > Personally I was going to post "nice job everyone... the Python Policy > looks like it is working". There are still a few niggly things, but if > Debian can go to Python 2.3 within days of it being released without > breaking anything else, I'd say thats pretty damn impressive. > ... -[ Domenico Andreoli, aka cavok --[ http://filibusta.crema.unimi.it/~cavok/gpgkey.asc ---[ 3A0F 2F80 F79C 678A 8936 4FEE 0677 9033 A20E BC50
