Re: coming back to packaging multiple versions of libraries
On Jan 04, 2011, at 07:30 AM, Robert Collins wrote:
>It really does look like having better upstream facilities would make
>this a no-brainer for us; what I'd like to achieve though is something
>that /works/ for the existing python platform - for 2.7 which will be
>around a good long time, and then we'll have plenty of data to discuss
>with upstream.
Let's assume you can work out the distro-specific issues (e.g. how to name the
packages so that you can install wadllib1 and wadllib2 side-by side). What
should work in Python would be to use .pth files.
http://docs.python.org/library/site.html
For example, let's say you have the following directory layout:
.../wadllib1/
wadllib/
__init__.py
...
.../waddlib2/
wadllib/
__init__.py
...
You can now install a wadllib.pth file that has the directory the version you
want to use by default, e.g.
.../wadllib.pth contains:
wadllib2
If wadllib.pth lives in the system site directory (or use site directory, but
let's ignore that), then when you 'import wadllib' you'd get version 2.
If wadllib.pth does not live in the site directory, then you have to
explicitly:
import site
site.addsitedir('/path/to/pth/files/')
to get Python to read and process the pth. Also note that Debian/Ubuntu's
site.py has been modified from upstream, e.g. to add the dist-packages
directory.
If you wanted something more dynamic, you could exploit a trick in .pth
processing where any line that starts with 'import' gets executed. Zope
packages exploit this; for example, when you have python-zope.interface
installed, see /usr/share/pyshared/zope.interface-3.6.1-nspkg.pth
Totally bogus strawman:
.../wadllib.pth contains:
import os, sys; p=os.environ.get('WADLLIBDIR'); (p not in sys.path) and
sys.path.append(p)
You could now do:
WADLLIB=/path/to/version/you/want python -c 'import wadllib'
I'm sure you can come up with a much better mechanism to decide how to
dynamically locate the version you want. The important things to remember are
that only lines that start with 'import' are executed, and you have to
obviously have your selection criteria set before Python reads your .pth file,
which in the case of those in the system site directory is at Python start up
time (unless you use python -S).
I hope that helps. It's the best I could come up with that works with current
Python. You still have to figure out the Debian packaging issues.
-Barry
signature.asc
Description: PGP signature
Re: coming back to packaging multiple versions of libraries
IMHO installing two versions of the same (public) Python module should be simply forbidden in policy. For one simple reason: if module foo uses bar in version 1 and spam uses bar in version 2, imagine what will happen with egg which uses both foo and spam. Right now I see only these options: 1) create separate binary packages for both versions of bar conflicting with each other 2) install only one version of bar as public module and the second one as private module 3) convince library author(s) to *not* break API so often (6 months? 1 year?) 4) convince library author(s) to rename it (bar, bar2, bar3, etc.) once he/she decides to break API 5) convince Python interpreter developers to provide something similar to sonames (bar.py, bar.2/__init__.py, bar.2.3/__init__.py, bar.3.py) and add support for syntax I mentioned last time or use something like 3rd party pkg_resources' require() or... allow foo/lib-packages/bar → ../../bar-1 symlinks (where dist-packages/bar is in version 2 and dist-packages/bar-1 in v1, note that bar-1 is not a valid module name)... with lots of problems Clint already mentioned my current vote is: 3, 4, 2 (if only one application uses it), 1, none of the above, 5 -- Piotr Ożarowski Debian GNU/Linux Developer www.ozarowski.pl www.griffith.cc www.debian.org GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645 signature.asc Description: Digital signature
Re: coming back to packaging multiple versions of libraries
2011/1/6 Piotr Ożarowski : > IMHO installing two versions of the same (public) Python module should > be simply forbidden in policy. For one simple reason: if module foo uses > bar in version 1 and spam uses bar in version 2, imagine what will > happen with egg which uses both foo and spam. This issue exists with C libraries too, but its not forbidden. Why should C libraries be expected to permit this, but not Python libraries? > Right now I see only these options: > 1) create separate binary packages for both versions of bar conflicting > with each other > 2) install only one version of bar as public module and the second one as > private module Neither of these meet the use cases I've described comprehensively before. > 3) convince library author(s) to *not* break API so often (6 months? 1 > year?) Once is enough; the ecosystem is huge and unless we handle transitions gracefully we'll get into nightmare situations routinely - which we do on complex stacks (like zope) on a regular basis. > 4) convince library author(s) to rename it (bar, bar2, bar3, etc.) once > he/she decides to break API I'm not sure that this is necessary or worth it. Certainly you'd need a massive change in the culture of Python upstream (python *itself* to have this act as a systematic fix). > 5) convince Python interpreter developers to provide something similar to > sonames (bar.py, bar.2/__init__.py, bar.2.3/__init__.py, bar.3.py) > and add support for syntax I mentioned last time or use something > like 3rd party pkg_resources' require() or... allow > foo/lib-packages/bar → ../../bar-1 symlinks (where dist-packages/bar > is in version 2 and dist-packages/bar-1 in v1, note that bar-1 is not > a valid module name)... with lots of problems Clint already mentioned Its already developed : pkg_resources.Require() I'm working on the mechanics to get default programs working; once thats done things using the soname equivalent should be trivial. -Rob -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/aanlkti=j9z+ddf3vdz5lcu3f9__c9zzypff+3qrta...@mail.gmail.com
Re: coming back to packaging multiple versions of libraries
On Jan 05, 2011, at 11:40 PM, Piotr Ożarowski wrote: >IMHO installing two versions of the same (public) Python module should >be simply forbidden in policy. For one simple reason: if module foo uses >bar in version 1 and spam uses bar in version 2, imagine what will >happen with egg which uses both foo and spam. Hilarity ensues? No, that's not quite right. Insanity. But anyway, such choices must be done at the application level, not at the library level. You just cannot safely integrate two libraries that require different versions of dependent libraries[*]. >Right now I see only these options: >1) create separate binary packages for both versions of bar conflicting > with each other >2) install only one version of bar as public module and the second one as > private module >3) convince library author(s) to *not* break API so often (6 months? 1 > year?) >4) convince library author(s) to rename it (bar, bar2, bar3, etc.) once > he/she decides to break API >5) convince Python interpreter developers to provide something similar to > sonames (bar.py, bar.2/__init__.py, bar.2.3/__init__.py, bar.3.py) > and add support for syntax I mentioned last time or use something > like 3rd party pkg_resources' require() or... allow > foo/lib-packages/bar → ../../bar-1 symlinks (where dist-packages/bar > is in version 2 and dist-packages/bar-1 in v1, note that bar-1 is not > a valid module name)... with lots of problems Clint already mentioned > >my current vote is: 3, 4, 2 (if only one application uses it), 1, none >of the above, 5 These are not necessarily mutually exclusive. ;) #3 and #4 are both worth pursuing in any case, but kind of outside the domain of either upstream (except were the stdlib is concerned) or debian-python. Still, as a Python programmer, if you find gratuitous API breaks in packages you care about, it's worth a bug report. (I'm still not quite sure why Robert doesn't just get the wadllib developers to fix their API breakage?) I really hate having the version numbers in the module names, and actually in package names too, but the latter tend to disappear after transitional periods. Names like urllib2 live forever unfortunately. I don't have any great ideas about how to improve that though. I agree with Piotr, I think trying to do this in a hidden way is fraught with problems and lots of unanswered questions. I will mention that for years Mailman shipped with its own version of the email package because it needed an exact API that might have been different than the default one, depending on the version of Python you were using. So a specific application that cares deeply about its library versions I think has no better choice than to do something like that, which is essentially your #2. There can be more principled ways of doing that, e.g. with buildout or virtualenv, but it boils down to the same thing. There's just no good distro-way of doing that though. -Barry [*] In a previous life I did a lot of development on Windows and it was possible to combine two different C run times in a Python embedded app. You were lucky if all your app did was crash when you passed FILE*'s from one to the other. Another anecdote: JWZ famously hated shared libraries for a similar reason and always recommended static linking. Same problem, different domain. signature.asc Description: PGP signature
Re: coming back to packaging multiple versions of libraries
On Thu, Jan 6, 2011 at 12:39 PM, Barry Warsaw wrote: > These are not necessarily mutually exclusive. ;) #3 and #4 are both worth > pursuing in any case, but kind of outside the domain of either upstream > (except were the stdlib is concerned) or debian-python. Still, as a Python > programmer, if you find gratuitous API breaks in packages you care about, it's > worth a bug report. > > (I'm still not quite sure why Robert doesn't just get the wadllib developers > to fix their API breakage?) wadllib was an example. I'm aware of -very- few libraries that never change things; what constitutes an API break is vaguely defined; the functional definition we need here is 'when consuming code is broken by upgrading the dependency' : that definition is much wider than 'something we advertised is no longer supported' which many upstream follow. > I really hate having the version numbers in the module names, and actually in > package names too, but the latter tend to disappear after transitional > periods. Names like urllib2 live forever unfortunately. I don't have any > great ideas about how to improve that though. having the language support N versions of urllib installed at once seems to work very well for C, C++, C#, Java and other languages. Perhaps Python should just do that. > I agree with Piotr, I think trying to do this in a hidden way is fraught with > problems and lots of unanswered questions. I will mention that for years > Mailman shipped with its own version of the email package because it needed an > exact API that might have been different than the default one, depending on > the version of Python you were using. So a specific application that cares > deeply about its library versions I think has no better choice than to do > something like that, which is essentially your #2. There can be more > principled ways of doing that, e.g. with buildout or virtualenv, but it boils > down to the same thing. There's just no good distro-way of doing that > though. I'm not trying to do this in a hidden way though? Why do you think that that is the case? -Rob -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/aanlktikbsuaogyztzmtmfnzenccxuxhy3bokdn4wm...@mail.gmail.com
Re: coming back to packaging multiple versions of libraries
On Jan 06, 2011, at 12:45 PM, Robert Collins wrote: >I'm not trying to do this in a hidden way though? Why do you think >that that is the case? Sorry, I meant "automatic". -Barry signature.asc Description: PGP signature
RFS: marave
Dear mentors, I am looking for a sponsor for my package "marave". * Package name: marave Version : 0.7-1 * URL : http://code.google.com/p/marave/ * License : gpl2 Section : editors It builds these binary packages: marave - Full screen editor written on Python The package appears to be lintian clean. The upload would fix these bugs: 592572 My motivation for maintaining this package is: I am trying to get inside linux helping with some packages. The package can be found on mentors.debian.net: - URL: http://mentors.debian.net/debian/pool/main/m/marave - Source repository: deb-src http://mentors.debian.net/debian unstable main contrib non-free - dget http://mentors.debian.net/debian/pool/main/m/marave/marave_0.7-1.dsc I would be glad if someone uploaded this package for me. Kind regards fulapol PD: please CC me, im not in the list. Thanks signature.asc Description: This is a digitally signed message part
Re: coming back to packaging multiple versions of libraries
On Thu, Jan 6, 2011 at 1:14 PM, Barry Warsaw wrote: > On Jan 06, 2011, at 12:45 PM, Robert Collins wrote: > >>I'm not trying to do this in a hidden way though? Why do you think >>that that is the case? > > Sorry, I meant "automatic". I'm not proposing anything magical: maintainer intent will always be present (just as it is for C libraries). We can look at building out heuristics for setting minimum dependency deps automatically, but thats orthogonal to basic mechanism which is what I'm focused on for now. -Rob -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/aanlktimnootrarpcxfwf6x=dr94jer3-rlicektmh...@mail.gmail.com
