[Python-Dev] Structural cleanups to the main CPython repo
I have a feature branch where I'm intermittently working on the bootstrapping changes described in PEP 432. As part of those changes, I've cleaned up a few aspects of the repo layout: * moved the main executable source file from Modules to a separate Apps directory * moved the _freezeimportlib and _testembed source files from Modules to a separate Tools directory * split the monster pythonrun.h/c pair into 3 separate header/impl pairs: * bootstrap.h/bootstrap.c * shutdown.h/shutdown.c * pythonrun.h/pythonrun.c These structural changes generally mean automatic merges touching the build machinery or the startup or shutdown code fail fairly spectacularly and need a lot of TLC to complete them without losing any changes from the main repo. Would anyone object if I went ahead and posted patches for making these changes to the main repo? I found they made the code *much* easier to follow when I started to turn the ideas in PEP 432 into working software, and implementing these shifts should make future merges to my feature branch simpler, as well as producing significantly cleaner diffs when PEP 432 gets closer to completion. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Structural cleanups to the main CPython repo
On May 28, 2013, at 10:15 PM, Nick Coghlan wrote: >Would anyone object if I went ahead and posted patches for making >these changes to the main repo? When you say "post[ed] patches", do you mean you want to put them some place for us to review? If so, sure, go ahead of course. -Barry ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Structural cleanups to the main CPython repo
Le Tue, 28 May 2013 22:15:25 +1000, Nick Coghlan a écrit : > I have a feature branch where I'm intermittently working on the > bootstrapping changes described in PEP 432. > > As part of those changes, I've cleaned up a few aspects of the repo > layout: > > * moved the main executable source file from Modules to a separate > Apps directory Sounds fine (I don't like "Apps" much, but hey :-)). > * moved the _freezeimportlib and _testembed source files from Modules > to a separate Tools directory Well, they should probably go to Apps too, no? > * split the monster pythonrun.h/c pair into 3 separate header/impl > pairs: >* bootstrap.h/bootstrap.c >* shutdown.h/shutdown.c >* pythonrun.h/pythonrun.c I don't think separating bootstrap from shutdown is a good idea. They are quite closely related since one undoes what the other did (and they may also use shared private functions or data). I don't know what goes in the remaining "pythonrun.c", could you detail a bit? Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Structural cleanups to the main CPython repo
On Tue, May 28, 2013 at 10:31 PM, Antoine Pitrou wrote: > Le Tue, 28 May 2013 22:15:25 +1000, > Nick Coghlan a écrit : >> I have a feature branch where I'm intermittently working on the >> bootstrapping changes described in PEP 432. >> >> As part of those changes, I've cleaned up a few aspects of the repo >> layout: >> >> * moved the main executable source file from Modules to a separate >> Apps directory > > Sounds fine (I don't like "Apps" much, but hey :-)). Unfortunately, I don't know any other short word for "things with main functions that we ship to end users" :) >> * moved the _freezeimportlib and _testembed source files from Modules >> to a separate Tools directory > > Well, they should probably go to Apps too, no? I wanted to split out "part of the build/test infrastructure" from "shipped to end users", but I could also live with a simple "Bin" directory that contained both kinds of executable. >> * split the monster pythonrun.h/c pair into 3 separate header/impl >> pairs: >>* bootstrap.h/bootstrap.c >>* shutdown.h/shutdown.c >>* pythonrun.h/pythonrun.c > > I don't think separating bootstrap from shutdown is a good idea. They > are quite closely related since one undoes what the other did (and they > may also use shared private functions or data). It was deliberate - a big part of PEP 432 is making sure that all the interpreter state lives *in* the interpreter state (as part of the config struct). Splitting the two into separate compilation modules makes it possible to ensure that all communication goes via the interpreter configuration (statics in other modules are still a problem, but also mostly out of scope for PEP 432). I *really* want to get us to clean phase separation of "the interpreter is starting up", "the interpreter is running normally" and "the interpreter is shutting down". I found that to be incredibly difficult to do when they were all intermixed in one file, which is why I decided to enlist the compiler's help by separating them. > I don't know what goes > in the remaining "pythonrun.c", could you detail a bit? While they have some of the PEP 432 changes in them, the header files in the branch give the general flavour of the separation: Bootstrap is mostly get/init type functions: https://bitbucket.org/ncoghlan/cpython_sandbox/src/ae7fef62b462fb6b559172bd4dbefc185ec28c40/Include/bootstrap.h?at=pep432_modular_bootstrap Pythonrun is mostly PyRun_*, PyParser_*, Py_Compile* and a few other odds and ends: https://bitbucket.org/ncoghlan/cpython_sandbox/src/ae7fef62b462fb6b559172bd4dbefc185ec28c40/Include/pythonrun.h?at=pep432_modular_bootstrap Shutdown covers the various finalisers, atexit handling, etc: https://bitbucket.org/ncoghlan/cpython_sandbox/src/ae7fef62b462fb6b559172bd4dbefc185ec28c40/Include/shutdown.h?at=pep432_modular_bootstrap Cheers, Nick. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 409 and the stdlib
20.05.13 18:46, Antoine Pitrou написав(ла): I think it is a legitimate case where to silence the original exception. However, the binascii.Error would be more informative if it said *which* non-base32 digit was encountered. Please open a new issue for this request (note that no other binascii or base64 functions provide such information). ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Structural cleanups to the main CPython repo
As part of those changes, I've cleaned up a few aspects of the repo layout: * moved the main executable source file from Modules to a separate Apps directory Do you mean things that go into the shared library (libpythonXX/pythonXX.dll) vs executables? ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Structural cleanups to the main CPython repo
On Tue, May 28, 2013 at 9:07 AM, Nick Coghlan wrote: > Unfortunately, I don't know any other short word for "things with main > functions that we ship to end users" :) We used to call such things "programs", but that term may no longer be in popular parlance. :-) Or is it just too long? -Fred -- Fred L. Drake, Jr. "A storm broke loose in my mind." --Albert Einstein ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Structural cleanups to the main CPython repo
2013/5/28 Nick Coghlan : > On Tue, May 28, 2013 at 10:31 PM, Antoine Pitrou wrote: >> Le Tue, 28 May 2013 22:15:25 +1000, >> Nick Coghlan a écrit : >>> I have a feature branch where I'm intermittently working on the >>> bootstrapping changes described in PEP 432. >>> >>> As part of those changes, I've cleaned up a few aspects of the repo >>> layout: >>> >>> * moved the main executable source file from Modules to a separate >>> Apps directory >> >> Sounds fine (I don't like "Apps" much, but hey :-)). > > Unfortunately, I don't know any other short word for "things with main > functions that we ship to end users" :) "Bin" is quite common (if ironic). I think it would be fine two if that stuff was in Python/; anywhere is better than modules. (Care to move the GC, too?) -- Regards, Benjamin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Structural cleanups to the main CPython repo
28.05.13 16:07, Nick Coghlan написав(ла): On Tue, May 28, 2013 at 10:31 PM, Antoine Pitrou wrote: Le Tue, 28 May 2013 22:15:25 +1000, Nick Coghlan a écrit : * moved the main executable source file from Modules to a separate Apps directory Sounds fine (I don't like "Apps" much, but hey :-)). Unfortunately, I don't know any other short word for "things with main functions that we ship to end users" :) main ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Structural cleanups to the main CPython repo
On Wed, May 29, 2013 at 12:03 AM, Serhiy Storchaka wrote: > 28.05.13 16:07, Nick Coghlan написав(ла): >> >> On Tue, May 28, 2013 at 10:31 PM, Antoine Pitrou >> wrote: >>> >>> Le Tue, 28 May 2013 22:15:25 +1000, >>> Nick Coghlan a écrit : * moved the main executable source file from Modules to a separate Apps directory >>> >>> Sounds fine (I don't like "Apps" much, but hey :-)). >> >> Unfortunately, I don't know any other short word for "things with main >> functions that we ship to end users" :) > > main IIRC, the reason I avoided that originally was due to the potential confusion between C's main and Python's main. I don't know why I didn't think of Fred's suggestion of "Programs" - I think that contrasts nicely with Modules, so I'd like to run with that. Cleanly separating out the main functions affected the PEP 432 feature branch directly because the whole point of that PEP is to make all of them simpler by moving more of the relevant code into the shared library. However, I really *don't* want to dive into the seemingly random allocation of some things between the Python/ subdir and the Modules/ subdir . If there's a consistent pattern there, I think it may be lost somewhere back in the 20th century, as I've never been able to figure one out... Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 8 and function names
On May 26, 2013, at 08:34 PM, Nick Coghlan wrote: >As far as I am aware, there's nothing to clarify: new code should use >underscores as word separators, code added to an existing module or >based on existing API should follow the conventions of that module or >API. This is what PEP 8 already says. Exactly so. -Barry ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bilingual scripts
On May 24, 2013, at 04:23 PM, R. David Murray wrote: >Gentoo has a (fairly complex) driver script that is symlinked to all >of these bin scripts. The system then has the concept of the >"current python", which can be set to python2 or python3. The default >bin then calls the current default interpreter. There are also >xxx2 and xxx3 versions of each bin script, which call the 'current' >version of python2 or python3, respectively. > >I'm sure one of the gentoo devs on this list can speak to this more >completely...I'm just a user :) But I must say that the system works >well from my point of view. Interesting approach, but it doesn't seem to me to be fundamentally different than the BPOS (big pile o' symlinks). Over in Debian-land one of the interesting points against a driver script was that folks like to be able to explicitly override the shebang line interpreter, e.g. $ head /usr/bin/foo #! /usr/bin/python3 -Es $ python3.4 /usr/bin/foo ... One other person mentioned they like to be able to execfile() - or the Python 3 moral equivalent - the /usr/bin script, which obvious would be harder with a sh or binary driver script. -Barry ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bilingual scripts
On Tue, 28 May 2013 11:35:00 -0400, Barry Warsaw wrote: > On May 24, 2013, at 04:23 PM, R. David Murray wrote: > > >Gentoo has a (fairly complex) driver script that is symlinked to all > >of these bin scripts. The system then has the concept of the > >"current python", which can be set to python2 or python3. The default > >bin then calls the current default interpreter. There are also > >xxx2 and xxx3 versions of each bin script, which call the 'current' > >version of python2 or python3, respectively. > > > >I'm sure one of the gentoo devs on this list can speak to this more > >completely...I'm just a user :) But I must say that the system works > >well from my point of view. > > Interesting approach, but it doesn't seem to me to be fundamentally different > than the BPOS (big pile o' symlinks). > > Over in Debian-land one of the interesting points against a driver script was > that folks like to be able to explicitly override the shebang line > interpreter, e.g. > > $ head /usr/bin/foo > #! /usr/bin/python3 -Es > $ python3.4 /usr/bin/foo > ... > > One other person mentioned they like to be able to execfile() - or the Python > 3 moral equivalent - the /usr/bin script, which obvious would be harder with a > sh or binary driver script. True. Another big disadvantage is that you can't just look in the file to find out what it is doing, which I *do* find be a significant drawback. I have the same complaint about setuptools entry-point scripts, where I still haven't figured out how to go from what is in the file to the code that actually gets called. --David ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bilingual scripts
On May 25, 2013, at 05:57 PM, Nick Coghlan wrote: >It seems to me the existing recommendation to use ``#!/usr/bin/env >python`` instead of referencing a particular binary already covers the >general case. The challenge for the distros is that we want a solution >that *ignores* user level virtual environments. Right. My general recommendation is that upstream's (development version) scripts use #! /usr/bin/env, but that distros and possibly even virtualenv/buildout installs, hardcode the #! to a specific interpreter. We've just had way too many problems when a /usr/bin script uses /usr/bin/env and breaks the world. We also recommend using -Es to isolate the environment as much as possible. -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bilingual scripts
On May 25, 2013, at 06:17 AM, Chris McDonough wrote: >I'm curious if folks have other concrete examples of global bindir >executables other than nosetests and pydoc that need to be disambiguated >by Python version. I'd hate to see it become standard practice to >append "3" to scripts generated by packages which happen to use Python >3, as it will just sort of perpetuate its otherness. tox https://bitbucket.org/hpk42/tox/issue/96/cant-have-a-python-3-setuppy -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 409 and the stdlib
On 5/28/2013 6:02 AM, Serhiy Storchaka wrote: 20.05.13 18:46, Antoine Pitrou написав(ла): I think it is a legitimate case where to silence the original exception. However, the binascii.Error would be more informative if it said *which* non-base32 digit was encountered. Please open a new issue for this request (note that no other binascii or base64 functions provide such information). Sounds like perhaps multiple issues would be useful to suggest enhancements for the error messages provided by other binascii and base64 functions. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bilingual scripts
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On 05/28/2013 11:41 AM, R. David Murray wrote:
> I have the same complaint about setuptools entry-point scripts, where
> I still haven't figured out how to go from what is in the file to the
> code that actually gets called.
Hmm, just dump the 'entry_points.txt' file in the named distribution's
EGG-INFO directory? E.g.:
$ cat bin/pip
#!/path/to/virtualenv/bin/pythonX.Y
# EASY-INSTALL-ENTRY-SCRIPT: 'pip==1.3.1','console_scripts','pip'
__requires__ = 'pip==1.3.1'
import sys
from pkg_resources import load_entry_point
if __name__ == '__main__':
sys.exit(
load_entry_point('pip==1.3.1', 'console_scripts', 'pip')()
)
$ cat
lib/pythonX.Y/site-packages/pip-1.3.1-pyX.Y.egg/EGG-INFO/entry_points.txt
[console_scripts]
pip = pip:main
pip-X.Y = pip:main
Tres.
- --
===
Tres Seaver +1 540-429-0999 [email protected]
Palladion Software "Excellence by Design"http://palladion.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with undefined - http://www.enigmail.net/
iEYEARECAAYFAlGk2K0ACgkQ+gerLs4ltQ6WaACZAbdz7k3sdM21DNx0mzcecY93
hvYAoJTwA2l3OvSoYStzGmsJ+N16JDwM
=YHcy
-END PGP SIGNATURE-
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Structural cleanups to the main CPython repo
Le Tue, 28 May 2013 23:07:37 +1000, Nick Coghlan a écrit : > > It was deliberate - a big part of PEP 432 is making sure that all the > interpreter state lives *in* the interpreter state (as part of the > config struct). Splitting the two into separate compilation modules > makes it possible to ensure that all communication goes via the > interpreter configuration (statics in other modules are still a > problem, but also mostly out of scope for PEP 432). > > I *really* want to get us to clean phase separation of "the > interpreter is starting up", "the interpreter is running normally" and > "the interpreter is shutting down". I found that to be incredibly > difficult to do when they were all intermixed in one file, which is > why I decided to enlist the compiler's help by separating them. It sounds a bit exagerated. We have encoders and decoders in the same (C) modules, compressors and decompressors ditto. Why not keep initialization and finalization in the same source file too? (how long are the resulting C files?) > > I don't know what goes > > in the remaining "pythonrun.c", could you detail a bit? > > While they have some of the PEP 432 changes in them, the header files > in the branch give the general flavour of the separation: > > Bootstrap is mostly get/init type functions: > https://bitbucket.org/ncoghlan/cpython_sandbox/src/ae7fef62b462fb6b559172bd4dbefc185ec28c40/Include/bootstrap.h?at=pep432_modular_bootstrap > > Pythonrun is mostly PyRun_*, PyParser_*, Py_Compile* and a few other > odds and ends: > https://bitbucket.org/ncoghlan/cpython_sandbox/src/ae7fef62b462fb6b559172bd4dbefc185ec28c40/Include/pythonrun.h?at=pep432_modular_bootstrap > > Shutdown covers the various finalisers, atexit handling, etc: > https://bitbucket.org/ncoghlan/cpython_sandbox/src/ae7fef62b462fb6b559172bd4dbefc185ec28c40/Include/shutdown.h?at=pep432_modular_bootstrap The fact that PyXXX_Init() and PyXXX_Fini() end up in different header files look like a red flag to me, modularization-wise. I agree to separate PyRun_* stuff from initialization/finalization routines, though. Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Structural cleanups to the main CPython repo
Am 28.05.13 15:07, schrieb Nick Coghlan: >> Sounds fine (I don't like "Apps" much, but hey :-)). > > Unfortunately, I don't know any other short word for "things with main > functions that we ship to end users" :) Bike-sheddingly: POSIX calls them "commands and utilities": https://www2.opengroup.org/ogsys/catalog/c436 Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Structural cleanups to the main CPython repo
Am 28.05.13 18:20, schrieb Antoine Pitrou: > Le Tue, 28 May 2013 23:07:37 +1000, > Nick Coghlan a écrit : >> It was deliberate - a big part of PEP 432 is making sure that all the >> interpreter state lives *in* the interpreter state (as part of the >> config struct). > > It sounds a bit exagerated. We have encoders and decoders in the same > (C) modules, compressors and decompressors ditto. Why not keep > initialization and finalization in the same source file too? I can sympathize with the motivation. Unlike encoders and decoders, it is *very* tempting to put interpreter state into global variables. With encoders and decoders, it's clear that globals won't work if you have multiple of them. With interpreter state, it's either singletons in the first place, or the globals can be swapped out when switching interpreters. By splitting initialization and finalization into distinct translation units, you make it much more difficult to introduce new "hidden" variables. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bilingual scripts
On May 27, 2013, at 11:38 AM, Toshio Kuratomi wrote: >Fedora is a bit of a mess... we try to work with upstream's intent when >upstream has realized this problem exists and have a single standard when >upstream does not. The full guidelines are here: > >http://fedoraproject.org/wiki/Packaging:Python#Naming Thanks. One of the reasons I've brought this up here is so that hopefully we can come up with recommendations for upstreams where this matters. One thing is for sure (IMO, anyway). Utilities that provide version-specific scripts should also provide -m invocation. E.g. there are various places where a package's tests (provided unittest, or other as-built tests) can be invoked. Where those might use nose, we recommend invoking them with `$python -m nose` instead using nosetests-X.Y. This also makes it easier to loop over all the versions of Python available on the system (which might not be known statically). >- If upstream doesn't deal with it, then we use a "python3-" prefix. This >matches with our package naming so it seemed to make sense. (But >Barry's point about locate and tab completion and such would be a reason >to revisit this... Perhaps standardizing on /usr/bin/foo2-python3 >[pathological case of having both package version and interpreter >version in the name.] Note that the Gentoo example also takes into account versions that might act differently based on the interpreter's implementation. So a -python3 suffix may not be enough. Maybe now we're getting into PEP 425 compatibility tag territory. > - (tangent from a different portion of this thread: we've found that this >is a larger problem than we would hope. There are some obvious ones >like >- ipython (implements a python interpreter so python2 vs python3 is > understandably important ad different). >- nosetests (the python source being operated on is run through the > python interpreter so the version has to match). >- easy_install (needs to install python modules to the correct > interpreter's site-packages. It decides the correct interpreter > according to which interpreter invoked it.) > >But recently we found a new class of problems: frameworks which are >bilinugual. For instance, if you have a web framework which has a >/usr/bin/django-admin script that can be used to quickstart a >project, run a python shell and automatically load your code, load your >ORM db schema and operate on it to make modifications to the db then >that script has to know whether your code is compatible with python2 or >python3. Yay. >> I think the simplest thing to do is just append the "3" to the binary >> name (as we do ourselves for pydoc) and then abide by the >> recommendations in PEP 394 to reference the correct system executable. >> >I'd rather not have a bare 3 for the issues notes above. Something like py3 >would be better. Same here. I definitely don't like the current Debian semi-convention (not standardized or consistent) of injecting a '3' in the middle of the name, e.g. py3compile or py3doc. Note that adopting PEP 425 conventions allows for -py3 suffix to mean any Python 3 version, compatible across minor version numbers or implementations. This probably translates into a shebang line of #! /usr/bin/python3 whereas -py33 would mean #! /usr/bin/python3.3 This might be overkill in some cases, but at least it builds on existing standards. >There's still room for confusion when distributions have to push multiple >versions of a package with scripts that fall into this category. Should the >format be: > >/usr/bin/foo2-py3 (My preference as it places the version next to the >thing that it's a version of.) > >or > >/usr/bin/foo-py3-2 (Confusing as the 2 is bare. Something like >/usr/bin/foo-py3-v2 is slightly better but still not as nice as the >previous IMHO) Definitely the former, especially if PEP 425 serves at the basis for standardization. -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bilingual scripts
On May 25, 2013, at 09:53 AM, Antoine Pitrou wrote: >How about always running the version specific targets, e.g. >nosetests-2.7? We have nosetests-2.7 and nosetests3 in /usr/bin, but we generally recommend folks not use these, especially for things like (build time) package tests. It's harder to iterate over when the installed versions are unknown statically, e.g. if you wanted to run all the tests over all available versions of Python. For those, we recommend people use `$python -m nose` since the available versions of Python can be queried from the system. This is why I would really like to see all scripts provide a -m equivalent for command line invocation. This might be a little awkward for < Python 2.7 (where IIRC -m doesn't work with packages). -Barry ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bilingual scripts
On May 25, 2013, at 03:12 AM, Chris McDonough wrote: >You probably already know this, but I'll mention it anyway. This >probably matters a lot for nose and pyflakes, but I'd say that for tox >it should not, it basically just scripts execution of shell commands. >I'd think maybe in cases like tox (and others that are compatible with >both Python 2 and 3) the hashbang should just be set to >"#!/usr/bin/python" unconditionally. Unfortunately, not entirely so: https://bitbucket.org/hpk42/tox/issue/96/cant-have-a-python-3-setuppy >Maybe we could also think about modifying pyflakes so that it can >validate both 2 and 3 code (choosing one or the other based on a header >line in the validated files and defaulting to the version of Python >being run). This is kind of the right thing anyway. Agreed. Auto-detection may need to be accompanied by a command line option to override in some cases. But I agree, that in general, it would be very nice if the script itself were actually bilingual. (But then, see my previous comment about cross-interpreter dependencies.) >Nose is a bit of a special case. I personally never run nosetests >directly, I always use setup.py nosetests, which makes it not matter. Which is morally equivalent to `$python -m nose`. >In general, I'd like to think that scripts that get installed to global >bindirs will execute utilities that are useful independent of the >version of Python being used to execute them. Agreed. I'm trying to tease out some conventions we can recommend for when this can't be the case for whatever reason. -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bilingual scripts
On Tue, May 28, 2013 at 1:30 PM, Barry Warsaw wrote: > On May 25, 2013, at 03:12 AM, Chris McDonough wrote: > >>You probably already know this, but I'll mention it anyway. This >>probably matters a lot for nose and pyflakes, but I'd say that for tox >>it should not, it basically just scripts execution of shell commands. >>I'd think maybe in cases like tox (and others that are compatible with >>both Python 2 and 3) the hashbang should just be set to >>"#!/usr/bin/python" unconditionally. > > Unfortunately, not entirely so: > > https://bitbucket.org/hpk42/tox/issue/96/cant-have-a-python-3-setuppy > >>Maybe we could also think about modifying pyflakes so that it can >>validate both 2 and 3 code (choosing one or the other based on a header >>line in the validated files and defaulting to the version of Python >>being run). This is kind of the right thing anyway. > > Agreed. Auto-detection may need to be accompanied by a command line option to > override in some cases. But I agree, that in general, it would be very nice > if the script itself were actually bilingual. (But then, see my previous > comment about cross-interpreter dependencies.) > >>Nose is a bit of a special case. I personally never run nosetests >>directly, I always use setup.py nosetests, which makes it not matter. > > Which is morally equivalent to `$python -m nose`. > >>In general, I'd like to think that scripts that get installed to global >>bindirs will execute utilities that are useful independent of the >>version of Python being used to execute them. > > Agreed. I'm trying to tease out some conventions we can recommend for when > this can't be the case for whatever reason. > > -Barry Wheel has no mechanism for renaming scripts (or any file) based on the Python version used to install. Instead you would have to build python-version-specific packages for each desired script name. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bilingual scripts
On Tue, 28 May 2013 13:27:18 -0400
Barry Warsaw wrote:
> On May 25, 2013, at 09:53 AM, Antoine Pitrou wrote:
>
> >How about always running the version specific targets, e.g.
> >nosetests-2.7?
>
> We have nosetests-2.7 and nosetests3 in /usr/bin, but we generally recommend
> folks not use these, especially for things like (build time) package tests.
> It's harder to iterate over when the installed versions are unknown
> statically, e.g. if you wanted to run all the tests over all available
> versions of Python.
It sounds like you want a dedicated script or utility for this ("run
all the tests over all available versions of Python") rather than hack
it every time you package a Python library.
Your use case also doesn't seem to impact end-users.
> This is why I would really like to see all scripts provide a -m equivalent for
> command line invocation. This might be a little awkward for < Python 2.7
> (where IIRC -m doesn't work with packages).
Do you still support Python < 2.7?
Regards
Antoine.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 409 and the stdlib
On Tue, 28 May 2013 16:02:00 +0300 Serhiy Storchaka wrote: > 20.05.13 18:46, Antoine Pitrou написав(ла): > > I think it is a legitimate case where to silence the original > > exception. However, the binascii.Error would be more informative if it > > said *which* non-base32 digit was encountered. > > Please open a new issue for this request (note that no other binascii or > base64 functions provide such information). No, my point was that the KeyError gives you this information (when displayed as a context), silencing it removes the information. Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 409 and the stdlib
On 29/05/13 04:00, Antoine Pitrou wrote: On Tue, 28 May 2013 16:02:00 +0300 Serhiy Storchaka wrote: 20.05.13 18:46, Antoine Pitrou написав(ла): I think it is a legitimate case where to silence the original exception. However, the binascii.Error would be more informative if it said *which* non-base32 digit was encountered. Please open a new issue for this request (note that no other binascii or base64 functions provide such information). No, my point was that the KeyError gives you this information (when displayed as a context), silencing it removes the information. That is an accidental side-effect of the specific implementation, and does not occur in any of the versions of Python I have access to (production versions of 2.4 through 2.7, plus 3.2 and 3.3.0rc3). If the implementation changes again in the future, that information will be lost again. Relying on the context in this case to display this information is harmful for at least three reasons: - it's an accident of implementation; - it suggests that the binascii.Error is a bug in the error handling, when that is not the case; - and it is semantically irrelevant to the error being reported. The semantics of the error are "an invalid character has been found", not "an expected key is not found". I try not to throw references to the Zen around too lightly, but I think "Explicit is better than implicit" is appropriate here. If it is helpful for the error to show the invalid character, and I hope that we all agree that it is, then the binascii.Error message should explicitly show that character, rather than rely on the implementation implicitly showing it as a side-effect. -- Steven ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bilingual scripts
On May 28, 2013, at 01:57 PM, Daniel Holth wrote: >Wheel has no mechanism for renaming scripts (or any file) based on the >Python version used to install. Instead you would have to build >python-version-specific packages for each desired script name. Note that I'm not trying to borrow any implementation details from wheels, just the file naming conventions (compatibility tags) described in PEP 425. It would still be up to upstream package or distro tools to fiddle the installed file names. -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bilingual scripts
On Tue, May 28, 2013 at 2:04 PM, Barry Warsaw wrote: > On May 28, 2013, at 01:57 PM, Daniel Holth wrote: > >>Wheel has no mechanism for renaming scripts (or any file) based on the >>Python version used to install. Instead you would have to build >>python-version-specific packages for each desired script name. > > Note that I'm not trying to borrow any implementation details from wheels, > just the file naming conventions (compatibility tags) described in PEP 425. > It would still be up to upstream package or distro tools to fiddle the > installed file names. I'm just saying that I prefer a setup.py without too many Python-version-specific differences, since it would look pretty silly to install a wheel of nose generated on Python 3.2 on Python 3.3 and have the wrong version suffix on the scripts. I like the plainly named scripts without version suffixes. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] pep 422 "Safe object finalization" question: why break weakrefs first?
Pep 422 proposes the following order for dealing with cyclic isolates: 1. Weakrefs to CI objects are cleared, and their callbacks called. At this point, the objects are still safe to use. 2. The finalizers of all CI objects are called. 3. The CI is traversed again to determine if it is still isolated. If it is determined that at least one object in CI is now reachable from outside the CI, this collection is aborted and the whole CI is resurrected. Otherwise, proceed. 4. The CI becomes a CT as the GC systematically breaks all known references inside it (using the tp_clear function). Why are weakrefs are broken first, before determining if any of the objects should become resurrected? Naively I would expect weakrefs to be broken after step 3, once the system is sure no objects have been resurrected. I request that this information be added to PEP 422. -- Russell ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bilingual scripts
On Tue, May 28, 2013 at 01:22:01PM -0400, Barry Warsaw wrote: > On May 27, 2013, at 11:38 AM, Toshio Kuratomi wrote: > > >- If upstream doesn't deal with it, then we use a "python3-" prefix. This > >matches with our package naming so it seemed to make sense. (But > >Barry's point about locate and tab completion and such would be a reason > >to revisit this... Perhaps standardizing on /usr/bin/foo2-python3 > >[pathological case of having both package version and interpreter > >version in the name.] > > Note that the Gentoo example also takes into account versions that might act > differently based on the interpreter's implementation. So a -python3 suffix > may not be enough. Maybe now we're getting into PEP 425 compatibility tag > territory. > This is an interesting, unmapped area in Fedora at the moment... I was hoping to talk to Nick and the Fedora python maintainer at our next Fedora conference. I've been looking at how Fedora's ruby guidelines are implemented wrt alternate interpreters and wondering if we could do something similar for python: https://fedoraproject.org/wiki/Packaging:Ruby#Different_Interpreters_Compatibility I'm not sure yet how much of that I'd (or Nick and the python maintainer [bkabrda, the current python maintainer is the one who wrote the rubypick script]) would want to use in python -- replacing /usr/bin/python with a script that chooses between CPython and pypy based on user preference gave me an instinctual feeling of dread the first time I looked at it but it seems to be working well for the ruby folks. My current feeling is that I wouldn't use this same system for interpreters which are not mostly compatible (for instance, python2 vs python3). but I also haven't devoted much actual time to thinking about whether that might have some advantages. -Toshio pgpKoSrX0710o.pgp Description: PGP signature ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] pep 422 "Safe object finalization" question: why break weakrefs first?
On Tue, 28 May 2013 12:21:25 -0700 "Russell E. Owen" wrote: > Pep 422 proposes the following order for dealing with cyclic isolates: > >1. Weakrefs to CI objects are cleared, and their callbacks called. At > this point, the objects are still safe to use. >2. The finalizers of all CI objects are called. >3. The CI is traversed again to determine if it is still isolated. If > it is determined that at least one object in CI is now reachable from > outside the CI, this collection is aborted and the whole CI is > resurrected. Otherwise, proceed. >4. The CI becomes a CT as the GC systematically breaks all known > references inside it (using the tp_clear function). > > Why are weakrefs are broken first, before determining if any of the > objects should become resurrected? Naively I would expect weakrefs to be > broken after step 3, once the system is sure no objects have been > resurrected. The answer is that this is how weakrefs currently work: they are cleared (and their callbacks executed) before __del__ is executed, therefore if __del__ revives the object, the weakrefs stay dead. The rationale is simply to minimize disruption for existing code. However, the PEP would indeed make it possible to change that behaviour, if desired. You can read http://hg.python.org/cpython/file/4e687d53b645/Modules/gc_weakref.txt for a detailed (and lengthy) explanation of why weakrefs work that way right now. Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython (merge 3.3 -> default): Merge with 3.3
On Tue, 28 May 2013 15:06:39 -0400 Terry Reedy wrote: > On 5/28/2013 2:14 AM, Benjamin Peterson wrote: > > 2013/5/27 terry.reedy : > >> http://hg.python.org/cpython/rev/c5d4c041ab47 > >> changeset: 83942:c5d4c041ab47 > >> parent: 83940:2ea849fde22b > >> parent: 83941:24c3e7e08168 > >> user:Terry Jan Reedy > >> date:Mon May 27 21:33:40 2013 -0400 > >> summary: > >>Merge with 3.3 > >> > >> files: > >>Lib/idlelib/CallTips.py | 4 +- > >>Lib/idlelib/PathBrowser.py| 3 +- > >>Lib/idlelib/idle_test/@README.txt | 63 +++ > > Is @README really the intended name of this file? > Yes, Nick suggested README instead of what I had. I want a prefix to > keep it near the top of a directory listing even when other non > 'test_xxx' files are added. I thing '_' wold be better though. I don't think "prefixing with a weird character so that the filename show up top" is a very elegant trick, and we don't use it for other directories. "README.txt" *will* be easily visible because of the all-caps filename. > If I can > find how to rename in hg "Rename in hg" -> "hg rename" Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 443 - Single-dispatch generic functions (including ABC support)
In article , Åukasz Langa wrote: > Hello, > Since the initial version, several minor changes have been made to the > PEP. The history is visible on hg.python.org. The most important > change in this version is that I introduced ABC support and completed > a reference implementation. > > No open issues remain from my point of view. Is it true that this cannot be used for instance and class methods? It dispatches based on the first argument, which is "self" for instance methods, whereas the second argument would almost certainly be the argument one would want to use for conditional dispatch. -- Russell ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 443 - Single-dispatch generic functions (including ABC support)
A question about the example: how hard would it be to modify the example @fun.register(list) ... to work with other collections? If it is easy, I think it would make a for a much more useful example. -- Russell ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 443 - Single-dispatch generic functions (including ABC support)
On Tue, May 28, 2013 at 3:41 PM, Russell E. Owen wrote:
> Is it true that this cannot be used for instance and class methods? It
> dispatches based on the first argument, which is "self" for instance
> methods, whereas the second argument would almost certainly be the
> argument one would want to use for conditional dispatch.
You can use a staticmethod and then delegate to it, of course. But it
probably wouldn't be too difficult to allow specifying which argument
to dispatch on, e.g.:
@singledispatch.on('someArg')
def my_method(self, someArg, ...):
...
The code would look something like this:
def singledispatch(func, argPosn=0):
...
# existing code here...
...
def wrapper(*args, **kw):
return dispatch(args[argPosn].__class__)(*args, **kw) #
instead of args[0]
def _dispatch_on(argname):
def decorate(func):
argPosn = # code to find argument position of argname for func
return dispatch(func, argPosn)
return decorate
singledispatch.on = _dispatch_on
So, it's just a few lines added, but of course additional doc, tests,
etc. would have to be added as well. (It also might be a good idea
for there to be some error checking in wrapper() to raise an
approriate TypeError if len(args)<=arg.)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Introduce importlib.util.ModuleManager which is a context manager to
On Tue, 28 May 2013 23:29:46 +0200 (CEST) brett.cannon wrote: > > +.. class:: ModuleManager(name) > + > +A :term:`context manager` which provides the module to load. The module > will > +either come from :attr:`sys.modules` in the case of reloading or a fresh > +module if loading a new module. Proper cleanup of :attr:`sys.modules` > occurs > +if the module was new and an exception was raised. What use case does this API solve? (FWIW, I think "ModuleManager" is a rather bad name :-)) Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bilingual scripts
On Tue, 28 May 2013 12:17:49 -0400, Tres Seaver wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> On 05/28/2013 11:41 AM, R. David Murray wrote:
> > I have the same complaint about setuptools entry-point scripts, where
> > I still haven't figured out how to go from what is in the file to the
> > code that actually gets called.
>
> Hmm, just dump the 'entry_points.txt' file in the named distribution's
> EGG-INFO directory? E.g.:
>
> $ cat bin/pip
> #!/path/to/virtualenv/bin/pythonX.Y
> # EASY-INSTALL-ENTRY-SCRIPT: 'pip==1.3.1','console_scripts','pip'
> __requires__ = 'pip==1.3.1'
> import sys
> from pkg_resources import load_entry_point
>
> if __name__ == '__main__':
> sys.exit(
> load_entry_point('pip==1.3.1', 'console_scripts', 'pip')()
> )
>
> $ cat
> lib/pythonX.Y/site-packages/pip-1.3.1-pyX.Y.egg/EGG-INFO/entry_points.txt
> [console_scripts]
> pip = pip:main
> pip-X.Y = pip:main
I'm afraid I'm still not enlightened.
I'm sure I would understand this if I had ever set up an entry point,
since I would have had to read the docs on how to do it. But I never
have.
So, my point is that the information on what python code is actually being
called ought to be in the stub script file, as a comment if nothing else,
for discoverability reasons.
I'm not bothered enough to work up a patch, though :)
--David
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython (merge 3.3 -> default): Merge with 3.3
On 5/28/2013 3:39 PM, Antoine Pitrou wrote: On Tue, 28 May 2013 15:06:39 -0400 Terry Reedy wrote: Yes, Nick suggested README instead of what I had. I want a prefix to keep it near the top of a directory listing even when other non 'test_xxx' files are added. I thing '_' wold be better though. I don't think "prefixing with a weird character '_' is not weird for Python names. so that the filename show up top" is a very elegant trick, I disagree. Books have Table of Contents, Preface, and Foreword sections at the front for a reason: if they are present, they are easy and obvious to find. READMEs are like a preface*, sometimes with an annotated Contents. They logically belong at the top for the same reason. A long title for this how-to file, which I would prefer, would be something like "_Writing-Testing-Running_Idle_Tests", or "_Idle_Test-Writing_Guidelines", or "A_Guide_to_Idle_Tests", or "An_Idle_Test_HOWTO". *At least this one is. Some are addenda that have little to do with the other files in the directory. They might better have a different name, like 'Manual Corrections' (which would sort after 'Manual', where it belongs), "Starting the game', 'Windows differences', etc. I don't know if 'readme.txt' was common before DOS 8.3 filename limitations, but that limit is mostly gone. I have used this 'trick' for decades. Another file that does not belong in the main alpha list is a project-specific template for the .py files in a directory. > and we don't use it for other directories. I think we should. Seriously. Maybe with more descriptive names. "README.txt" *will* be easily visible because of the all-caps filename. Somewhat easy, but only if one thinks to look for it. I only found 4 used outside of Lib/test/*. Which of the following big directories have a README? /Lib /Lib/idlelib /Lib/test /Tools/scripts Would it not be easier to discover if the 'preface' file were always at or near the top? "Rename in hg" -> "hg rename" Thanks. Found it with Google and read that it works well in hg. I will also check if TortoiseHG has an easy gui equivalent (rt click on file to rename). Terry ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython (merge 3.3 -> default): Merge with 3.3
On 29/05/2013 00:29, Terry Jan Reedy wrote: On 5/28/2013 3:39 PM, Antoine Pitrou wrote: On Tue, 28 May 2013 15:06:39 -0400 Terry Reedy wrote: Yes, Nick suggested README instead of what I had. I want a prefix to keep it near the top of a directory listing even when other non 'test_xxx' files are added. I thing '_' wold be better though. I don't think "prefixing with a weird character '_' is not weird for Python names. so that the filename show up top" is a very elegant trick, I disagree. Books have Table of Contents, Preface, and Foreword sections at the front for a reason: if they are present, they are easy and obvious to find. READMEs are like a preface*, sometimes with an annotated Contents. They logically belong at the top for the same reason. A long title for this how-to file, which I would prefer, would be something like "_Writing-Testing-Running_Idle_Tests", or "_Idle_Test-Writing_Guidelines", or "A_Guide_to_Idle_Tests", or "An_Idle_Test_HOWTO". [snip] I'm somehow not happy about "_README", what with a single underscore indicating "internal" in Python code. Perhaps it would be a bit more Pythonic to have "_README_" instead (dunder would be overdoing it, perhaps). :-) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Introduce importlib.util.ModuleManager which is a context manager to
On Tue, May 28, 2013 at 5:40 PM, Antoine Pitrou wrote: > On Tue, 28 May 2013 23:29:46 +0200 (CEST) > brett.cannon wrote: >> >> +.. class:: ModuleManager(name) >> + >> +A :term:`context manager` which provides the module to load. The module >> will >> +either come from :attr:`sys.modules` in the case of reloading or a fresh >> +module if loading a new module. Proper cleanup of :attr:`sys.modules` >> occurs >> +if the module was new and an exception was raised. > > What use case does this API solve? See http://bugs.python.org/issue18088 for the other part of this story. I'm basically replacing what importlib.util.module_for_loader does after I realized there is no way in a subclass to override what/how attributes are set on a module before the code object is executed. Instead of using the decorator people will be able to use this context manager with a new method to get the same effect with the ability to better control attribute initialization. > (FWIW, I think "ModuleManager" is a rather bad name :-) I'm open to suggestions, but the thing does manage the module so it at least makes sense. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython (merge 3.3 -> default): Merge with 3.3
On 5/28/2013 7:42 PM, MRAB wrote: "A_Guide_to_Idle_Tests", or "An_Idle_Test_HOWTO". [snip] I'm somehow not happy about "_README", what with a single underscore indicating "internal" in Python code. The file is internal to the subset of IDLE developers writing tests, but... Perhaps it would be a bit more Pythonic to have "_README_" instead (dunder would be overdoing it, perhaps). :-) Guido said README.txt on the committers list. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 443 - Single-dispatch generic functions (including ABC support)
On 29/05/13 07:27, PJ Eby wrote:
On Tue, May 28, 2013 at 3:41 PM, Russell E. Owen wrote:
Is it true that this cannot be used for instance and class methods? It
dispatches based on the first argument, which is "self" for instance
methods, whereas the second argument would almost certainly be the
argument one would want to use for conditional dispatch.
You can use a staticmethod and then delegate to it, of course. But it
probably wouldn't be too difficult to allow specifying which argument
to dispatch on, e.g.:
@singledispatch.on('someArg')
def my_method(self, someArg, ...):
...
[...]
So, it's just a few lines added, but of course additional doc, tests,
etc. would have to be added as well. (It also might be a good idea
for there to be some error checking in wrapper() to raise an
approriate TypeError if len(args)<=arg.)
I feel that specifying the dispatch argument in full generality is overkill,
and that supporting two use-cases should be sufficient:
- dispatch on the first argument of functions;
- dispatch on the second argument of methods, skipping self/cls.
After all, is this not supposed to be *simple* generics? :-)
I'm vaguely leaning towards @singledispatch and @singledispatch.method for the
colour of this bike shed.
--
Steven
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Structural cleanups to the main CPython repo
On Wed, May 29, 2013 at 2:47 AM, "Martin v. Löwis" wrote: > Am 28.05.13 18:20, schrieb Antoine Pitrou: >> Le Tue, 28 May 2013 23:07:37 +1000, >> Nick Coghlan a écrit : >>> It was deliberate - a big part of PEP 432 is making sure that all the >>> interpreter state lives *in* the interpreter state (as part of the >>> config struct). >> >> It sounds a bit exagerated. We have encoders and decoders in the same >> (C) modules, compressors and decompressors ditto. Why not keep >> initialization and finalization in the same source file too? > > I can sympathize with the motivation. Unlike encoders and decoders, > it is *very* tempting to put interpreter state into global variables. > With encoders and decoders, it's clear that globals won't work if you > have multiple of them. With interpreter state, it's either singletons > in the first place, or the globals can be swapped out when switching > interpreters. > > By splitting initialization and finalization into distinct translation > units, you make it much more difficult to introduce new "hidden" > variables. Yep, that was a key part of my motivation (the other part was also to find out what global state we *already had* by making the build blow up for anything that was static and referenced by more than just the bootstrapping code). The part I didn't think through when I did it in a long-lived branch was just how much of nightmare it was going to make any merges that touched pythonrun.h or pythonrun.c :) I'd also be open to a setup with a single "lifecycle.h" header file, which was split into the bootstrap and shutdown implementation units, since that makes it easier to check that the appropriate setup/finalize pairs exist (by looking at the combined header file), while still enlisting the build chain's assistance in avoiding hidden global state. Anway, I'll come up with some specific patches and put them on the tracker, starting with moving the source files for the binary executables and making the simpler pythonrun/lifecycle split. I can look into splitting lifecycle.c into separate bootstrap and shutdown translation units after those less controversial changes have been reviewed (the split may not even be all that practical outside the PEP 432 branch, since it would involve exposing quite a few currently static variables to the linker). Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bilingual scripts
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On 05/28/2013 05:52 PM, R. David Murray wrote:
> On Tue, 28 May 2013 12:17:49 -0400, Tres Seaver
> wrote:
>> -BEGIN PGP SIGNED MESSAGE- Hash: SHA1
>>
>> On 05/28/2013 11:41 AM, R. David Murray wrote:
>>> I have the same complaint about setuptools entry-point scripts,
>>> where I still haven't figured out how to go from what is in the
>>> file to the code that actually gets called.
>>
>> Hmm, just dump the 'entry_points.txt' file in the named
>> distribution's EGG-INFO directory? E.g.:
>>
>> $ cat bin/pip #!/path/to/virtualenv/bin/pythonX.Y #
>> EASY-INSTALL-ENTRY-SCRIPT: 'pip==1.3.1','console_scripts','pip'
>> __requires__ = 'pip==1.3.1' import sys from pkg_resources import
>> load_entry_point
>>
>> if __name__ == '__main__': sys.exit( load_entry_point('pip==1.3.1',
>> 'console_scripts', 'pip')() )
>>
>> $ cat
>> lib/pythonX.Y/site-packages/pip-1.3.1-pyX.Y.egg/EGG-INFO/entry_points.txt
>>
>>
[console_scripts]
>> pip = pip:main pip-X.Y = pip:main
>
> I'm afraid I'm still not enlightened.
>
> I'm sure I would understand this if I had ever set up an entry point,
> since I would have had to read the docs on how to do it. But I never
> have.
>
> So, my point is that the information on what python code is actually
> being called ought to be in the stub script file, as a comment if
> nothing else, for discoverability reasons.
>
> I'm not bothered enough to work up a patch, though :)
It is there already:
# EASY-INSTALL-ENTRY-SCRIPT: 'pip==1.3.1','console_scripts','pip'
Which says, load the entry point named 'pip' from the 'console_scripts'
entry point group in the 'pip 1.3.1' distribution.
The 'entry_points.txt' metadata file specifies that that entry point is a
function named 'main' inside the 'pip' package itself.
Ters.
- --
===
Tres Seaver +1 540-429-0999 [email protected]
Palladion Software "Excellence by Design"http://palladion.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with undefined - http://www.enigmail.net/
iEUEARECAAYFAlGlZesACgkQ+gerLs4ltQ50xACeJUBMjAvMBaOm63Viigz2bvkP
S5gAl2w4WAxgasXie10DMtHJOyRRFvA=
=34KH
-END PGP SIGNATURE-
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 443 - Single-dispatch generic functions (including ABC support)
On Wed, May 29, 2013 at 5:41 AM, Russell E. Owen wrote: > In article , > Łukasz Langa wrote: > >> Hello, >> Since the initial version, several minor changes have been made to the >> PEP. The history is visible on hg.python.org. The most important >> change in this version is that I introduced ABC support and completed >> a reference implementation. >> >> No open issues remain from my point of view. > > Is it true that this cannot be used for instance and class methods? It > dispatches based on the first argument, which is "self" for instance > methods, whereas the second argument would almost certainly be the > argument one would want to use for conditional dispatch. Correct. OO and generic functions are different development paradigms, and there are limitations on mixing them. Generic functions are for stateless algorithms, which expect to receive all required input through their arguments. By contrast, class and instance methods expect to receive some state implicitly - in many respects, they *already are* generic functions. Thus, this is really a request for dual dispatch in disguise: you want to first dispatch on the class or instance (through method dispatch) and *then* dispatch on the second argument (through generic function dispatch). Dual dispatch is much harder than single dispatch and "functools.singledispatch" does not and should not support it (it's in the name). As PJE noted, you *can* use singledispatch with staticmethods, as that eliminates the dual dispatch behaviour by removing the class and instance based dispatch step. You can also register already bound class and instance methods as implementations for a generic function, as that also resolves the dual dispatch in a way that means the single dispatch implementation doesn't even need to be aware it is happening. I expect we will see improved tools for integrating class based dispatch and generic function dispatch in the future, but we should *not* try to engineer a solution up front. Doing so would involve too much guessing about possible use cases, rather than letting the design be informed by the *actual* use cases that emerge in practice. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bilingual scripts
On Wed, May 29, 2013 at 5:23 AM, Toshio Kuratomi wrote: > On Tue, May 28, 2013 at 01:22:01PM -0400, Barry Warsaw wrote: >> On May 27, 2013, at 11:38 AM, Toshio Kuratomi wrote: >> >> >- If upstream doesn't deal with it, then we use a "python3-" prefix. This >> >matches with our package naming so it seemed to make sense. (But >> >Barry's point about locate and tab completion and such would be a reason >> >to revisit this... Perhaps standardizing on /usr/bin/foo2-python3 >> >[pathological case of having both package version and interpreter >> >version in the name.] >> >> Note that the Gentoo example also takes into account versions that might act >> differently based on the interpreter's implementation. So a -python3 suffix >> may not be enough. Maybe now we're getting into PEP 425 compatibility tag >> territory. >> > This is an interesting, unmapped area in Fedora at the moment... I > was hoping to talk to Nick and the Fedora python maintainer at our next > Fedora conference. > > I've been looking at how Fedora's ruby guidelines are implemented wrt > alternate interpreters and wondering if we could do something similar for > python: > > https://fedoraproject.org/wiki/Packaging:Ruby#Different_Interpreters_Compatibility > > I'm not sure yet how much of that I'd (or Nick and the python maintainer > [bkabrda, the current python maintainer is the one who wrote the rubypick > script]) would want to use in python -- replacing /usr/bin/python with a > script that chooses between CPython and pypy based on user preference gave > me an instinctual feeling of dread the first time I looked at it but it > seems to be working well for the ruby folks. > > My current feeling is that I wouldn't use this same system for interpreters > which are not mostly compatible (for instance, python2 vs python3). but I > also haven't devoted much actual time to thinking about whether that might > have some advantages. PEP 432 is also related, as it includes the "pysystem" proposal [1] (an alternate Python CLI that will default to -Es behaviour, but is otherwise similar to the standard "python" interpreter). The rest of the discussion though makes me think we may actually need a *nix equivalent of PEP 397 (which describes the "py" launcher we created to get around the limitations of Windows file associations). Between that and the interpreter identification mechanism defined for the PEP 425 compatibility tags it should be possible to come up with an upstream solution for 3.4 that the distros can backport to work with earlier versions (similar to the way users can download the Windows launcher directly from https://bitbucket.org/pypa/pylauncher/downloads even though we only started shipping it upstream as part of the Python 3.3 installer) Cheers, Nick. [1] http://www.python.org/dev/peps/pep-0432/#a-system-python-executable -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Introduce importlib.util.ModuleManager which is a context manager to
On Wed, May 29, 2013 at 10:14 AM, Brett Cannon wrote: >> (FWIW, I think "ModuleManager" is a rather bad name :-) > > I'm open to suggestions, but the thing does manage the module so it at > least makes sense. I suggest ModuleInitialiser as the CM name, with a helper function to make usage read better: with initialise_module(name) as m: # Module initialisation code goes here # Module is rolled back if initialisation fails Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 443 - Single-dispatch generic functions (including ABC support)
On Wed, 29 May 2013 12:40:32 +1000 Nick Coghlan wrote: > On Wed, May 29, 2013 at 5:41 AM, Russell E. Owen wrote: > > In article , > > Łukasz Langa wrote: > > > >> Hello, > >> Since the initial version, several minor changes have been made to the > >> PEP. The history is visible on hg.python.org. The most important > >> change in this version is that I introduced ABC support and completed > >> a reference implementation. > >> > >> No open issues remain from my point of view. > > > > Is it true that this cannot be used for instance and class methods? It > > dispatches based on the first argument, which is "self" for instance > > methods, whereas the second argument would almost certainly be the > > argument one would want to use for conditional dispatch. > > Correct. OO and generic functions are different development paradigms, > and there are limitations on mixing them. Generic functions are for > stateless algorithms, which expect to receive all required input > through their arguments. By contrast, class and instance methods > expect to receive some state implicitly - in many respects, they > *already are* generic functions. There are actual use cases for generic methods, think pickle.py. (also, often a "stateless" function will eventually become stateful, if used as part of a sufficiently complex application / library; e.g. some logging will be added, or some kind of configuration object) Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 443 - Single-dispatch generic functions (including ABC support)
On Wed, 29 May 2013 08:08:14 +0200 Antoine Pitrou wrote: > On Wed, 29 May 2013 12:40:32 +1000 > Nick Coghlan wrote: > > On Wed, May 29, 2013 at 5:41 AM, Russell E. Owen wrote: > > > In article , > > > Łukasz Langa wrote: > > > > > >> Hello, > > >> Since the initial version, several minor changes have been made to the > > >> PEP. The history is visible on hg.python.org. The most important > > >> change in this version is that I introduced ABC support and completed > > >> a reference implementation. > > >> > > >> No open issues remain from my point of view. > > > > > > Is it true that this cannot be used for instance and class methods? It > > > dispatches based on the first argument, which is "self" for instance > > > methods, whereas the second argument would almost certainly be the > > > argument one would want to use for conditional dispatch. > > > > Correct. OO and generic functions are different development paradigms, > > and there are limitations on mixing them. Generic functions are for > > stateless algorithms, which expect to receive all required input > > through their arguments. By contrast, class and instance methods > > expect to receive some state implicitly - in many respects, they > > *already are* generic functions. > > There are actual use cases for generic methods, think pickle.py. That said, I admit this is a case where the generic method use is private, i.e. is not exposed for other code to extend. (the public extension protocol being in the form of plain methods: __getstate__, etc.) Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
