Re: [Python-Dev] I was just thinking that os.path could use some love...
Le Wed, 30 Jan 2013 13:26:08 -0800,
Guido van Rossum a écrit :
> Thoughts on os.path? What happened to the idea of a new path object?
I plan to launch another round of discussions following the changes in
PEP 428.
Regards
Antoine.
>
> --Guido
>
>
> -- Forwarded message --
> From: Talin
> Date: Wed, Jan 30, 2013 at 12:34 PM
> Subject: Re: I was just thinking that os.path could use some love...
> To: Guido van Rossum
>
>
> On Wed, Jan 30, 2013 at 11:33 AM, Guido van Rossum
> wrote:
> >
> > Hmm... Mind if I just forward to python-dev? IIRC there's been some
> > discussion about a newfangled path object, but I lost track of where
> > the discussion went. (Or you can post there yourself. :-)
>
>
> Sure - forward away. Is this the same as the path object idea that was
> floated 5 years ago? I've yet to see any set of convenience methods
> for paths that are so compelling as to be worth all of the time and
> energy needed to update all of the various APIs which now expect paths
> to be passed in as strings.
> >
> >
> > On Wed, Jan 30, 2013 at 11:13 AM, Talin wrote:
> > > I just realized that os.path hasn't changed in a long time.
> > > Here's a couple of ideas for additions:
> > >
> > > os.path.splitall(path) - splits all path components into a tuple
> > > - so for example, 'this/is/a/path' turns into ('this', 'is', 'a',
> > > 'path'). If there's a trailing slash, the last item in the tuple
> > > will be a zero-length string. The main reason for having this in
> > > os.path is so that we can remain separator-character-agnostic.
> >
> > Would it also return a leading empty string if the path starts
> > with /? What about Windows C: or //host/ prefixes???
> >
> I would say that it should only split on directory separators, not any
> other kind of delimiter, so that each component is a valid filesystem
> identifier. Further, the operation should be reversible by calling
> os.path.join(*dirnames). So it's a little more complex than just
> string.split('/').
>
> Part of the reason for wanting the splitall function is to implement
> the common prefix function - you take all the paths, bust them into
> tuples, look for the longest tuple prefix, and then join the result
> back into a path. This means that
> os.path.join(*os.path.splitall(path)) must reproduce the original path
> exactly.
>
> >
> > > An alternative would be to add an optional 'maxsplits' parameter
> > > to os.path.split. Default would be 1; 0 = unlimited. Main
> > > disadvantage would be that no one would ever use values 2, 3,
> > > etc... but it would be more compatible with other split apis.
> > >
> > > A version of os.path.commonprefix that always produces valid
> > > paths (and works by path component, not character-by-character).
> > >
> > > I can probably think of others if I look at some other path
> > > manipulation apis. llvm::sys::path has a bunch if I recall...
> > >
> > > --
> > > -- Talin
> >
> >
> >
> > --
> > --Guido van Rossum (python.org/~guido)
>
>
>
>
> --
> -- Talin
>
>
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] importlib.find_loader
In Python3.3, I thought that every loaded module has a __loader__ attribute. Apparently this is not the case for a few builtin modules: Python 3.3.0 (v3.3.0:bd8afb90ebf2, Sep 29 2012, 10:57:17) [MSC v.1600 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import importlib >>> >>> def bug(): ... for name in list(sys.modules.keys()): ... try: ... importlib.find_loader(name) ... except Exception as details: ... print(name, type(details), details) ... >>> bug() _frozen_importlib 'module' object has no attribute '__loader__' builtins 'module' object has no attribute '__loader__' signal 'module' object has no attribute '__loader__' importlib._bootstrap 'module' object has no attribute '__loader__' >>> However, the importlib docs talk about a ValueError instead of the AttributeError that I see above: Find the loader for a module, optionally within the specified path. If the module is in sys.modules, then sys.modules[name].__loader__ > is returned (unless the loader would be None, in which case > ValueError is raised). Is this a bug? Thomas ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Anyone building Python --without-doc-strings?
We do that, of course, but compiling python without the doc strings removes those from all built-in modules as well. That's quite a lot of static data. K -Original Message- From: Victor Stinner [mailto:[email protected]] Sent: 27. janúar 2013 21:58 To: Kristján Valur Jónsson Cc: R. David Murray; [email protected] Subject: Re: [Python-Dev] Anyone building Python --without-doc-strings? Why don't you compile using python -OO and distribute only .pyo code? Victor 2013/1/27 Kristján Valur Jónsson : > We (CCP) are certainly compiling python without docstrings for our > embedded platforms (that include the PS3) Anyone using python as en engine to > be used by programs and not users will appreciate the deletion of unneeded > memory. > K > > -Original Message- > From: Python-Dev > [mailto:[email protected]] On Behalf > Of R. David Murray > Sent: 27. janúar 2013 00:38 > To: [email protected] > Subject: Re: [Python-Dev] Anyone building Python --without-doc-strings? > > On Sat, 26 Jan 2013 17:19:32 +0100, Antoine Pitrou > wrote: >> On Sat, 26 Jan 2013 17:03:59 +0100 >> Stefan Krah wrote: >> > Stefan Krah wrote: >> > > I'm not sure how accurate the output is for measuring these >> > > things, but according to ``ls'' and ``du'' the option is indeed quite >> > > worthless: >> > > >> > > ./configure CFLAGS="-Os -s" LDFLAGS="-s" && make >> > > 1.8M Jan 26 16:36 python >> > > ./configure --without-doc-strings CFLAGS="-Os -s" LDFLAGS="-s" && make >> > > 1.6M Jan 26 16:33 python >> > >> > The original contribution *was* in fact aiming for "10% smaller", see: >> > >> > http://docs.python.org/release/2.3/whatsnew/node20.html >> > >> > So apparently people thought it was useful. >> >> After a bit of digging, I found the following discussions: >> http://mail.python.org/pipermail/python-dev/2001-November/018444.html >> http://mail.python.org/pipermail/python-dev/2002-January/019392.html >> http://bugs.python.org/issue505375 >> >> Another reason for accepting the patch seemed to be that it >> introduced the Py_DOCSTR() macros, which were viewed as helpful for >> other reasons (some people talked about localizing docstrings). >> >> I would point out that if 200 KB is really a big win for someone, >> then Python (and especially Python 3) is probably not the best >> language for them. >> >> It is also ironic how the executable size went up since then (from >> 0.6 to more than 1.5 MB) :-) > > 200K can make a difference. It does on the QNX platform, for example, > where there is no virtual memory. It would be nice to reduce that > executable size, toobut I'm not volunteering to try (at least not > yet) :) > > --David > ___ > Python-Dev mailing list > [email protected] > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/kristjan%40ccpgames. > com > > > ___ > Python-Dev mailing list > [email protected] > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/victor.stinner%40gma > il.com ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython (2.7): - Issue #17086: Backport the patches from the 3.3 branch to cross-build
On Thu, 31 Jan 2013 23:52:27 +0100 (CET) matthias.klose wrote: > http://hg.python.org/cpython/rev/8ee6d96a1019 > changeset: 81859:8ee6d96a1019 > branch: 2.7 > parent: 81855:df9f8feb7444 > user:[email protected] > date:Thu Jan 31 23:52:03 2013 +0100 > summary: > - Issue #17086: Backport the patches from the 3.3 branch to cross-build > the package. You aren't supposed to add new features to bugfix branches. Did you have a specific reason to do this? Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Release or not release the GIL
Hi, While working on the implementation of the PEP 433, I saw different places where Python asks for operating resources without releasing the GIL. In my implementation, I released the GIL in the following places. Is it correct? - os.dup() - os.dup2() - os.pipe() - socket.socketpair() os.listdir() releases the GIL to call dup(), whereas os.dup() doesn't. Victor ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] importlib.find_loader
Yep, looks like a bug in the bootstrapping, failing to set __loader__ properly. However, I also think the current handling of the "no __loader__ attribute" case is a separate bug - since we generally aim to tolerate non-modules in sys.modules (albeit somewhat grudgingly), importlib should probably be using "getattr(mod, '__loader__', None)" rather than assuming the attribute will always be present. Cheers, Nick. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython (2.7): - Issue #17086: Backport the patches from the 3.3 branch to cross-build
On Fri, Feb 1, 2013 at 9:50 AM, Antoine Pitrou wrote: > On Thu, 31 Jan 2013 23:52:27 +0100 (CET) > matthias.klose wrote: >> http://hg.python.org/cpython/rev/8ee6d96a1019 >> changeset: 81859:8ee6d96a1019 >> branch: 2.7 >> parent: 81855:df9f8feb7444 >> user:[email protected] >> date:Thu Jan 31 23:52:03 2013 +0100 >> summary: >> - Issue #17086: Backport the patches from the 3.3 branch to cross-build >> the package. > > You aren't supposed to add new features to bugfix branches. Did you > have a specific reason to do this? One of the reasons for the long maintenance period on 2.7 is to keep it building as the underlying platforms change. With the rise of ARM systems, being able to reliably cross-build Python 2.7 for ARM from an x86_64 system is fairly important. That said, as a procedural point for build related new features in 2.7, I agree they should be proposed, with an explicit rationale, on python-dev before proceeding with the commit. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Release or not release the GIL
I'm guessing those system calls are just implemented by the kernel and cannot block for I/O, so it was deemed unnecessary to release the GIL around them. I don't mind changing that though, you can never know what happens when you make a system call. On Thu, Jan 31, 2013 at 4:07 PM, Victor Stinner wrote: > Hi, > > While working on the implementation of the PEP 433, I saw different > places where Python asks for operating resources without releasing the > GIL. > > In my implementation, I released the GIL in the following places. Is it > correct? > > - os.dup() > - os.dup2() > - os.pipe() > - socket.socketpair() > > os.listdir() releases the GIL to call dup(), whereas os.dup() doesn't. > > Victor > ___ > Python-Dev mailing list > [email protected] > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] cpython (2.7): #16128: clarify that instances of user-defined classes compare equal with
On 1 Feb 2013 13:22, "ezio.melotti" wrote: > > http://hg.python.org/cpython/rev/79a021beaf58 > changeset: 81861:79a021beaf58 > branch: 2.7 > parent: 81859:8ee6d96a1019 > user:Ezio Melotti > date:Fri Feb 01 05:18:44 2013 +0200 > summary: > #16128: clarify that instances of user-defined classes compare equal with themselves. > > files: > Doc/glossary.rst | 3 ++- > 1 files changed, 2 insertions(+), 1 deletions(-) > > > diff --git a/Doc/glossary.rst b/Doc/glossary.rst > --- a/Doc/glossary.rst > +++ b/Doc/glossary.rst > @@ -330,7 +330,8 @@ >All of Python's immutable built-in objects are hashable, while no mutable >containers (such as lists or dictionaries) are. Objects which are >instances of user-defined classes are hashable by default; they all > - compare unequal, and their hash value is their :func:`id`. > + compare unequal (except with themselves), and their hash value is their > + :func:`id`. The hash(x) == id(x) behaviour is a CPython implementation detail. It shouldn't be mentioned here. > > IDLE >An Integrated Development Environment for Python. IDLE is a basic editor > > -- > Repository URL: http://hg.python.org/cpython > > ___ > Python-checkins mailing list > [email protected] > http://mail.python.org/mailman/listinfo/python-checkins > ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] importlib.find_loader
Am 01.02.2013 01:42, schrieb Nick Coghlan: Yep, looks like a bug in the bootstrapping, failing to set __loader__ properly. It also has the effect that reload does not work: Type "help", "copyright", "credits" or "license" for more information. >>> import imp >>> import math >>> imp.reload(math) >>> import signal >>> imp.reload(signal) Traceback (most recent call last): File "", line 1, in File "C:\Python33-64\lib\imp.py", line 252, in reload return module.__loader__.load_module(name) AttributeError: 'module' object has no attribute '__loader__' >>> However, I also think the current handling of the "no __loader__ attribute" case is a separate bug - since we generally aim to tolerate non-modules in sys.modules (albeit somewhat grudgingly), importlib should probably be using "getattr(mod, '__loader__', None)" rather than assuming the attribute will always be present. Thomas ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython (merge 3.3 -> default): #16128: merge with 3.3.
On Fri, 1 Feb 2013 04:20:46 +0100 (CET) ezio.melotti wrote: > > diff --git a/Doc/glossary.rst b/Doc/glossary.rst > --- a/Doc/glossary.rst > +++ b/Doc/glossary.rst > @@ -320,7 +320,8 @@ >All of Python's immutable built-in objects are hashable, while no > mutable >containers (such as lists or dictionaries) are. Objects which are >instances of user-defined classes are hashable by default; they all > - compare unequal, and their hash value is their :func:`id`. > + compare unequal (except with themselves), and their hash value is their > + :func:`id`. Python 3.3.0+ (3.3:127abd16168a, Sep 29 2012, 18:46:33) [GCC 4.5.2] on linux Type "help", "copyright", "credits" or "license" for more information. >>> o = object() >>> hash(o) == id(o) False Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython (2.7): - Issue #17086: Backport the patches from the 3.3 branch to cross-build
On Fri, 1 Feb 2013 11:00:24 +1000 Nick Coghlan wrote: > On Fri, Feb 1, 2013 at 9:50 AM, Antoine Pitrou wrote: > > On Thu, 31 Jan 2013 23:52:27 +0100 (CET) > > matthias.klose wrote: > >> http://hg.python.org/cpython/rev/8ee6d96a1019 > >> changeset: 81859:8ee6d96a1019 > >> branch: 2.7 > >> parent: 81855:df9f8feb7444 > >> user:[email protected] > >> date:Thu Jan 31 23:52:03 2013 +0100 > >> summary: > >> - Issue #17086: Backport the patches from the 3.3 branch to cross-build > >> the package. > > > > You aren't supposed to add new features to bugfix branches. Did you > > have a specific reason to do this? > > One of the reasons for the long maintenance period on 2.7 is to keep > it building as the underlying platforms change. With the rise of ARM > systems, being able to reliably cross-build Python 2.7 for ARM from an > x86_64 system is fairly important. I would like to see a better argument for this. The rise of ARM systems is the rise of ARM systems powerful enough to build Python without cross-compiling (which is why we *now* have ARM buildbots). The very small ARM systems which need cross-compiling have existed for decades. > That said, as a procedural point for build related new features in > 2.7, I agree they should be proposed, with an explicit rationale, on > python-dev before proceeding with the commit. I think this huge changeset should be reverted. It's a complete failure in terms of procedure and following the rules. "Just because it can be useful" is not a good enough reason to violate our release model without even asking. Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
