Re: [Python-Dev] PEP 451 update
On Thu, Oct 24, 2013 at 2:05 AM, Eric Snow wrote: > I've had some offline discussion with Brett and Nick about PEP 451 > which has led to some meaningful clarifications in the PEP. In the > interest of pulling further discussions back onto this > (archived/public) list, here's an update of what we'd discussed and > where things are at. :) > > * path entry finders indicate that they found part of a possible > namespace package by returning a spec with no loader set (but with > submodule_search_locations set). Brett wanted some clarification on > this. > * The name/path signature and attributes of file-based finders in > importlib will no longer be changing. Brett had some suggestions on > the proposed change and it became clear that the the change was > actually pointless. > * I've asserted that there shouldn't be much difficulty in adjusting > pkgutil and other modules to work with ModuleSpec. > * Brett asked for clarification on whether the "load()" example from > the PEP would be realized implicitly by the import machinery or > explicitly as a method on ModuleSpec. This has bearing on the ability > of finders to return instances of ModuleSpec subclasses or even > ModuleSpec-like objects (a la duck typing). The answer is the it will > not be a method on ModuleSpec, so it is effectively just part of the > general import system implementation. Finders may return any object > that provides the attributes of ModuleSpec. I will be updating the > PEP to make these points clear. > > * Nick suggested writing a draft patch for the language reference > changes (the import page). Such a patch will be a pretty good > indicator of the impact of PEP 451 on the import system and should > highlight any design flaws in the API. This is on my to-do list > (hopefully by tomorrow). > Eric's got an initial patch for this up on http://bugs.python.org/issue18864. > * Nick also suggested moving all ModuleSpec methods to a separate > class that will simply make use of a separate, existing ModuleSpec > instance. This will help address several issues, particularly by > relaxing the constraints on what finders can return, but also by > avoiding the unnecessary exposure of the methods via every > module.__spec__. I plan on going with this, but currently am trying > out the change to see if there are any problems I've missed. Once I > feel good about it I'll update the PEP. > > That about sums up our discussions. I have a couple of outstanding > updates to the PEP to make when I get a chance, as well as putting up > a language reference patch for review. > After reading Eric's doc patch, I realized there is one change I want to make to the current semantics and that's not to backfill __package__ when set to None. Since import is now going to take over the job of setting __package__ (along with other attributes), this seems like a slight waste of effort. It also kills (or at least complicates) having a loader which does lazy loading since reading the attribute to see if it is None would trigger the load before leaving the import code, thus killing any postponed loading. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 451 update
On Fri, Oct 25, 2013 at 9:40 AM, Brett Cannon wrote: > After reading Eric's doc patch, I realized there is one change I want to > make to the current semantics and that's not to backfill __package__ when > set to None. Since import is now going to take over the job of setting > __package__ (along with other attributes), this seems like a slight waste of > effort. It also kills (or at least complicates) having a loader which does > lazy loading since reading the attribute to see if it is None would trigger > the load before leaving the import code, thus killing any postponed loading. Fine with me. I may have time today to make the outstanding updates to the PEP. -eric ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3 to be default in Fedora 22
On Fri, Oct 25, 2013 at 01:32:36PM +1000, Nick Coghlan wrote: > > On 25 Oct 2013 09:02, "Terry Reedy" wrote: > > > http://lwn.net/Articles/571528/ > > https://fedoraproject.org/wiki/Changes/Python_3_as_Default > > Note that unlike Arch, the Fedora devs currently plan to leave "/usr/bin/ > python" referring to Python 2 (see the "User Experience" part of the > proposal). > The tangible changes for this are just that we're hoping to only have python3, not python2 on our default LiveCD and cloud images. This has been a bit hard since many of our core packaging tools (and the large number of release engineering, package-maintainer, distro installer, etc scripts built on top of them) were written in python2. The F22 release is hoping to have a set of C libraries for those tools with both python3 and python2 bindings. That will hopefully allow us to port the user-visible tools (installer and things present on the selected images) to python3 for F22 while leaving the release-engineering and packager-oriented scripts until a later Fedora release. -Toshio pgpr8jz4t1Ec9.pgp Description: PGP signature ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Summary of Python tracker Issues
ACTIVITY SUMMARY (2013-10-18 - 2013-10-25) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open4197 (+13) closed 26965 (+93) total 31162 (+106) Open issues with patches: 1955 Issues opened (55) == #18777: Cannot compile _ssl.c using openssl > 1.0 http://bugs.python.org/issue18777 reopened by christian.heimes #18999: Support different contexts in multiprocessing http://bugs.python.org/issue18999 reopened by sbt #19286: error: can't copy '': doesn't exist or not a regular http://bugs.python.org/issue19286 opened by jason.coombs #19290: Clarify compile and eval interaction. http://bugs.python.org/issue19290 opened by terry.reedy #19291: Add docs for asyncio package (Tulip, PEP 3156) http://bugs.python.org/issue19291 opened by gvanrossum #19292: Make SSLContext.set_default_verify_paths() work on Windows http://bugs.python.org/issue19292 opened by gvanrossum #19296: Compiler warning when compiling dbm module http://bugs.python.org/issue19296 opened by vajrasky #19298: Use __attribute__(cleanup ...) to detect refleaks http://bugs.python.org/issue19298 opened by brett.cannon #19302: Add support for AIX pollset efficient event I/O polling http://bugs.python.org/issue19302 opened by David.Edelsohn #19308: Tools/gdb/libpython.py does not support GDB linked against Pyt http://bugs.python.org/issue19308 opened by dcoles #19316: devguide: compiler - wording http://bugs.python.org/issue19316 opened by numerodix #19317: ctypes.util.find_library should examine binary's RPATH on Sola http://bugs.python.org/issue19317 opened by automatthias #19320: Tkinter tests ran with wantobjects is false http://bugs.python.org/issue19320 opened by serhiy.storchaka #19325: _osx_support imports many modules http://bugs.python.org/issue19325 opened by christian.heimes #19328: Improve PBKDF2 documentation http://bugs.python.org/issue19328 opened by christian.heimes #19329: Faster compiling of charset regexpes http://bugs.python.org/issue19329 opened by serhiy.storchaka #19330: Use public classes for contextlib.suppress and redirect_stdout http://bugs.python.org/issue19330 opened by ncoghlan #19331: Revise PEP 8 recommendation for class names http://bugs.python.org/issue19331 opened by barry #19332: Guard against changing dict during iteration http://bugs.python.org/issue19332 opened by serhiy.storchaka #19333: distutils.util.grok_environment_error loses the error message http://bugs.python.org/issue19333 opened by mgedmin #19334: test_asyncio hanging for 1 hour (non-AIX version) http://bugs.python.org/issue19334 opened by gvanrossum #19335: codeop misclassifies incomplete code with 'nonlocal' http://bugs.python.org/issue19335 opened by Rosuav #19336: No API to get events from epoll without allocating a list http://bugs.python.org/issue19336 opened by alex #19338: multiprocessing: sys.exit() from a child with a non-int exit c http://bugs.python.org/issue19338 opened by brodie #19339: telnetlib: time.monotonic() should be used instead of time.tim http://bugs.python.org/issue19339 opened by haypo #19342: Improve grp module docstrings http://bugs.python.org/issue19342 opened by mgedmin #19343: Expose FreeBSD-specific APIs in resource module http://bugs.python.org/issue19343 opened by koobs #19346: Build fails when there are no shared extensions to be built http://bugs.python.org/issue19346 opened by alanh #19347: PEP 453 implementation tracking issue http://bugs.python.org/issue19347 opened by ncoghlan #19348: Building _testcapimodule as a builtin results in compile error http://bugs.python.org/issue19348 opened by alanh #19349: Not so correct exception message when running venv http://bugs.python.org/issue19349 opened by vajrasky #19351: python msi installers - silent mode http://bugs.python.org/issue19351 opened by Jason.Bray #19353: on RHEL6 simple build fails with test_linux_constants (test.te http://bugs.python.org/issue19353 opened by mcepl #19354: test_format fails on RHEL-6 http://bugs.python.org/issue19354 opened by skrah #19355: Initial modernization of OpenWatcom support http://bugs.python.org/issue19355 opened by Jeffrey.Armstrong #19358: Integrate "Argument Clinic" into CPython build http://bugs.python.org/issue19358 opened by larry #19359: reversed() does not work with weakref.proxy of sequences http://bugs.python.org/issue19359 opened by abingham #19361: Specialize exceptions thrown by JSON parser http://bugs.python.org/issue19361 opened by musically_ut #19362: Documentation for len() fails to mention that it works on sets http://bugs.python.org/issue19362 opened by Gareth.Rees #19363: Python 2.7's future_builtins.map is not compatible with Python http://bugs.python.org/issue19363 opened by Gareth.Rees #19367: IDLE wont open http://bugs.python.org/issue19367 opened by MrE.Merry #19371: tim
Re: [Python-Dev] PEP 451 update
On Fri, Oct 25, 2013 at 11:44 AM, Eric Snow wrote: > On Fri, Oct 25, 2013 at 9:40 AM, Brett Cannon wrote: > > After reading Eric's doc patch, I realized there is one change I want to > > make to the current semantics and that's not to backfill __package__ when > > set to None. Since import is now going to take over the job of setting > > __package__ (along with other attributes), this seems like a slight > waste of > > effort. It also kills (or at least complicates) having a loader which > does > > lazy loading since reading the attribute to see if it is None would > trigger > > the load before leaving the import code, thus killing any postponed > loading. > > Fine with me. I may have time today to make the outstanding updates to > the PEP. > I should also mention this applies to __loader__ as well. Basically everything from http://hg.python.org/cpython/file/eb1edc9e3722/Lib/importlib/_bootstrap.py#l1575on down. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 451 update
I've not really had time to review this PEP yet, but from skimming discussion to date, the only thing I'm still worried about is whether this will break lazy import schemes that use a module subclass that hooks __getattribute__ and calls reload() in order to perform what's actually an *initial* load. IOW, does anything in this proposal rely on a module object having *any* attributes besides __name__ set at reload() time? That is, is there an assumption that a module being reloaded has 1. Been loaded, and 2. Is being reloaded via the same location, __loader__, etc. as before? At least through all 2.x, reload() just uses module.__name__ to restart the module find-and-load process, and does not assume that __loader__ is valid in advance. (Also, if this has changed in recent Python versions independent of this PEP, it's a backwards-compatibility break that should be documented somewhere.) On Thu, Oct 24, 2013 at 2:05 AM, Eric Snow wrote: > I've had some offline discussion with Brett and Nick about PEP 451 > which has led to some meaningful clarifications in the PEP. In the > interest of pulling further discussions back onto this > (archived/public) list, here's an update of what we'd discussed and > where things are at. :) > > * path entry finders indicate that they found part of a possible > namespace package by returning a spec with no loader set (but with > submodule_search_locations set). Brett wanted some clarification on > this. > * The name/path signature and attributes of file-based finders in > importlib will no longer be changing. Brett had some suggestions on > the proposed change and it became clear that the the change was > actually pointless. > * I've asserted that there shouldn't be much difficulty in adjusting > pkgutil and other modules to work with ModuleSpec. > * Brett asked for clarification on whether the "load()" example from > the PEP would be realized implicitly by the import machinery or > explicitly as a method on ModuleSpec. This has bearing on the ability > of finders to return instances of ModuleSpec subclasses or even > ModuleSpec-like objects (a la duck typing). The answer is the it will > not be a method on ModuleSpec, so it is effectively just part of the > general import system implementation. Finders may return any object > that provides the attributes of ModuleSpec. I will be updating the > PEP to make these points clear. > > * Nick suggested writing a draft patch for the language reference > changes (the import page). Such a patch will be a pretty good > indicator of the impact of PEP 451 on the import system and should > highlight any design flaws in the API. This is on my to-do list > (hopefully by tomorrow). > * Nick also suggested moving all ModuleSpec methods to a separate > class that will simply make use of a separate, existing ModuleSpec > instance. This will help address several issues, particularly by > relaxing the constraints on what finders can return, but also by > avoiding the unnecessary exposure of the methods via every > module.__spec__. I plan on going with this, but currently am trying > out the change to see if there are any problems I've missed. Once I > feel good about it I'll update the PEP. > > That about sums up our discussions. I have a couple of outstanding > updates to the PEP to make when I get a chance, as well as putting up > a language reference patch for review. > > -eric > ___ > Python-Dev mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/pje%40telecommunity.com ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 451 update
On Fri, Oct 25, 2013 at 12:24 PM, PJ Eby wrote: > I've not really had time to review this PEP yet, but from skimming > discussion to date, the only thing I'm still worried about is whether > this will break lazy import schemes that use a module subclass that > hooks __getattribute__ and calls reload() in order to perform what's > actually an *initial* load. > Depends on how you implement it probably. I have a scheme in my head using __getattribute__ and loaders which will work with this PEP if __package__ and __loader__ are not explicitly checked after execute_module() is called, so lazy imports are not going to be fundamentally broken. Since Python 3.3 lazy loaders relying on __getattribute__ have been broken due to those __package__/__loader__ checks so this is actually going to be an improvement. > > IOW, does anything in this proposal rely on a module object having > *any* attributes besides __name__ set at reload() time? As it stands in Python 3.3 it requires __name__ and __loader__ be defined: http://hg.python.org/cpython/file/e2c3f638c3d0/Lib/importlib/__init__.py#l101 > That is, is > there an assumption that a module being reloaded has > > 1. Been loaded, and > 2. Is being reloaded via the same location, __loader__, etc. as before? > Just 2 in terms of __loader__ since loaders have to work with or without the module already existing. > > At least through all 2.x, reload() just uses module.__name__ to > restart the module find-and-load process, and does not assume that > __loader__ is valid in advance. > That doesn't make much sense in a post-importlib world where import makes sure that __loader__ is set (which it has since Python 3.3). Otherwise you are asking for not just a reload but a re-find as well. As for the PEP, it will probably keep the name/loader requirement but shift it to having to be set on the spec object at __spec__. I guess you could make the argument a reload should include re-running the finder step as well, but I don't think it's worth the code tweaking to make it happen when the finder/loader steps are divided as they are. > > (Also, if this has changed in recent Python versions independent of > this PEP, it's a backwards-compatibility break that should be > documented somewhere.) > http://bugs.python.org/issue19392 -Brett > > > On Thu, Oct 24, 2013 at 2:05 AM, Eric Snow > wrote: > > I've had some offline discussion with Brett and Nick about PEP 451 > > which has led to some meaningful clarifications in the PEP. In the > > interest of pulling further discussions back onto this > > (archived/public) list, here's an update of what we'd discussed and > > where things are at. :) > > > > * path entry finders indicate that they found part of a possible > > namespace package by returning a spec with no loader set (but with > > submodule_search_locations set). Brett wanted some clarification on > > this. > > * The name/path signature and attributes of file-based finders in > > importlib will no longer be changing. Brett had some suggestions on > > the proposed change and it became clear that the the change was > > actually pointless. > > * I've asserted that there shouldn't be much difficulty in adjusting > > pkgutil and other modules to work with ModuleSpec. > > * Brett asked for clarification on whether the "load()" example from > > the PEP would be realized implicitly by the import machinery or > > explicitly as a method on ModuleSpec. This has bearing on the ability > > of finders to return instances of ModuleSpec subclasses or even > > ModuleSpec-like objects (a la duck typing). The answer is the it will > > not be a method on ModuleSpec, so it is effectively just part of the > > general import system implementation. Finders may return any object > > that provides the attributes of ModuleSpec. I will be updating the > > PEP to make these points clear. > > > > * Nick suggested writing a draft patch for the language reference > > changes (the import page). Such a patch will be a pretty good > > indicator of the impact of PEP 451 on the import system and should > > highlight any design flaws in the API. This is on my to-do list > > (hopefully by tomorrow). > > * Nick also suggested moving all ModuleSpec methods to a separate > > class that will simply make use of a separate, existing ModuleSpec > > instance. This will help address several issues, particularly by > > relaxing the constraints on what finders can return, but also by > > avoiding the unnecessary exposure of the methods via every > > module.__spec__. I plan on going with this, but currently am trying > > out the change to see if there are any problems I've missed. Once I > > feel good about it I'll update the PEP. > > > > That about sums up our discussions. I have a couple of outstanding > > updates to the PEP to make when I get a chance, as well as putting up > > a language reference patch for review. > > > > -eric > > ___ > > Python-Dev maili
Re: [Python-Dev] PEP 451 update
On Fri, Oct 25, 2013 at 10:24 AM, PJ Eby wrote: > I've not really had time to review this PEP yet, but from skimming > discussion to date, the only thing I'm still worried about is whether > this will break lazy import schemes that use a module subclass that > hooks __getattribute__ and calls reload() in order to perform what's > actually an *initial* load. > > IOW, does anything in this proposal rely on a module object having > *any* attributes besides __name__ set at reload() time? That is, is > there an assumption that a module being reloaded has > > 1. Been loaded, and > 2. Is being reloaded via the same location, __loader__, etc. as before? > > At least through all 2.x, reload() just uses module.__name__ to > restart the module find-and-load process, and does not assume that > __loader__ is valid in advance. > > (Also, if this has changed in recent Python versions independent of > this PEP, it's a backwards-compatibility break that should be > documented somewhere.) Yeah, this actually changed in 3.3 IIRC, where reload now calls module.__loader__.load_module(module.__name__). I'll double-check. PEP 451 simply changes that to be (basically) module.__loader__.exec_module(module). -eric ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 451 update
On Fri, Oct 25, 2013 at 1:15 PM, Brett Cannon wrote: > > On Fri, Oct 25, 2013 at 12:24 PM, PJ Eby wrote: >> At least through all 2.x, reload() just uses module.__name__ to >> restart the module find-and-load process, and does not assume that >> __loader__ is valid in advance. > > > That doesn't make much sense in a post-importlib world where import makes > sure that __loader__ is set (which it has since Python 3.3). Otherwise you > are asking for not just a reload but a re-find as well. That's a feature, not a bug. A reload() after changing sys.path *should* take into account the change, not to mention any changes to meta_path, path hooks, etc. (And it's how reload() worked before importlib.) I suppose it's not really documented all that well, but way way back in the 2.3 timeframe I asked for a tweak to PEP 302 to make sure that reload() (in the re-find sense) would work properly with PEP 302 loaders like zipimport -- some of the language still in the PEP is there specifically to support this use case. (Specifically, the bit that says loaders *must* use the existing module object in sys.modules if there's one there already, so that reload() will work. It was actually in part to ensure that reload() would work in the case of a re-find.) It appears that since then, the PEP has been changed in a way that invalidates part of the purpose of the prior change; I guess I missed the discussion of that change last year. :-( ISTM there should've been such a discussion, since IIRC importlib wasn't supposed to change any Python semantics, and this is a non-trivial change to the semantics of reload() even in cases that aren't doing lazy imports or other such munging. reload() used to take sys.* and __path__ changes into account, and IMO should continue to do so. If this is an intentional change in reload() semantics, other Python implementations need to know about this too! (That being said, I'm not saying I shouldn't or couldn't have tested this in 3.3 and found out about it that way. And the existence of issue18698 suggests that nobody's relying yet on even the *fundamental* semantics of PEP 302 reload() working properly in 3.3, since that was an even bigger change that nobody spotted till a couple of months ago.) ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 451 update
On Fri, Oct 25, 2013 at 2:10 PM, PJ Eby wrote: > On Fri, Oct 25, 2013 at 1:15 PM, Brett Cannon wrote: > > > > On Fri, Oct 25, 2013 at 12:24 PM, PJ Eby wrote: > >> At least through all 2.x, reload() just uses module.__name__ to > >> restart the module find-and-load process, and does not assume that > >> __loader__ is valid in advance. > > > > > > That doesn't make much sense in a post-importlib world where import makes > > sure that __loader__ is set (which it has since Python 3.3). Otherwise > you > > are asking for not just a reload but a re-find as well. > > That's a feature, not a bug. A reload() after changing sys.path > *should* take into account the change, not to mention any changes to > meta_path, path hooks, etc. (And it's how reload() worked before > importlib.) > Fair enough, but in my mind that obviously doesn't really click for what I view as a reload in an importlib world where secret import code no longer exists. When I think re-load I think "load again", not "find the module again and execute a load with a possibly new loader". It's not worth changing in Python 3.3, but if you want to propose to redefine importlib.reload() to use importlib.find_spec() to reset the __spec__ on a module then I'm not going to argue against it (but I'm not going to fight for it either). And in a PEP 451 world it should be dead-simple to make this work the way you want in your own code even if this doesn't go the way you want:: spec = importlib.find_spec(name) module.__spec__ = spec importlib.reload(module) # Which in itself is essentially init_module_attrs(spec, module); spec.loader.exec_module(module) Heck, you can do this in Python 3.3 right now:: loader = importlib.find_loader(name) module = sys.modules[name] module.__loader__ = loader importlib.reload(module) > I suppose it's not really documented all that well, Nope =) > but way way back > in the 2.3 timeframe I asked for a tweak to PEP 302 to make sure that > reload() (in the re-find sense) would work properly with PEP 302 > loaders like zipimport -- some of the language still in the PEP is > there specifically to support this use case. (Specifically, the bit > that says loaders *must* use the existing module object in sys.modules > if there's one there already, so that reload() will work. It was > actually in part to ensure that reload() would work in the case of a > re-find.) > Ah, okay. That is not explicit in the PEP beyond coming off a total nuisance in order to support reloading by the loader, not an explicit finder + loader use-case. > > It appears that since then, the PEP has been changed in a way that > invalidates part of the purpose of the prior change; I guess I missed > the discussion of that change last year. :-( > > ISTM there should've been such a discussion, since IIRC importlib > wasn't supposed to change any Python semantics, and this is a > non-trivial change to the semantics of reload() even in cases that > aren't doing lazy imports or other such munging. Well, not change them to the best of its abilities. I'm sure there are edge cases that I couldn't match properly since I could only go by the test suite and bug reports filed since Python 3.1. > reload() used to > take sys.* and __path__ changes into account, and IMO should continue > to do so. Unfortunately this was never explicitly documented (at least that I noticed) nor was there a test in the stdlib to enforce compliance with it, else this never would have happened. > If this is an intentional change in reload() semantics, > other Python implementations need to know about this too! > Not really. As of Python 3.3 the semantic shift included re-implementing the function in pure Python so they pick up the change for free; remember the function is not a builtin anymore to somewhat de-emphasize its use since it's kind of a hack and really only used typically at the interpreter prompt as an easy part of a load-edit-reload dev cycle, not some fundamental operation. > > (That being said, I'm not saying I shouldn't or couldn't have tested > this in 3.3 and found out about it that way. And the existence of > issue18698 suggests that nobody's relying yet on even the > *fundamental* semantics of PEP 302 reload() working properly in 3.3, > since that was an even bigger change that nobody spotted till a couple > of months ago.) > Yep, never got a bug report on this (although there have been reports of typos in the docs as well as not returning what's in sys.modules so it is being used by people). ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 451 update
On Fri, Oct 25, 2013 at 3:15 PM, Brett Cannon wrote: > On Fri, Oct 25, 2013 at 2:10 PM, PJ Eby wrote: >> On Fri, Oct 25, 2013 at 1:15 PM, Brett Cannon wrote: >> > On Fri, Oct 25, 2013 at 12:24 PM, PJ Eby wrote: >> >> At least through all 2.x, reload() just uses module.__name__ to >> >> restart the module find-and-load process, and does not assume that >> >> __loader__ is valid in advance. >> > >> > >> > That doesn't make much sense in a post-importlib world where import >> > makes >> > sure that __loader__ is set (which it has since Python 3.3). Otherwise >> > you >> > are asking for not just a reload but a re-find as well. >> >> That's a feature, not a bug. A reload() after changing sys.path >> *should* take into account the change, not to mention any changes to >> meta_path, path hooks, etc. (And it's how reload() worked before >> importlib.) > > > Fair enough, but in my mind that obviously doesn't really click for what I > view as a reload in an importlib world where secret import code no longer > exists. When I think re-load I think "load again", not "find the module > again and execute a load with a possibly new loader". Sure, and the reference manual is rather vague on this point. However, I would guess that at least some web frameworks with automatic reload support are going to barf on this change in at least some edge cases. (OTOH, it's unlikely the bugs will ever be reported, because the problem will mysteriously go away once the process is restarted, probably never to occur again.) Mostly, this just seems like an ugly wart -- Python should be dynamic by default, and that includes reloading. While the import machinery has lots of ugly caching under the hood, a user-level function like reload() should not require you to do the equivalent of saying, "no, really... I want you to *really* reload, not just pull in whatever exists where you found it last time, while ignoring whether I switched from module to package or vice versa, or just fixed my sys.path so I can load the right version of the module." It is a really tiny thing in the overall scheme of things, because reload() is not used all that often, but it's still a thing. If this isn't treated as a bug, then the docs for reload() at least need to include a forward-supported workaround so you can say "no, really... *really* reload" in an approved fashion. (ISTM that any production code out there that currently uses reload() would want to perform the "really reload" incantation in order to avoid the edge cases, even if they haven't actually run into any of them yet.) > And in a PEP 451 world it should be dead-simple to make this work the way > you want in your own code even if this doesn't go the way you want:: > > spec = importlib.find_spec(name) > module.__spec__ = spec > importlib.reload(module) # Which in itself is essentially > init_module_attrs(spec, module); spec.loader.exec_module(module) > > Heck, you can do this in Python 3.3 right now:: > > loader = importlib.find_loader(name) > module = sys.modules[name] > module.__loader__ = loader > importlib.reload(module) And will that later version still work correctly in a PEP 451 world, or will you have to detect which world you live in before waving this particular dead chicken? ;-) > Ah, okay. That is not explicit in the PEP beyond coming off a total nuisance > in order to support reloading by the loader, not an explicit finder + loader > use-case. Yeah, it actually was to ensure that you could reload a module using a different loader than the one that originally loaded it, e.g. due to a change in path hooks, etc. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Merge codeaccess.txt from wesnoth-contribcommunity project in Google Code Hosting
Here's the codeaccess.txt file. https://wesnoth-contribcommunity.googlecode.com/svn/codeaccess.txt All committer names have (No): Python; check the sixth line. Merging the names of committers in the sixth line to the Misc/ACKS file might be good. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Merge codeaccess.txt from wesnoth-contribcommunity project in Google Code Hosting
On 10/25/2013 6:55 PM, Tae Wong wrote: Here's the codeaccess.txt file. https://wesnoth-contribcommunity.googlecode.com/svn/codeaccess.txt This seems to be an old copy of Misc/ACKS except that non-ascii chars are not displayed correctly as a result of some encoding snafu. All committer names have (No): Python; check the sixth line. Merging the names of committers in the sixth line to the Misc/ACKS file might be good. Tae Wong's followup on python-list is even less comprehensible. I think we can ignore the suggestion. -- Terry Jan Reedy ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Merge codeaccess.txt from wesnoth-contribcommunity project in Google Code Hosting
In article , Terry Reedy wrote: > On 10/25/2013 6:55 PM, Tae Wong wrote: > > Here's the codeaccess.txt file. > > https://wesnoth-contribcommunity.googlecode.com/svn/codeaccess.txt > Tae Wong's followup on python-list is even less comprehensible. > I think we can ignore the suggestion. He has a long reputation for spamming mailing lists. Please ignore him. -- Ned Deily, [email protected] ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Merge codeaccess.txt from wesnoth-contribcommunity project in Google Code Hosting
You wrote: > https://wesnoth-contribcommunity.googlecode.com/svn/codeaccess.txt You can view this file as all non-ASCII characters are rendered correctly if the file is encoded in UTF-8. And you are not subscribed to this mailing list! -- Tae Wong ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Merge codeaccess.txt from wesnoth-contribcommunity project in Google Code Hosting
Somebody please block him. --Guido van Rossum (sent from Android phone) On Oct 25, 2013 5:53 PM, "Ned Deily" wrote: > In article , Terry Reedy > wrote: > > On 10/25/2013 6:55 PM, Tae Wong wrote: > > > Here's the codeaccess.txt file. > > > https://wesnoth-contribcommunity.googlecode.com/svn/codeaccess.txt > > Tae Wong's followup on python-list is even less comprehensible. > > I think we can ignore the suggestion. > > He has a long reputation for spamming mailing lists. Please ignore him. > > -- > Ned Deily, > [email protected] > > ___ > Python-Dev mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/guido%40python.org > ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Merge codeaccess.txt from wesnoth-contribcommunity project in Google Code Hosting
The names that are included in the https://wesnoth-contribcommunity.googlecode.com/svn/codeaccess.txt file but were missing on the Misc/ACKS file are: Jeff Allen Jim Baker John Benediktsson Shashank Bharadwaj Martin Blais Carl Friedrich Bolz George Boutsioukis Josiah Carlson Ron DuPlain Phillip Eby Maciej Fijalkowski Erik Forsberg Alex Gaynor Alex Gronholm Jeff Hardy Doug Hellmann Daniel Holth Oti Humbel Josh Juneau Alan Kennedy Anselm Kruis Patrick Maupin Wayne Meissner Chris Monson Dirkjan Ochtman Travis Oliphant Runar Petursson Rodrigo Bernardo Pimentel Allison Randal Mateusz Rukowicz Robert Schuppenies Stefan Seefeld Jeff Senn Ask Solem Donald Stufft Talin Michael Twomey Jeroen Ruigrok van der Werven Jeffrey Yasskin Some names in Misc/ACKS are wrongly encoded in ASCII. Michal Bozon as Michal Božoň Richard Jones as Richard W.M. Jones R. M. Oudkerk as Richard M. Oudkerk Giampaolo Rodola as Giampaolo Rodolà Victor Terrón as Víctor Terrón Artur Zaprzala as Artur Zaprzała -- Tae Wong ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] OrderedDict.values() behavior for modified instance
Hello,
The documentation says the following about modifying a dict while
iterating through its view:
| Iterating views while adding or deleting entries in the dictionary may
| raise a RuntimeError or fail to iterate over all entries.
(http://docs.python.org/3/library/stdtypes.html#dict-views)
The OrderedDict documentation doesn't have anything to say on the
subject. In practice, its implementation actually mostly correponds to
naive expectations:
>>> from collections import OrderedDict
>>> d = OrderedDict()
>>> for i in range(5):
...d[i] = i
...
>>> i = iter(d.values())
>>> next(i)
0
>>> del d[0]
>>> next(i)
1
>>> del d[2]
>>> next(i)
3
>>> d.move_to_end(1)
>>> next(i)
4
>>> next(i)
1
>>>
etc.
However, there is one case that results in a rather confusing error:
>>> a = OrderedDict()
>>> a[0] = 0
>>> a[1] = 1
>>> a[2] = 2
>>> i = iter(a.values())
>>> next(i)
0
>>> del a[0]
>>> del a[1]
>>> next(i)
Traceback (most recent call last):
File "", line 1, in
File "/usr/lib/python3.3/collections/abc.py", line 495, in __iter__
yield self._mapping[key]
KeyError: 1
What happens here is that a[0] is returned from the linked list, but it
still contains links to a[1]. When a[1] is deleted, the links of its
predecessor and successor are updated, but a[0] still points to
a[1]. The OrderedList then hands a non-existing key to the values()
implementation in collections.abc.
The result is not only mightily confusing, but it is also not covered by
the documentation (KeyError isn't a RuntimeError).
I would very much like to see this fixed, but I'm not sure if there's a
good way to do that.
I see the following options:
(a) When deleting an element from an OrderedList, update the pointers in
the corresponding linked list element to the end of the list. Then
removing the currently "active" element during the iteration would
automatically end the iteration.
For that, we'd just have to add two lines to __delitem__:
def __delitem__(self, key, dict_delitem=dict.__delitem__):
dict_delitem(self, key)
link = self.__map.pop(key)
link_prev = link.prev
link_next = link.next
link_prev.next = link_next
link_next.prev = link_prev
link.prev = root # new
link.next = root # new
The new behavior is explicitly covered by the documentation
(changing the dict may result in incomplete iteration), but it's a
change to what happened before.
(b) When iterating through an OrderedDict, check that the next element
is actually still in the hash, i.e. change
def __iter__(self):
root = self.__root
curr = root.next
while curr is not root:
yield curr.key
curr = curr.next
to
def __iter__(self):
root = self.__root
curr = root.next
while curr is not root and curr.key in self:
yield curr.key
curr = curr.next
that would terminate the iteration only in the special case, but
incur an extra dict lookup at every iteration.
Alternatively, one could try very hard to not stop the iteration:
while curr is not root:
yield curr.key
while curr is not root:
curr = curr.next
if curr.key in self:
break
(c) Catch the KeyError in values(), and re-raise the proper exception in
class ValuesView:
def __iter__(self):
for key in self._mapping:
try:
yield self._mapping[key]
except KeyError:
raise RuntimeError("underlying Mapping instance modified during
iteration")
I consider this a bit ugly, because it may mask other problems in a
custom Mapping implementation (that may raise a KeyError because of
a bug in the Mapping implementation itself).
(d) Extend the OrderedDict documentation to explicity document this
case.
This has the drawback that it would probably be nicer to just have
the behavior be consistent and correspond to intuitive expectations.
Would any of those fixes be acceptable? Or is there an even better option?
Best,
Nikolaus
--
Encrypted emails preferred.
PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C
»Time flies like an arrow, fruit flies like a Banana.«
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
