Re: [Python-Dev] Merge codeaccess.txt from wesnoth-contribcommunity project in Google Code Hosting
Your suggestion needs to be accepted in the Misc/ACKS file. -- Tae Wong ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Block module import during Python finalization?
Hi, While working on a fix, I got an assertion error during Python finalization because Python tried to import the "io" module whereas the module was just unloaded. Python tried to import the io module to display a warning and display the Python line where the warning was emitted. See the following issue for detail: http://bugs.python.org/issue19421 I propose to block importing new modules (or import again unloaded modules) during Python finalization to avoid such tricky bugs. What do you think? Does anyone rely on the ability of importing Python modules at exit? :-) At least, the whole test suite pass with my import_finalization.patch which block import during Python finalization. Victor ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 455: TransformDict
2013/10/4 Raymond Hettinger : >> On Sep 22, 2013, at 6:16 PM, Ethan Furman wrote: >> > Are we close to asking for pronouncement? > > When you're ready, let me know. The deadline for new features is in less than 1 month, so what is the status of the PEP 455 (TransformDict)? Victor ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Block module import during Python finalization?
2013/10/28 Victor Stinner : > Hi, > > While working on a fix, I got an assertion error during Python > finalization because Python tried to import the "io" module whereas > the module was just unloaded. Python tried to import the io module to > display a warning and display the Python line where the warning was > emitted. See the following issue for detail: > http://bugs.python.org/issue19421 > > I propose to block importing new modules (or import again unloaded > modules) during Python finalization to avoid such tricky bugs. > > What do you think? Though the practice of importing during finalization seems dubious, baning it may be a compatibility problem. -- Regards, Benjamin ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Block module import during Python finalization?
On Mon, 28 Oct 2013 16:13:36 -0400 Benjamin Peterson wrote: > 2013/10/28 Victor Stinner : > > Hi, > > > > While working on a fix, I got an assertion error during Python > > finalization because Python tried to import the "io" module whereas > > the module was just unloaded. Python tried to import the io module to > > display a warning and display the Python line where the warning was > > emitted. See the following issue for detail: > > http://bugs.python.org/issue19421 > > > > I propose to block importing new modules (or import again unloaded > > modules) during Python finalization to avoid such tricky bugs. > > > > What do you think? > > Though the practice of importing during finalization seems dubious, > baning it may be a compatibility problem. It may just be because some finalizer is calling an API that uses lazy imports. Honestly, I don't event think this issue is a bug. Importing modules at shutdown isn't worse than creating functions or classes at shutdown. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Block module import during Python finalization?
On Mon, Oct 28, 2013 at 12:45 PM, Victor Stinner wrote: > I propose to block importing new modules (or import again unloaded > modules) during Python finalization to avoid such tricky bugs. Alternatively, we could set sys.modules[name] to None for each builtin/frozen module when they get finalized. This will prevent importing those modules afterward. -eric ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 454 (tracemalloc) disable ==> clear?
reset() function: Clear traces of memory blocks allocated by Python. Does this do anything besides clear? If not, why not just re-use the 'clear' name from dicts? disable() function: Stop tracing Python memory allocations and clear traces of memory blocks allocated by Python. I would disable to stop tracing, but I would not expect it to clear out the traces it had already captured. If it has to do that, please put in some sample code showing how to save the current traces before disabling. -jJ ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 451 update
On Sat, Oct 26, 2013 at 11:03 PM, Nick Coghlan wrote:
> I don't think we can postpone it to later, as we need to be clear on:
>
> 1. Does reload use __name__ or __spec__.name when both are available?
> 2. Does __name__ get restored to its original value if reloading via
> __spec__.name?
> 3. Do other import related attributes get restored to their original values?
> 4. Does create_module get called if the loader has an exec_module method?
> 5. Does the loader have access to the previous spec when reloading a module?
>
> My answers to these questions (note that my answer to 2 differs from
> what I had in my initial sketch):
>
> 1. __spec__.name
> 2. Yes, __name__ is updated to match __spec__.name, *expect* if it is
> currently "__main__"
> 3. Yes, other import related attributes are restored to their baseline values
> 4. No, create_module is never called when reloading
I agree on all of these. I'm adding a "How reloading will work"
section to the PEP.
> 5. Currently no, but I'm thinking that may be worth changing (more on
> that below)
>
> The reload() implementation in my message is actually based on the
> current importlib.reload implementation. The only PEP 451 related
> changes were:
>
> - using __spec__.name (if available) instead of __name__
> - checking all parent modules exist rather than just the immediate parent
> module
> - calling import.find_spec() rather than using the __loader__ attribute
> directly
> - being explicit that __name__ is left at the value it had prior to the reload
> - handling the namespace package, exec_module and no exec_module cases
>
> I also added an explicit check that the module was re-used properly,
> but I agree *that* change is out of scope for the PEP and should be
> considered as a separate question.
>
> Now, regarding the signature of exec_module(): I'm back to believing
> that loaders should receive a clear indication that a reload is taking
> place. Legacy loaders have to figure that out for themselves (by
> seeing that the module already exists in sys.modules), but we can do
> better for the new API by making the exec_module signature look like:
>
> def exec_module(self, module, previous_spec=None):
> # module is as per the current PEP 451 text
> # previous_spec would be set *only* in the reload() case
> # loaders that don't care still need to accept it, but can
> just ignore it
> ...
I'd rather give loaders another optional method:
supports_reload(name). Complicating the spec methods signatures (to
support passing the old spec through) just for the sake of reload
seems less than optimal. I don't see any benefit to exposing the old
spec to loader.exec_module().
>
> So, with those revisions, the reload() semantics would be defined as:
>
> def reload(module):
> # Get the name to reload from the spec if it is available,
> # otherwise use __name__ directly
> current_spec = getattr(module, "__spec__")
> current_name = module.__name__
> name_to_reload = current_name if current_spec is None else
> current_spec.name
>
> # Check this module is properly imported before trying to reload it
> loaded_module = sys.modules.get(name_to_reload)
> if loaded_module is not module:
> msg = "module {} not in sys.modules"
> raise ImportError(msg.format(name_to_reload), name=name_to_reload)
> parent_name = name_to_reload.rpartition('.')[0]
> while parent_name:
> if parent_name not in sys.modules:
> msg = "parent {!r} not in sys.modules"
> raise ImportError(msg.format(parent_name), name=parent_name)
> parent_name = parent_name.rpartition('.')[0]
>
> # Check for a modified spec (e.g. if the import hooks have changed)
> updated_spec = importlib.find_spec(name_to_reload)
>
> # The import-related module attributes get updated here
> _init_module_attrs(loaded_module, updated_spec)
> if current_name == "__main__":
> loaded_module.__name__ = "__main__"
>
> # Now re-execute the module
> loader = updated_spec.loader
> if loader is None:
> # Namespace package, already reinitialised above
> return loaded_module
> elif hasattr(loader, 'exec_module'):
> loader.exec_module(module, current_spec)
> else:
> loader.load_module(name_to_reload)
>
> # Allow for the module to be replaced in sys.modules
> # (even though that likely won't work very well)
> return sys.modules[name_to_reload]
This is pretty much the same thing I've implemented. :)
-eric
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 451 update
On Sun, Oct 27, 2013 at 4:41 PM, PJ Eby wrote: > I'm talking about userspace code that implements lazy importing > features, like the lazyModule() function in this module: > >http://svn.eby-sarna.com/Importing/peak/util/imports.py?view=markup > > Specifically, I'm trying to get an idea of how much that code will > need to change under the PEP (and apparently under importlib in > general). Depending on the outcome of issue19413, you shouldn't have to change anything. PEP 451 is aiming for strict backward compatibility. > Honestly, I'm finding all this stuff *really* confusing, which is kind > of worrying. I mean, I gather I'm one of the handful of people who > really understood how importing *used to work*, and I'm having a lot > of trouble wrapping my brain around the new world. > > (Granted, I think that may be because I understand how a lot of old > corner cases work, Like Nick, I would love more tests that cover these corner cases. It would have saved us (Brett especially) a lot of headache. At the least any description you can offer would be great. > but what's bugging me is that I no longer > understand how those old corners work under the new regime, nor do I > feel I understand what the *new* corners will be. This may also just > be communication problems, and the fact that it's been months since I > really walked through importlib line by line, and have never really > walked through it (or PEP 451) quite as thoroughly as I have import.c. > I also seem to be having trouble grokking why the motivating use > cases for PEP 451 can't be solved by just providing people with good > base classes to use for writing loaders -- i.e., I don't get why the > core protocol has to change to address the use case of writing loaders > more easily. The new protocol seems way more complex than PEP 302, What new protocol specifically? Finder.find_module() now returns a module spec instead of a loader. Loader.exec_module() gets used now rather than load_module(). Loaders don't have to worry about all the boilerplate stuff anymore (managing sys.modules and import-related module attributes). From my perspective PEP 451 is simplifying protocols. -eric > and ISTM the complexity could just be pushed off to the loader side of > the protocol without creating more interdependency between importlib > and the loaders.) ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Issue #18408: Add a new PyFrame_FastToLocalsWithError() function to handle
Am 29.10.2013 01:19, schrieb victor.stinner: > http://hg.python.org/cpython/rev/4ef4578db38a > changeset: 86715:4ef4578db38a > user:Victor Stinner > date:Tue Oct 29 01:19:37 2013 +0100 > summary: > Issue #18408: Add a new PyFrame_FastToLocalsWithError() function to handle > exceptions when merging fast locals into f_locals of a frame. > PyEval_GetLocals() now raises an exception and return NULL on failure. You'll have to either make this private or document it. Georg ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Issue #18408: Fix PyUnicode_AsUTF8AndSize(), raise MemoryError exception on
Am 29.10.2013 01:46, schrieb victor.stinner:
> http://hg.python.org/cpython/rev/e1d51c42e5a1
> changeset: 86716:e1d51c42e5a1
> user:Victor Stinner
> date:Tue Oct 29 01:28:23 2013 +0100
> summary:
> Issue #18408: Fix PyUnicode_AsUTF8AndSize(), raise MemoryError exception on
> memory allocation failure
>
> files:
> Objects/unicodeobject.c | 1 +
> 1 files changed, 1 insertions(+), 0 deletions(-)
>
>
> diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
> --- a/Objects/unicodeobject.c
> +++ b/Objects/unicodeobject.c
> @@ -3766,6 +3766,7 @@
> return NULL;
> _PyUnicode_UTF8(unicode) = PyObject_MALLOC(PyBytes_GET_SIZE(bytes) +
> 1);
> if (_PyUnicode_UTF8(unicode) == NULL) {
> +PyErr_NoMemory();
> Py_DECREF(bytes);
> return NULL;
> }
Shouldn't this (and related commits from #18408) have been committed to the 3.3
branch?
Georg
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
