[issue41798] [C API] Revisit usage of the PyCapsule C API with multi-phase initialization API
Change by pkerling <9b6ab...@casix.org>: -- nosy: +pkerling ___ Python tracker <https://bugs.python.org/issue41798> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42298] Documented interaction of single-stage init and sub-interpreters inaccurate
New submission from pkerling <9b6ab...@casix.org>:
The C API documentation says this about single-phase initialization and
sub-interpreters:
"[T]he first time a particular extension is imported, it is initialized
normally, and a (shallow) copy of its module’s dictionary is squirreled away.
When the same extension is imported by another (sub-)interpreter, a new module
is initialized and filled with the contents of this copy; the extension’s init
function is not called."
- from https://docs.python.org/3.10/c-api/init.html#c.Py_NewInterpreter
I was investigating crashes relating to the _datetime module and
sub-interpreters and from my observations, this does not seem to be true.
I have tracked this functionality down to the m_base.m_copy dictionary of the
PyModuleDef of an extension and the functions _PyImport_FixupExtensionObject
and _PyImport_FindExtensionObject in Python/import.c. However, modules are only
ever added to the `extensions` global when imported in the main interpreter,
see
https://github.com/python/cpython/blob/1f73c320e2921605c4963e202f6bdac1ef18f2ce/Python/import.c#L480
Furthermore, even if they were added and m_base.m_copy was set, it would be
cleared again on sub-interpreter shutdown here:
https://github.com/python/cpython/blob/1f73c320e2921605c4963e202f6bdac1ef18f2ce/Python/pystate.c#L796
- implying that the module will be loaded and initialized again next time due
to this check:
https://github.com/python/cpython/blob/1f73c320e2921605c4963e202f6bdac1ef18f2ce/Python/import.c#L556
These observations are supported by the fact that in my tests, if "import
_datetime" is ran subsequently in two different sub-interpreters,
PyInit__datetime is indeed called twice.
Test code - set a breakpoint on PyInit__datetime to observe the behavior:
#include
#include
int main()
{
Py_Initialize();
for (int i = 0; i < 100; ++i) {
PyThreadState* ts = Py_NewInterpreter();
assert(ts);
int result = PyRun_SimpleString("import _datetime");
assert(result == 0);
Py_EndInterpreter(ts);
}
return 0;
}
In summary, it seems to me that the documented behavior is not accurate (any
more?) - so either the docs or the implementation should change.
--
assignee: docs@python
components: Documentation
messages: 380602
nosy: docs@python, pkerling
priority: normal
severity: normal
status: open
title: Documented interaction of single-stage init and sub-interpreters
inaccurate
versions: Python 3.10, Python 3.6, Python 3.7, Python 3.8, Python 3.9
___
Python tracker
<https://bugs.python.org/issue42298>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30654] signal module always overwrites SIGINT on interpreter shutdown
New submission from pkerling: The signal module checks the SIGINT handler on startup. It only registers a new custom handler if the default OS handler is still installed, so that when embedding python in an application the SIGINT handler of that application is not overwritten. But on shutdown in finisignal, it *always* sets SIGINT to SIG_DFL. The reason is that it saves the old handler in old_siginthandler, but *only* if the signal handler is overwritten in init, which only happens when it was SIG_DFL in the first place! If there was already a handler in place in init (-> no overwriting), old_siginthandler will default to the initialization value, which is also SIG_DFL. This means that when an application embeds Python and needs a custom SIGINT handler, it will stop to work as soon as it shuts down Python since it will always be reset to SIG_DFL. -- components: Interpreter Core messages: 295915 nosy: pkerling priority: normal severity: normal status: open title: signal module always overwrites SIGINT on interpreter shutdown type: behavior versions: Python 2.7, Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7 ___ Python tracker <http://bugs.python.org/issue30654> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30654] signal module always overwrites SIGINT on interpreter shutdown
Change by pkerling <9b6ab...@casix.org>: -- keywords: +patch pull_requests: +6780 stage: -> patch review ___ Python tracker <https://bugs.python.org/issue30654> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30654] signal module always overwrites SIGINT on interpreter shutdown
pkerling <9b6ab...@casix.org> added the comment: Thanks! I'm a bit disappointed that it won't make it into 2.7, but I can understand the decision. To give some context: I came across this while working on Kodi and noticing that it does not shutdown cleanly via Ctrl+C or SIGTERM. After investigating, I came to the conclusion that it is due to this bug. Kodi shuts down the Python interpreter every time no add-on is doing active work, which is almost guaranteed to happen shortly after application startup. Then this bug caused a reset of the SIGTERM handler to the default handler, making the Kodi handler that does a clean shutdown useless. Now there are plans to switch to Python 3 in Kodi, but it won't happen until the major release after the next, so we're stuck with 2.7 for some time. I guess we'll have to work around this in Kodi for the time being. -- ___ Python tracker <https://bugs.python.org/issue30654> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
