[issue6627] threading.local() does not work with C-created threads

Roundup Robot added the comment: New changeset f4eade5df217 by Benjamin Peterson in branch '3.3': document that a new Python thread context is created in ctypes callbacks (closes #6627) http://hg.python.org/cpython/rev/f4eade5df217 New changeset 9cd2d7a3f9f2 by Benjamin Peterson in branch '2.7'

[issue6627] threading.local() does not work with C-created threads

Nikolaus Rath added the comment: (adding the documentation and ctypes experts from http://docs.python.org/devguide/experts.html to noisy list in the hope to get this moving again.) -- nosy: +belopolsky, eric.araujo, ezio.melotti, georg.brandl, meador.inge versions: -Python 2.6, Python

[issue6627] threading.local() does not work with C-created threads

Nikolaus Rath added the comment: Here's a patch that (I think) incorporates all the comments. If someone could apply it, that would be great :-). -- keywords: +patch versions: +Python 2.7, Python 3.1, Python 3.2, Python 3.3, Python 3.4, Python 3.5 Added file: http://bugs.python.org/fil

[issue6627] threading.local() does not work with C-created threads

Changes by Ronald Oussoren : -- nosy: +ronaldoussoren ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mai

[issue6627] threading.local() does not work with C-created threads

Mark Lawrence added the comment: msg116747 suggests wording for a doc patch. Could this be applied and this issue closed? msg117029 talks about a potential feature request. Was this ever discussed? -- nosy: +BreamoreBoy ___ Python tracker

[issue6627] threading.local() does not work with C-created threads

Nick Coghlan added the comment: On Tue, Sep 21, 2010 at 2:39 PM, Swapnil Talekar wrote: > Swapnil Talekar added the comment: > Nick, the last statement, > "While this is correct for most purposes, it does mean that..." > can be simplified to, > "It means...". > I had to read it several times b

[issue6627] threading.local() does not work with C-created threads

Swapnil Talekar added the comment: Nick, the last statement, "While this is correct for most purposes, it does mean that..." can be simplified to, "It means...". I had to read it several times before I realized, there is no "not" after "does" :) BTW, since this particular arrangement of havi

[issue6627] threading.local() does not work with C-created threads

Nick Coghlan added the comment: The suggestion in http://bugs.python.org/issue6627#msg116722 is a good one. This a case where the user may legitimately not realise that threading.local is stored in the *temporary* state created by ctypes rather than in something more persistent inside the int

[issue6627] threading.local() does not work with C-created threads

Nikolaus Rath added the comment: One point of ctypes is to save the user the trouble of having to create a full blown C extension, so I don't think it's reasonable to expect a ctypes user to have read the full C API documentation as well. Only a very small subset of the page that you gave is

[issue6627] threading.local() does not work with C-created threads

Amaury Forgeot d'Arc added the comment: This is not specific to ctypes. Please read http://docs.python.org/c-api/init.html#thread-state-and-the-global-interpreter-lock specially the paragraph that says "...when threads are created from C...". Is it explicit enough? How would you change it? ---

[issue6627] threading.local() does not work with C-created threads

Nikolaus Rath added the comment: To be a bit more constructive, why not add something like this in paragraph to http://docs.python.org/library/ctypes.html#callback-functions: "Note that if the callback function is called in a new thread that has been created outside of Python's control (i.e.,

[issue6627] threading.local() does not work with C-created threads

Nikolaus Rath added the comment: No, I am not saying that the behaviour of ctypes is wrong. It just happens to have some effects on threading.local that I think should be documented. That's why I reassigned this as a documentation bug. Please reconsider closing this bug. I'm also happy to ch

[issue6627] threading.local() does not work with C-created threads

Amaury Forgeot d'Arc added the comment: > ctypes performs the initializations in a way that break the > threading.local functionality. Not quite. ctypes (or more exactly: the PyGILState_Ensure() and PyGILState_Release() functions, which are the standard API to do this) is in a situation wher

[issue6627] threading.local() does not work with C-created threads

Changes by Nikolaus Rath : -- resolution: invalid -> ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mai

[issue6627] threading.local() does not work with C-created threads

Nikolaus Rath added the comment: @Swapnil: the rules you quote are correct for the C extension, but do not apply when using ctypes, because ctypes is doing the required initializations automatically. However, if Amaury is correct, ctypes performs the initializations in a way that break the t

[issue6627] threading.local() does not work with C-created threads

Amaury Forgeot d'Arc added the comment: One more thing: in the sample code, the thread initializes no PyThreadState. So the ctypes callback creates a temporary PyThreadState just for the duration of the call. This explains the difference between threading.local and _threading_local: - threadi

[issue6627] threading.local() does not work with C-created threads

Nick Coghlan added the comment: Swapnil's analysis looks correct to me - there are certain rules you have to follow before calling back into the Python interpreter core. If you don't follow them, the behaviour you will get is completely undefined. If the problem still occurs even when the C t

[issue6627] threading.local() does not work with C-created threads

Swapnil Talekar added the comment: As far as I know, the thread creation done in the file is not correct. While creating threads in C extension, there are certain rules to follow. Firstly, Python should be made thread-aware if it is not already i.e. call PyEval_InitThreads in the C callback f

[issue6627] threading.local() does not work with C-created threads

Changes by Matt Joiner : -- nosy: +anacrolix ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.

[issue6627] threading.local() does not work with C-created threads

Changes by Giorgos Verigakis : -- nosy: +verigak ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.pyt

[issue6627] threading.local() does not work with C-created threads

Changes by Nick Coghlan : -- nosy: +ncoghlan ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.

[issue6627] threading.local() does not work with C-created threads

New submission from Nikolaus Rath : When threads are created by a C extension loaded with ctypes, threading.local() objects are always empty. If one uses _threading_local.local() instead of threading.local(), the problem does not occur. More information and example program showing the behaviour