[issue1758146] Crash in PyObject_Malloc

Terry J. Reedy added the comment: OK. If Graham or anyone has concrete suggestions for improving the current 3.2a1 doc for threading, open a fresh issue. -- resolution: -> invalid status: open -> closed ___ Python tracker

[issue1758146] Crash in PyObject_Malloc

Graham Dumpleton added the comment: The actual reported problem was likely because of known issues with running subversion Python wrappers in a sub interpreter. The rest of the conversation was for a completely different issue which relates to mod_python not using thread APIs in Python in the

[issue1758146] Crash in PyObject_Malloc

Terry J. Reedy added the comment: Is this still an issue for 2.7 or 3.x? Is it actually a Python issue or should it be closed? -- nosy: +tjreedy ___ Python tracker ___ ___

[issue1758146] Crash in PyObject_Malloc

Graham Dumpleton <[EMAIL PROTECTED]> added the comment: I do understand. The initial thread, which is effectively a foreign thread to Python to begin with, when used to initialise Python, ie., call Py_Initialize(), is treated in a special way in as much as as a side effect it does that initia

[issue1758146] Crash in PyObject_Malloc

Franco DiRosa <[EMAIL PROTECTED]> added the comment: I'm unsure if you are understanding what I'm doing so here is the story... I stepped through Py_Initialize and this function takes the main interpreter and it's initial thread state and makes that the GIL thread state. The following code i

[issue1758146] Crash in PyObject_Malloc

Graham Dumpleton <[EMAIL PROTECTED]> added the comment: Franco, you said 'I found that you cannot create additional thread states against the first interpreter and swap between them w/o this assertion occurring. ...' Since the Py_DEBUG check is checking against the simplified GIL state API t

[issue1758146] Crash in PyObject_Malloc

Adam Olsen <[EMAIL PROTECTED]> added the comment: Graham, I appreciate the history of sub-interpreters and how entrenched they are. Changing those practises requires a significant investment. This is an important factor to consider. The other factor is the continuing maintenance and developmen

[issue1758146] Crash in PyObject_Malloc

Franco DiRosa <[EMAIL PROTECTED]> added the comment: By the way. I switched to using the GIL functions on the main interpreter and everything works great now. It is a better solution to use the GIL functions because I also had my own code that prevented dead lock from occuring when a python

[issue1758146] Crash in PyObject_Malloc

Franco DiRosa <[EMAIL PROTECTED]> added the comment: "Also, that Py_DEBUG check effectively says that if you use simplified GIL API for a particular thread against the first interpreter, you are prohibited from creating additional thread states for that thread." I found that you cannot create

[issue1758146] Crash in PyObject_Malloc

Graham Dumpleton <[EMAIL PROTECTED]> added the comment: I know the discussions more or less says this, but I want to add some additional information. For the record, the reason that mod_python crashes with 'Invalid thread state for this thread' when Py_DEBUG is defined in part relates to: h

[issue1758146] Crash in PyObject_Malloc

Franco DiRosa <[EMAIL PROTECTED]> added the comment: OK, I think I found my problem. I was using the main interpreter state (the one created by Py_Initialize) to create new thread states with. It seems that this interpreter state is reserved for GIL functions so one will need to create a ne

[issue1758146] Crash in PyObject_Malloc

Adam Olsen <[EMAIL PROTECTED]> added the comment: It's only checking that the original tstate *for the current thread* and the new tstate have a different subinterpreter. A subinterpreter can have multiple tstates, so long as they're all in different threads. The documentation is referring spec

[issue1758146] Crash in PyObject_Malloc

Franco DiRosa <[EMAIL PROTECTED]> added the comment: Thanks Adam but I'm still confused because... There is a new rule in version 2.3.5. Which is one interpreter with many thread states are supported for the GIL functions. So this code breaks that rule since this if statement is checki

[issue1758146] Crash in PyObject_Malloc

ays. It seems since 2.3.5 they don't want you to use multiple interpreters is my guess when threading is involved. - Franco - Original Message - From: "Adam Olsen" <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Tuesday, July 08, 2008 6:25 PM Subject: [iss

[issue1758146] Crash in PyObject_Malloc

Adam Olsen <[EMAIL PROTECTED]> added the comment: Franco, you need to look at the line above that check: PyThreadState *check = PyGILState_GetThisThreadState(); if (check && check->interp == newts->interp && check != newts) Py_FatalError("Invalid thread state for

[issue1758146] Crash in PyObject_Malloc

Franco DiRosa <[EMAIL PROTECTED]> added the comment: I believe PyThreadState_Swap function in ceval.c has a bug as I stated earlier. However, I have not seen it included in the latest patches so now I wonder... The following line in PyThreadState_Swap... if (check && check->interp == newts->i

[issue1758146] Crash in PyObject_Malloc

Adam Olsen <[EMAIL PROTECTED]> added the comment: Ahh, I did miss that bit, but it doesn't really matter. Tell modwsgi to only use the main interpreter ("PythonInterpreter main_interpreter"), and if you want multiple modules of the same name put them in different packages. Any other problems (t

[issue1758146] Crash in PyObject_Malloc

Vaclav Slavik <[EMAIL PROTECTED]> added the comment: I'm sorry, did you actually read my comments? Once again, this has nothing to do with threads and everything to do with isolation of independent Python apps running in the same *process*. Hope it got through this time :-/ _

[issue1758146] Crash in PyObject_Malloc

Adam Olsen <[EMAIL PROTECTED]> added the comment: Apparently modwsgi uses subinterpreters because some third-party packages aren't sufficiently thread-safe - modwsgi can't fix those packages, so subinterpreters are the next best thing. http://groups.google.com/group/modwsgi/browse_frm/thread/988

[issue1758146] Crash in PyObject_Malloc

Vaclav Slavik <[EMAIL PROTECTED]> added the comment: > This could be done either by not using the normal import mechanism This is completely unrealistic suggestion, people use libraries and frameworks in their code, you're in effect suggestion that no library that could possibly be used in webap

[issue1758146] Crash in PyObject_Malloc

Changes by Benjamin Peterson <[EMAIL PROTECTED]>: -- type: -> crash ___ Python tracker <[EMAIL PROTECTED]> ___ ___ Python-bugs-list

[issue1758146] Crash in PyObject_Malloc

Adam Olsen <[EMAIL PROTECTED]> added the comment: Right, so it's only the python modules loaded as part of the app that need to be isolated. You don't need the stdlib or any other part of the interpreter to be isolated. This could be done either by not using the normal import mechanism (build y

[issue1758146] Crash in PyObject_Malloc

Vaclav Slavik <[EMAIL PROTECTED]> added the comment: > Does the PythonInterpreter option create multiple interpreters > within a single process Yes. > They aren't truly isolated interpreters and nobody I've asked has yet > provided a use case for it. If you ignore mod_python and mod_wsgi, the

[issue1758146] Crash in PyObject_Malloc

Adam Olsen <[EMAIL PROTECTED]> added the comment: Does the PythonInterpreter option create multiple interpreters within a single process, rather than spawning separate processes? IMO, that API should be ripped out. They aren't truly isolated interpreters and nobody I've asked has yet provided a

[issue1758146] Crash in PyObject_Malloc

Franco DiRosa <[EMAIL PROTECTED]> added the comment: The documentation states that thread states are supported within a single interpreter and not supported across other interpreters (specifically for the GIL functions which are just wrapper functions around the PyEval_ functions). So I woul

[issue1758146] Crash in PyObject_Malloc

Vaclav Slavik <[EMAIL PROTECTED]> added the comment: I'm able to reliably reproduce this bug (using Apache 2.2.8, otherwise same as above), although not with mod_python's simple tests, but only with Trac (apparently, Trac creates some threads while processing the request). How to reproduce: conf