Bugs item #1590864, was opened at 2006-11-05 17:06
Message generated for change (Comment added) made by ronaldoussoren
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1590864&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Michael Tsai (michaeltsai)
Assigned to: Nobody/Anonymous (nobody)
Summary: import deadlocks when using PyObjC threads
Initial Comment:
When I use subprocess.py from a child thread, sometimes it deadlocks. I
determined that the new process is blocked during an import:
#0 0x90024427 in semaphore_wait_signal_trap ()
#1 0x90028414 in pthread_cond_wait ()
#2 0x004c77bf in PyThread_acquire_lock (lock=0x3189a0, waitflag=1)
at Python/thread_pthread.h:452
#3 0x004ae2a6 in lock_import () at Python/import.c:266
#4 0x004b24be in PyImport_ImportModuleLevel (name=0xaad74 "errno",
globals=0xbaed0, locals=0x502aa0, fromlist=0xc1378, level=-1) at
Python/import.c:2054
#5 0x0048d2e2 in builtin___import__ (self=0x0, args=0x53724c90,
kwds=0x0) at Python/bltinmodule.c:47
#6 0x0040decb in PyObject_Call (func=0xa94b8, arg=0x53724c90,
kw=0x0) at Objects/abstract.c:1860
and that the code in question is in os.py:
def _execvpe(file, args, env=None):
from errno import ENOENT, ENOTDIR
I think the problem is that since exec (the C function) hasn't yet been
called in the new process, it's inherited from the fork a lock that's already
held. The main process will eventually release its copy of the lock, but this
will not unlock it in the new process, so it deadlocks.
If I change os.py so that it imports the constants outside of
_execvpe, the new process no longer blocks in this way.
This is on Mac OS X 10.4.8.
----------------------------------------------------------------------
>Comment By: Ronald Oussoren (ronaldoussoren)
Date: 2007-07-09 10:15
Message:
Logged In: YES
user_id=580910
Originator: NO
Do you have sample code that reproduces this problem? (Not necessarily
code that has this problem 100% of the time)
----------------------------------------------------------------------
Comment By: Kosuha (kosuha)
Date: 2007-02-27 17:54
Message:
Logged In: YES
user_id=1684258
Originator: NO
I confirm that problem with deadlock on execution of
PyImport_ImportModuleLevel exists. Here is a working example:
1) Python was embbeded with C++ application using Python for scripting
support:
------------------------------------------------------------------
PyThreadState * InitThreadScripting()
{
ASSERT_KOBOLD( g_pyMainThreadState );
// get the global lock
PyEval_AcquireLock();
// get a reference to the PyInterpreterState
PyInterpreterState * mainInterpreterState = g_pyMainThreadState->interp;
ASSERT_KOBOLD( mainInterpreterState );
// create a thread state object for this thread
PyThreadState * myThreadState = PyThreadState_New(mainInterpreterState);
// free the lock
PyEval_ReleaseLock();
return myThreadState;
}
---------------------------------------------------------------------
void DeadLock()
{
Py_Initialize();
PyEval_InitThreads();
g_AiGlobals = new Py::Dict();
g_pyInterpState = PyInterpreterState_New();
// save a pointer to the main PyThreadState object
g_pyMainThreadState = PyThreadState_Get();
ASSERT_KOBOLD( g_pyMainThreadState );
// release the lock
PyEval_ReleaseLock();
g_pyWorldThreadState = InitThreadScripting();
// import sys
// sys.path.append ("./scripts")
//
PyObject *p = PyImport_ImportModuleEx ("sys", **g_AiGlobals, NULL, NULL);
Py::Module mod_sys (p);
Py::List path = mod_sys.getAttr ("path");
path.append (Py::String ("scripts"));
path.append (Py::String ("scripts/sys"));
path.append (Py::String ("../../scripts"));
path.append (Py::String ("../../scripts/sys"));
Py_XDECREF (p);
// HERE IT OCCURS //
Log.ScriptsSrc("Python", "Running startup python scripts...");
PyObject *p = PyImport_ImportModuleEx ("startup", **g_AiGlobals, NULL,
NULL); // <<< Here
if (reload) PyImport_ReloadModule (p);
Py::Module module (p);
Py_XDECREF (p);
}
Execution locks right on PyImport_ImportModuleEx.
Code from sturtup.py:
------------------------------------------------------------------------------------
# This module is sturtup script.
# Here we are redirecting output and checking for server version.
################################################################################
import sys
from consts import * # Import of constants
import config as cfg # Import of configuration constants
reload(cfg)
################################################################################
#
----------------------------------------------------------------------------------------------------------------------------------------------------------------
class OurLogStdErr:
def write (self, txt):
printLog (txt, PRINT_ERROR)
#
----------------------------------------------------------------------------------------------------------------------------------------------------------------
class OurLogStdOut:
def write (self, txt):
printLog (txt)
#
----------------------------------------------------------------------------------------------------------------------------------------------------------------
def CheckServerVersion():
# Checking for server build
if GetServerBuild() < MIN_SERVER_BUILD:
printLog( "YOU ARE TRYING TO RUN PYTHON SCRIPTS ON OUTDATED SERVER
BUILD!\
\nREQUIRED SERVER BUILD: %s\
\nPlease Update your server core before running server!\
\nScripting Engine will be Shut Down!"\
% (MIN_SERVER_BUILD), PRINT_ERROR )
killScripting()
#
----------------------------------------------------------------------------------------------------------------------------------------------------------------
def GetScriptsVersion():
return SCRIPTS_VERSION
################################################################################
# Startup code here:
# Redirecting errors:
sys.stderr = OurLogStdErr()
# Redirecting output:
sys.stdout = OurLogStdOut()
---------------------------------------------------------------------------------------
----------------------------------------------------------------------
Comment By: Peter Åstrand (astrand)
Date: 2007-01-13 23:42
Message:
Logged In: YES
user_id=344921
Originator: NO
Since both the reporter and I believes that this is not a bug in the
subprocess module, I'm stepping back.
----------------------------------------------------------------------
Comment By: Michael Tsai (michaeltsai)
Date: 2007-01-07 18:09
Message:
Logged In: YES
user_id=817528
Originator: YES
I don't have time at the moment to write sample code that reproduces this.
But, FYI, I was using PyObjC to create the threads. It might not happen
with "threading" threads. And second, I think it's a bug in os.py, not in
subprocess.py. Sorry for the confusion.
----------------------------------------------------------------------
Comment By: Peter Åstrand (astrand)
Date: 2007-01-07 15:10
Message:
Logged In: YES
user_id=344921
Originator: NO
Can you provide a test case or sample code that demonstrates this problem?
I'm a bit unsure of if this really is a subprocess bug or a more general
Python bug.
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1590864&group_id=5470
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
