Bugs item #1199282, was opened at 2005-05-10 18:24
Message generated for change (Comment added) made by atila-cheops
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1199282&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.4
Status: Open
Resolution: None
Priority: 5
Submitted By: cheops (atila-cheops)
Assigned to: Peter Ã
strand (astrand)
Summary: subprocess _active.remove(self) self not in list _active
Initial Comment:
I start a subprocess in a seperate thread (25 concurrent
threads)
in some of the threads the following error occurs
Exception in thread Thread-4:
Traceback (most recent call last):
File "C:\Python24\lib\threading.py", line 442, in
__bootstrap
self.run()
File "upgrade.py", line 45, in run
returncode = p.wait()
File "C:\Python24\lib\subprocess.py", line 765, in wait
_active.remove(self)
ValueError: list.remove(x): x not in list
this is the code that starts the subprocess and where I
wait for the result
p = subprocess.Popen('command', \
stdin=None,
stdout=subprocess.PIPE, \
stderr=subprocess.STDOUT,
universal_newlines=True)
returncode = p.wait()
errormessage = p.stdout.readlines()
----------------------------------------------------------------------
>Comment By: cheops (atila-cheops)
Date: 2006-04-11 20:10
Message:
Logged In: YES
user_id=1276121
there are 2 definitions of the function poll, one for
windows and one for POSIX systems
in the windows version _deadlock is not used
in the POSIX version _deadlock is used
see also modification made by loewis in committed version
45234
----------------------------------------------------------------------
Comment By: HVB bei TUP (hvb_tup)
Date: 2006-04-11 07:21
Message:
Logged In: YES
user_id=1434251
I looked at the patch #1467770 you mentioned and
downloaded the patch.
Is it correct that _deadlock variable is used in the
definition of poll but it is not in the argument list?
----------------------------------------------------------------------
Comment By: cheops (atila-cheops)
Date: 2006-04-10 14:57
Message:
Logged In: YES
user_id=1276121
see patch #1467770
----------------------------------------------------------------------
Comment By: Tristan Faujour (tfaujour)
Date: 2006-03-29 13:50
Message:
Logged In: YES
user_id=1488657
> Simply list operations such as remove() and append() are
> thread safe,
OK, my apologies... I did not know.
I did some more tests. On Linux, I found lots of:
File "./subprocess.py", line 428, in call
return Popen(*args, **kwargs).wait()
File "./subprocess.py", line 1023, in wait
pid, sts = os.waitpid(self.pid, 0)
OSError: [Errno 10] No child processes
The try...except solution does not handle this (since we are
in the "posix" section).
In my opinion, the call to _cleanup() in Popen.__init__() is
useless (it just checks if older processes have stopped when
a new one is started. I cannot see why it could be
mandatory) and it is the root of this bug.
I commented it out (line 506) and retried my tests on Linux
& Windows plateforms: I had no exception at all.
----------------------------------------------------------------------
Comment By: Peter Ã
strand (astrand)
Date: 2006-03-29 05:11
Message:
Logged In: YES
user_id=344921
>I think accesses to _active should be serialized in a
>thread-safe way. _active could be a Queue (from the Queue
>module) for example
Simply list operations such as remove() and append() are
thread safe, so there should be no need for a Queue. Also, a
Queue cannot be used since the subprocess module needs to be
compatible with Python 2.2.
----------------------------------------------------------------------
Comment By: Tristan Faujour (tfaujour)
Date: 2006-03-28 23:17
Message:
Logged In: YES
user_id=1488657
I am running into the same problem on a Windows 2k/XP
plateform with a multi-threaded application. It occurs randomly.
It has never appened (yet) on a Linux plateform.
I think accesses to _active should be serialized in a
thread-safe way. _active could be a Queue (from the Queue
module) for example.
----------------------------------------------------------------------
Comment By: HVB bei TUP (hvb_tup)
Date: 2006-01-25 08:10
Message:
Logged In: YES
user_id=1434251
Wouldn't it be best to completely remove any references
to "_active" and "_cleanup"?
----------------------------------------------------------------------
Comment By: cheops (atila-cheops)
Date: 2006-01-25 07:08
Message:
Logged In: YES
user_id=1276121
As suggested by astrand
adding a try ... except clause in the file subprocess.py did
the job
I had to add that try ... except clause in 2 places
if you look in the file there are 2 instances were
list.remove(x) occurs unprotected.
try:
list.remove(x)
except:
pass
I have worked with 2.4.0, 2.4.1 and 2.4.2 and all three
needed the patch.
Hope this helps.
----------------------------------------------------------------------
Comment By: HVB bei TUP (hvb_tup)
Date: 2006-01-23 16:34
Message:
Logged In: YES
user_id=1434251
BTW: In my case, I call python.exe from a Windows service.
----------------------------------------------------------------------
Comment By: HVB bei TUP (hvb_tup)
Date: 2006-01-23 16:30
Message:
Logged In: YES
user_id=1434251
I have a similar problem.
Python 2.4.1 under MS Windows 2003,
Multi-Threaded application (about concurrent 10 threads).
In my case the same error occurs during _cleanup called
from __init__ :
File "E:\lisa_ins\ewu\coop\reporting\python\tup_lisa\util\t
hreadutil.py", line 582, in callSubProcess
creationflags = creationflags
File "C:\Python24\lib\subprocess.py", line 506, in
__init__
_cleanup()
File "C:\Python24\lib\subprocess.py", line 414, in
_cleanup
inst.poll()
File "C:\Python24\lib\subprocess.py", line 755, in poll
_active.remove(self)
ValueError: list.remove(x): x not in list
Is there a work-around?
----------------------------------------------------------------------
Comment By: cheops (atila-cheops)
Date: 2005-09-19 09:29
Message:
Logged In: YES
user_id=1276121
I noticed this bug under windows
to reproduce the bug, I attached the script I use, but this
needs input, I tried with a simpler example (pinging a number
of host concurrently) but that did not cause the bug.
----------------------------------------------------------------------
Comment By: Peter Ã
strand (astrand)
Date: 2005-06-23 16:03
Message:
Logged In: YES
user_id=344921
I believe it should be sufficient to add a try...except
clause around _active.remove(). Can you upload a complete
example that triggers the bug? Have you noticed this bug on
Windows, UNIX or both platforms?
----------------------------------------------------------------------
Comment By: cheops (atila-cheops)
Date: 2005-05-12 10:17
Message:
Logged In: YES
user_id=1276121
this might be related to bug 1183780
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1199282&group_id=5470
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
