[ python-Bugs-853411 ] After fork in subthread, signals are blocked
Bugs item #853411, was opened at 2003-12-03 16:55 Message generated for change (Comment added) made by mwh You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=853411&group_id=5470 Category: Threads Group: Python 2.2.3 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Göran Uddeborg (goeran) Assigned to: Nobody/Anonymous (nobody) Summary: After fork in subthread, signals are blocked Initial Comment: When a Python program starts a new thread, and this new thread forks, the forked process ingores signals. It will not terminate, or dump core if that would be applicable, when it receives a signal. I attach a test case which illustrates the behaviour. In this example, the child process is sent a SEGV signal from the subthread of the parent process. This should cause the child process to die and produce a core. But execution of the parent program threads finishes, there is still a python process around, continuing to sleep. Apparently, Python subthreads blocks signals. On Linux, /proc/*/status for subthreads includes the line SigBlk: 7ffbfeff The Python documentation says one should only install signal handlers in the main thread, and only the main thread will recieve signals. So this makes sense. But when the subthread forks, the new process inherits this signal mask, which seems to be incorrect behaviour. I would assume, if Python sets this signal mask for threads, it should also reset it again after a fork. I've seen this on these two platforms: Python 2.2.3 on Red Hat Linux RHEL 3WS (IA32) Python 2.2.1 on Sun Solaris 8 (Sparc) -- >Comment By: Michael Hudson (mwh) Date: 2005-03-26 10:41 Message: Logged In: YES user_id=6656 Good. Closing. -- Comment By: Erik Osheim (moculus) Date: 2005-03-21 05:07 Message: Logged In: YES user_id=542811 I can confirm that on my install of 2.4, I no longer get this problem. -- Comment By: Michael Hudson (mwh) Date: 2005-03-18 08:59 Message: Logged In: YES user_id=6656 I think this should be fixed in Python 2.4. -- Comment By: Erik Osheim (moculus) Date: 2005-03-18 03:45 Message: Logged In: YES user_id=542811 This bug is still around. I have been experiencing it firsthand: if I write a simple program with one parent and one child thread and run rsync in the parent thread (via os.system), all is well. In the child thread it hangs indefinitely. After putting a bunch of debugging into rsync, I discovered that a USR2 signal was getting sent but never received, and rsyncs parent thread was waiting for the child to exit, and that the child was sleeping (having not gotten the signal). Is there any clue as to why this happens? This has been widely observed on Linux 2.6.* (this particular case affects Gentoo). -- Comment By: gmanipon (pymonger) Date: 2004-12-06 21:24 Message: Logged In: YES user_id=1173063 Sorry for the bother. Was there any resolution to this bug report? -- Comment By: Jack Jansen (jackjansen) Date: 2004-01-26 10:02 Message: Logged In: YES user_id=45365 I have absolutely no idea where the bug could be, someone better versed in the threading ideosyncracies of the various platforms needs to look at this, but I do want to point at hairy_flange's comment that fink-python 2.2.3 on OSX does *not* exhibit the bug (while on other platforms it does). It may be worthwhile to download a fink-python in source form, and see whether they do any configure magic. -- Comment By: Steve Muir (hairy_flange) Date: 2004-01-26 04:04 Message: Logged In: YES user_id=960132 I just submitted a bug report that is a duplicate of this bug (apologies!), I observed the same behaviour with the Python shipped with Mac OS X (Python 2.3), and Python 2.2.2 on RedHat/x86, but Fink Python 2.2.3 for OS X does not have this bug. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=853411&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1171023 ] Thread.join() fails to release Lock on KeyboardInterrupt
Bugs item #1171023, was opened at 2005-03-26 09:40 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1171023&group_id=5470 Category: Threads Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Peter Hansen (phansen) Assigned to: Nobody/Anonymous (nobody) Summary: Thread.join() fails to release Lock on KeyboardInterrupt Initial Comment: In threading.Thread.join(), the Condition/Lock object called self.__block is acquired upon entry, and released on exit without an enclosing try/finally to ensure that the release really occurs. If the join() call has no timeout, the wait() call that occurs inside can never be interrupted so there is no problem. If the join() call has a timeout, however, the wait() occurs in a loop which can be interrupted by a Ctrl-C, raising a KeyboardInterrupt which skips the self.__block.release() call and leaves the Lock acquired. This is a problem if the main thread (which is the only one that will see a KeyboardInterrupt) is the one calling join() and only if the other thread on which the join() is being called is a non-daemon thread or, in the case of a daemon thread, if the main thread subsequently attempts to wait for the other thread to terminate (for example, by monitoring isAlive() on the other thread). In any event, the lack of a try/finally means the joined thread will never really finish because any attempt by it to call its __stop() method will block forever. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1171023&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1170766 ] weakref.proxy incorrect behaviour
Bugs item #1170766, was opened at 2005-03-25 21:54 Message generated for change (Comment added) made by arigo You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1170766&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Alexander Kozlovsky (kozlovsky) Assigned to: Nobody/Anonymous (nobody) Summary: weakref.proxy incorrect behaviour Initial Comment: According documentation, proxy in most contexts must have the same behaviour as object itself. PROBLEM: bool(proxy_object) != bool(object_itself) EXAMPLE: >>> class X(list): pass # collection class ... >>> x = X() # empty collection >>> import weakref >>> proxy = weakref.proxy(x) >>> bool(x) False >>> bool(proxy) True PYTHON VERSION: 2.4 (#60, Nov 30 2004, 11:49:19) [MSC v.1310 32 bit (Intel)] also tested on: 2.3.4 (#53, May 25 2004, 21:17:02) [MSC v.1200 32 bit (Intel)] -- >Comment By: Armin Rigo (arigo) Date: 2005-03-26 14:50 Message: Logged In: YES user_id=4771 The bug is in weakrefobject:proxy_nonzero(), which calls the underlying object's nb_nonzero slot instead of going through the general PyObject_IsTrue() mecanism. Built-in types like list and dict don't have a nb_nonzero slot. As a related bug, (Callable)ProxyType should implement tp_richcompare in addition to tp_compare. In fact, we should review the code to check if ProxyType knows about the most recent type slots... -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1170766&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1167930 ] threading.Thread.join() cannot be interrupted by a Ctrl-C
Bugs item #1167930, was opened at 2005-03-21 17:19 Message generated for change (Comment added) made by phansen You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1167930&group_id=5470 Category: Threads Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: Nicholas Veeser (nveeser) Assigned to: Nobody/Anonymous (nobody) Summary: threading.Thread.join() cannot be interrupted by a Ctrl-C Initial Comment: I write a python program that that starts several threads and then waits on them all. If I use join() to wait on the threads, there is no way to Stop the process with Ctrl-C. Threading.Thread.join() uses a lock (thread.allocate_lock()) to put itself on the "wait_queue".It then calls thread.Lock.acquire(), which blocks indefinitely. Lock.acquire() (at least in POSIX) seems to work in such a way as to ignore any signals. (both semaphore and condition variable type). PyThread_acquire_lock() will not return until the lock is acquired, even if a signal is sent. Effectively, Ctrl-C is "masked" until the lock is released, (the joined thread is done), and the main thread comes back to the interpreter and handles the signal, producing a KeyboardInterrupt Exception. But not before the lock is released, which is once the thread is finished, which is too late if you want to kill the main thread and have it gracefully stop its child threads. So the "main" thread has no way to call, join() and still respond to Ctrl-C in some way. One solution could be to change threading.Thread.join() to use other methods of synchronization which respond to Ctrl-C more effectively. Another solution would be to have Lock.acquire() throw a KeyboardInterruped exception like other system calls. This IHMO would make the python objects behave more like Java, which requires catching the InterruptedException, giving the developer more control over how to handle this case. -- Comment By: Peter Hansen (phansen) Date: 2005-03-26 09:59 Message: Logged In: YES user_id=567267 Coincidentally this issue came up in a thread in comp.lang.python just now. See tim one's reply at http://groups.google.ca/groups?selm=mailman.884.815188.1799.python-list%40python.org which indicates that "it's simply not possible for Ctrl+C to interrupt a mutex acquire". A workaround is to be sure to call join() with a timeout value, inside a loop if you absolutely need an indefinite timeout, but that won't necessarily work because of a different problem which I just reported as bug 1171023. There's a workaround for that problem too though, provided you can subclass threading.Thread: provide your own join() which wraps the builtin one and which attempts to release the lock safely. I'll attach an example if I can figure out how... there's no option to do so on this particular page in Sourceforge. :-( -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1167930&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1171023 ] Thread.join() fails to release Lock on KeyboardInterrupt
Bugs item #1171023, was opened at 2005-03-26 09:40 Message generated for change (Comment added) made by phansen You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1171023&group_id=5470 Category: Threads Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Peter Hansen (phansen) Assigned to: Nobody/Anonymous (nobody) Summary: Thread.join() fails to release Lock on KeyboardInterrupt Initial Comment: In threading.Thread.join(), the Condition/Lock object called self.__block is acquired upon entry, and released on exit without an enclosing try/finally to ensure that the release really occurs. If the join() call has no timeout, the wait() call that occurs inside can never be interrupted so there is no problem. If the join() call has a timeout, however, the wait() occurs in a loop which can be interrupted by a Ctrl-C, raising a KeyboardInterrupt which skips the self.__block.release() call and leaves the Lock acquired. This is a problem if the main thread (which is the only one that will see a KeyboardInterrupt) is the one calling join() and only if the other thread on which the join() is being called is a non-daemon thread or, in the case of a daemon thread, if the main thread subsequently attempts to wait for the other thread to terminate (for example, by monitoring isAlive() on the other thread). In any event, the lack of a try/finally means the joined thread will never really finish because any attempt by it to call its __stop() method will block forever. -- >Comment By: Peter Hansen (phansen) Date: 2005-03-26 10:19 Message: Logged In: YES user_id=567267 A workaround (tested) is to subclass Thread and ensure that the lock is released. I'm not certain this is completely safe as written, but I'm assuming that you can safely attempt to release a lock that you don't own and the worst that can happen is you'll get an exception (which the workaround code catches and ignores). -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1171023&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1167930 ] threading.Thread.join() cannot be interrupted by a Ctrl-C
Bugs item #1167930, was opened at 2005-03-21 17:19 Message generated for change (Comment added) made by phansen You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1167930&group_id=5470 Category: Threads Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: Nicholas Veeser (nveeser) Assigned to: Nobody/Anonymous (nobody) Summary: threading.Thread.join() cannot be interrupted by a Ctrl-C Initial Comment: I write a python program that that starts several threads and then waits on them all. If I use join() to wait on the threads, there is no way to Stop the process with Ctrl-C. Threading.Thread.join() uses a lock (thread.allocate_lock()) to put itself on the "wait_queue".It then calls thread.Lock.acquire(), which blocks indefinitely. Lock.acquire() (at least in POSIX) seems to work in such a way as to ignore any signals. (both semaphore and condition variable type). PyThread_acquire_lock() will not return until the lock is acquired, even if a signal is sent. Effectively, Ctrl-C is "masked" until the lock is released, (the joined thread is done), and the main thread comes back to the interpreter and handles the signal, producing a KeyboardInterrupt Exception. But not before the lock is released, which is once the thread is finished, which is too late if you want to kill the main thread and have it gracefully stop its child threads. So the "main" thread has no way to call, join() and still respond to Ctrl-C in some way. One solution could be to change threading.Thread.join() to use other methods of synchronization which respond to Ctrl-C more effectively. Another solution would be to have Lock.acquire() throw a KeyboardInterruped exception like other system calls. This IHMO would make the python objects behave more like Java, which requires catching the InterruptedException, giving the developer more control over how to handle this case. -- Comment By: Peter Hansen (phansen) Date: 2005-03-26 10:22 Message: Logged In: YES user_id=567267 It appears only the original submitter (and admins?) can attach files, so I've attached the example workaround in the other bug report that I submitted. It makes more sense in that one anyway. -- Comment By: Peter Hansen (phansen) Date: 2005-03-26 09:59 Message: Logged In: YES user_id=567267 Coincidentally this issue came up in a thread in comp.lang.python just now. See tim one's reply at http://groups.google.ca/groups?selm=mailman.884.815188.1799.python-list%40python.org which indicates that "it's simply not possible for Ctrl+C to interrupt a mutex acquire". A workaround is to be sure to call join() with a timeout value, inside a loop if you absolutely need an indefinite timeout, but that won't necessarily work because of a different problem which I just reported as bug 1171023. There's a workaround for that problem too though, provided you can subclass threading.Thread: provide your own join() which wraps the builtin one and which attempts to release the lock safely. I'll attach an example if I can figure out how... there's no option to do so on this particular page in Sourceforge. :-( -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1167930&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-666958 ] repr.repr not always safe
Bugs item #666958, was opened at 2003-01-12 23:17 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=666958&group_id=5470 Category: Python Library Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Bob Ippolito (etrepum) Assigned to: Nobody/Anonymous (nobody) Summary: repr.repr not always safe Initial Comment: >>> class A: ... def __repr__(self): ... raise Exception ... def someMethod(self): ... pass ... >>> A() Traceback (most recent call last): File "", line 1, in ? File " ", line 3, in __repr__ Exception >>> repr.repr(A()) '' >>> repr.repr(A().someMethod) Traceback (most recent call last): File " ", line 1, in ? File "/usr/lib/python2.2/repr.py", line 15, in repr return self.repr1(x, self.maxlevel) File "/usr/lib/python2.2/repr.py", line 24, in repr1 s = `x` File " ", line -- >Comment By: Raymond Hettinger (rhettinger) Date: 2005-03-26 19:01 Message: Logged In: YES user_id=80475 The behavior of repr.repr(A().someMethod) is the same as `repr (A().someMethod)`. Also, in list example provided, a debugger takes you right to the heart of the OP's issue (resolving a faulty user implementation of __repr__). I believe the current behavior both useful and correct. The offered use case is somewhat rare and better solved with pdb. Changing the current behavior would likely impact someone's existing code (doctests for example). Recommend closing this bug. -- Comment By: Facundo Batista (facundobatista) Date: 2004-11-15 19:10 Message: Logged In: YES user_id=752496 What seems like a bug to me is that "repr.repr(A())" and "repr.repr(A().someMethod)" doesn't have the same behaviour. As this still happens in Py2.3.3, I updated the bug metadata. .Facundo -- Comment By: Neal Norwitz (nnorwitz) Date: 2003-02-06 20:33 Message: Logged In: YES user_id=33168 Bob, I'm not sure I understand what you want (ie, what behaviour you desire). If you could provide a patch to the repr module, that could help me. -- Comment By: Bob Ippolito (etrepum) Date: 2003-02-03 15:27 Message: Logged In: YES user_id=139309 Yes, of course repr.repr is for debugging use.. and it *really* sucks if your debugger crashes! I'm sure that's why it's inconsistent with __builtins__.repr, the bug I'm reporting is that it's inconsistent with itself. Going back to the example of a list again. If you're debugging a list that has one or two items that raise an exception, it's infinitely more useful to know *which* items have a broken __repr__ (along with the context of the valid objects), rather than knowing *one or more* items in the list has a broken __repr__. repr.repr is often called recursively; if something bad happens far down in the call stack it can really be a pain to track down where it happened (all tracebacks to repr are going to look mostly the same) unless repr.repr is fixed to handle this situation. -- Comment By: Christos Georgiou (tzot) Date: 2003-01-21 15:58 Message: Logged In: YES user_id=539787 Sorry, incomplete last post, I clicked in the wrong place (ie SUBMIT) to come back to IE. The case is that repr.py is documented as for debugging use... you suggest that all exceptions are caught, but that would not be consistent with __builtins__.repr . If the module was intended for generic use, I would suggest the opposite; no exceptions should be caught (for consistency). -- Comment By: Christos Georgiou (tzot) Date: 2003-01-21 15:47 Message: Logged In: YES user_id=539787 Do you suggest to remove the exception catching in Repr.repr_instance or adding a try clause in the s = `x` line of Repr.repr1 ? -- Comment By: Bob Ippolito (etrepum) Date: 2003-01-20 09:11 Message: Logged In: YES user_id=139309 It's not my module, it comes with Python (at least in my 2.2, and 2.3 CVS HEAD). Look for yourself! The issue has nothing to do with the fact that it has the same name as the builtin. -- Comment By: Christos Georgiou (tzot) Date: 2003-01-20 00:28 Message: Logged In: YES user_id=539787 Did you try to change the module name to, say, repr2 and then run it again? Any specific reason you named the module as an internal function? BTW, what is repr.repr supposed to do? Please supply the repr.py script, along with the full traceback.
[ python-Bugs-666958 ] repr.repr not always safe
Bugs item #666958, was opened at 2003-01-12 23:17 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=666958&group_id=5470 Category: Python Library Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Bob Ippolito (etrepum) Assigned to: Nobody/Anonymous (nobody) Summary: repr.repr not always safe Initial Comment: >>> class A: ... def __repr__(self): ... raise Exception ... def someMethod(self): ... pass ... >>> A() Traceback (most recent call last): File "", line 1, in ? File " ", line 3, in __repr__ Exception >>> repr.repr(A()) '' >>> repr.repr(A().someMethod) Traceback (most recent call last): File " ", line 1, in ? File "/usr/lib/python2.2/repr.py", line 15, in repr return self.repr1(x, self.maxlevel) File "/usr/lib/python2.2/repr.py", line 24, in repr1 s = `x` File " ", line -- >Comment By: Raymond Hettinger (rhettinger) Date: 2005-03-26 21:50 Message: Logged In: YES user_id=80475 That previous comment should have started with: The behavior of repr.repr(A().someMethod) is the same as __builtin__.repr(A().someMethod) -- Comment By: Raymond Hettinger (rhettinger) Date: 2005-03-26 19:01 Message: Logged In: YES user_id=80475 The behavior of repr.repr(A().someMethod) is the same as `repr (A().someMethod)`. Also, in list example provided, a debugger takes you right to the heart of the OP's issue (resolving a faulty user implementation of __repr__). I believe the current behavior both useful and correct. The offered use case is somewhat rare and better solved with pdb. Changing the current behavior would likely impact someone's existing code (doctests for example). Recommend closing this bug. -- Comment By: Facundo Batista (facundobatista) Date: 2004-11-15 19:10 Message: Logged In: YES user_id=752496 What seems like a bug to me is that "repr.repr(A())" and "repr.repr(A().someMethod)" doesn't have the same behaviour. As this still happens in Py2.3.3, I updated the bug metadata. .Facundo -- Comment By: Neal Norwitz (nnorwitz) Date: 2003-02-06 20:33 Message: Logged In: YES user_id=33168 Bob, I'm not sure I understand what you want (ie, what behaviour you desire). If you could provide a patch to the repr module, that could help me. -- Comment By: Bob Ippolito (etrepum) Date: 2003-02-03 15:27 Message: Logged In: YES user_id=139309 Yes, of course repr.repr is for debugging use.. and it *really* sucks if your debugger crashes! I'm sure that's why it's inconsistent with __builtins__.repr, the bug I'm reporting is that it's inconsistent with itself. Going back to the example of a list again. If you're debugging a list that has one or two items that raise an exception, it's infinitely more useful to know *which* items have a broken __repr__ (along with the context of the valid objects), rather than knowing *one or more* items in the list has a broken __repr__. repr.repr is often called recursively; if something bad happens far down in the call stack it can really be a pain to track down where it happened (all tracebacks to repr are going to look mostly the same) unless repr.repr is fixed to handle this situation. -- Comment By: Christos Georgiou (tzot) Date: 2003-01-21 15:58 Message: Logged In: YES user_id=539787 Sorry, incomplete last post, I clicked in the wrong place (ie SUBMIT) to come back to IE. The case is that repr.py is documented as for debugging use... you suggest that all exceptions are caught, but that would not be consistent with __builtins__.repr . If the module was intended for generic use, I would suggest the opposite; no exceptions should be caught (for consistency). -- Comment By: Christos Georgiou (tzot) Date: 2003-01-21 15:47 Message: Logged In: YES user_id=539787 Do you suggest to remove the exception catching in Repr.repr_instance or adding a try clause in the s = `x` line of Repr.repr1 ? -- Comment By: Bob Ippolito (etrepum) Date: 2003-01-20 09:11 Message: Logged In: YES user_id=139309 It's not my module, it comes with Python (at least in my 2.2, and 2.3 CVS HEAD). Look for yourself! The issue has nothing to do with the fact that it has the same name as the builtin. -- Comment By: Christos Georgiou (tzot) Date: 2003
