[ python-Bugs-1579370 ] Segfault provoked by generators and exceptions
Bugs item #1579370, was opened at 2006-10-18 04:23 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1579370&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Interpreter Core Group: Python 2.5 >Status: Closed >Resolution: Fixed Priority: 9 Private: No Submitted By: Mike Klaas (mklaas) Assigned to: Nobody/Anonymous (nobody) Summary: Segfault provoked by generators and exceptions Initial Comment: A reproducible segfault when using heavily-nested generators and exceptions. Unfortunately, I haven't yet been able to provoke this behaviour with a standalone python2.5 script. There are, however, no third-party c extensions running in the process so I'm fairly confident that it is a problem in the core. The gist of the code is a series of nested generators which leave scope when an exception is raised. This exception is caught and re-raised in an outer loop. The old exception was holding on to the frame which was keeping the generators alive, and the sequence of generator destruction and new finalization caused the segfault. -- >Comment By: Martin v. Löwis (loewis) Date: 2007-01-23 22:13 Message: Logged In: YES user_id=21627 Originator: NO This is now fixed in r53531 and r53532. For the trunk, it is likely that f_tstate will get eliminated altogether in the near future. People who had the problem are really encouraged to test 2.5.1c1 when it is released. -- Comment By: Andrew Waters (awaters) Date: 2007-01-22 09:46 Message: Logged In: YES user_id=1418249 Originator: NO A quick test on code that always segfaulted with unpatched Python 2.5 seems to work. Needs more extensive testing... -- Comment By: Martin v. Löwis (loewis) Date: 2007-01-22 08:51 Message: Logged In: YES user_id=21627 Originator: NO I don't like mklaas' patch, since I think it is conceptually wrong to have PyTraceBack_Here() use the frame's thread state (mklaas describes it as dirty, and I agree). I'm proposing an alternative patch (tr.diff); please test this as well. File Added: tr.diff -- Comment By: Neal Norwitz (nnorwitz) Date: 2007-01-17 08:01 Message: Logged In: YES user_id=33168 Originator: NO Bumping priority to see if this should go into 2.5.1. -- Comment By: Martin v. Löwis (loewis) Date: 2007-01-04 11:42 Message: Logged In: YES user_id=21627 Originator: NO Why do frame objects have a thread state in the first place? In particular, why does PyTraceBack_Here get the thread state from the frame, instead of using the current thread? Introduction of f_tstate goes back to r7882, but it is not clear why it was done that way. -- Comment By: Andrew Waters (awaters) Date: 2007-01-04 10:35 Message: Logged In: YES user_id=1418249 Originator: NO This fixes the segfault problem that I was able to reliably reproduce on Linux. We need to get this applied (assuming it is the correct fix) to the source to make Python 2.5 usable for me in production code. -- Comment By: Mike Klaas (mklaas) Date: 2006-11-27 19:41 Message: Logged In: YES user_id=1611720 Originator: YES The following patch resets the thread state of the generator when it is resumed, which prevents the segfault for me: Index: Objects/genobject.c === --- Objects/genobject.c (revision 52849) +++ Objects/genobject.c (working copy) @@ -77,6 +77,7 @@ Py_XINCREF(tstate->frame); assert(f->f_back == NULL); f->f_back = tstate->frame; +f->f_tstate = tstate; gen->gi_running = 1; result = PyEval_EvalFrameEx(f, exc); -- Comment By: Eric Noyau (eric_noyau) Date: 2006-11-27 19:07 Message: Logged In: YES user_id=1388768 Originator: NO We are experiencing the same segfault in our application, reliably. Running our unit test suite just segfault everytime on both Linux and Mac OS X. Applying Martin's patch fixes the segfault, and makes everything fine and dandy, at the cost of some memory leaks if I understand properly. This particular bug prevents us to upgrade to python 2.5 in production. -- Comment By: Tim Peters (tim_one) Date: 2006-10-28 07:18 Message: Logged In: YES user_id=31435 > I tried Tim'
[ python-Bugs-1377858 ] segfaults when using __del__ and weakrefs
Bugs item #1377858, was opened at 2005-12-10 22:20 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1377858&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Interpreter Core Group: Python 2.5 Status: Open >Resolution: Accepted Priority: 9 Private: No Submitted By: Carl Friedrich Bolz (cfbolz) >Assigned to: Brett Cannon (bcannon) Summary: segfaults when using __del__ and weakrefs Initial Comment: You can segfault Python by creating a weakref to an object in its __del__ method, storing it somewhere and then trying to dereference the weakref afterwards. the attached file shows the described behaviour. -- >Comment By: Martin v. Löwis (loewis) Date: 2007-01-23 22:26 Message: Logged In: YES user_id=21627 Originator: NO The first comment has a non-sensical (to me) phrase: "rely on part of theof the object". Otherwise, it looks fine to me. Please apply, if you can, before 2.5c1. -- Comment By: Brett Cannon (bcannon) Date: 2007-01-17 19:38 Message: Logged In: YES user_id=357491 Originator: NO I have just been waiting on someone to do a final code review on it. As soon as someone else signs off I will commit it. -- Comment By: Neal Norwitz (nnorwitz) Date: 2007-01-17 08:02 Message: Logged In: YES user_id=33168 Originator: NO Brett, Michael, Armin, can we get this patch checked in for 2.5.1? -- Comment By: Brett Cannon (bcannon) Date: 2006-08-20 06:31 Message: Logged In: YES user_id=357491 After finally figuring out where *list was made NULL (and adding a comment about it where it occurs), I added a test to test_weakref.py . Didn't try to tackle classic classes. -- Comment By: Armin Rigo (arigo) Date: 2006-08-12 13:31 Message: Logged In: YES user_id=4771 The clear_weakref(*list) only clears the first weakref to the object. You need a while loop in your patch. (attached proposed fix) Now we're left with fixing the same bug in old-style classes (surprize surprize), and turning the crasher into a test. -- Comment By: Brett Cannon (bcannon) Date: 2006-06-29 19:36 Message: Logged In: YES user_id=357491 So after staring at this crasher it seemed to me to be that clearing the new weakrefs w/o calling their finalizers after calling the object's finalizer was the best solution. I couldn't think of any other good way to communicate to the new weakrefs that the object they refer to was no longer viable memory without doing clear_weakref() work by hand. Attached is a patch to do this. Michael, can you have a look? -- Comment By: Georg Brandl (birkenfeld) Date: 2006-01-10 20:29 Message: Logged In: YES user_id=1188172 Added to outstanding_crashes.py. -- Comment By: Michael Hudson (mwh) Date: 2006-01-09 12:58 Message: Logged In: YES user_id=6656 Hmm, maybe the referenced mayhem is more to do with clearing __dict__ than calling __del__. What breaks if we do things in this order: 1. call __del__ 2. clear weakrefs 3. clear __dict__ ? -- Comment By: Michael Hudson (mwh) Date: 2006-01-09 12:54 Message: Logged In: YES user_id=6656 Hmm, I was kind of hoping this report would get more attention. The problem is obvious if you read typeobject.c around line 660: the weakref list is cleared before __del__ is called, so any weakrefs added during the execution of __del__ are never informed of the object's death. One fix for this would be to clear the weakref list _after_ calling __del__ but that led to other mayhem in ways I haven't boethered to understand (see SF bug #742911). I guess we could just clear out any weakrefs created in __del__ without calling their callbacks. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1377858&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1377858 ] segfaults when using __del__ and weakrefs
Bugs item #1377858, was opened at 2005-12-10 13:20 Message generated for change (Comment added) made by bcannon You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1377858&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Interpreter Core Group: Python 2.5 >Status: Closed Resolution: Accepted Priority: 9 Private: No Submitted By: Carl Friedrich Bolz (cfbolz) Assigned to: Brett Cannon (bcannon) Summary: segfaults when using __del__ and weakrefs Initial Comment: You can segfault Python by creating a weakref to an object in its __del__ method, storing it somewhere and then trying to dereference the weakref afterwards. the attached file shows the described behaviour. -- >Comment By: Brett Cannon (bcannon) Date: 2007-01-23 15:22 Message: Logged In: YES user_id=357491 Originator: NO rev. 53533 (for 25-maint) and rev. 53535 (trunk) have the patch with an improved comment. Py3K should eventually have its crasher file for this test deleted since classic classes will no longer be an issue. -- Comment By: Martin v. Löwis (loewis) Date: 2007-01-23 13:26 Message: Logged In: YES user_id=21627 Originator: NO The first comment has a non-sensical (to me) phrase: "rely on part of theof the object". Otherwise, it looks fine to me. Please apply, if you can, before 2.5c1. -- Comment By: Brett Cannon (bcannon) Date: 2007-01-17 10:38 Message: Logged In: YES user_id=357491 Originator: NO I have just been waiting on someone to do a final code review on it. As soon as someone else signs off I will commit it. -- Comment By: Neal Norwitz (nnorwitz) Date: 2007-01-16 23:02 Message: Logged In: YES user_id=33168 Originator: NO Brett, Michael, Armin, can we get this patch checked in for 2.5.1? -- Comment By: Brett Cannon (bcannon) Date: 2006-08-19 21:31 Message: Logged In: YES user_id=357491 After finally figuring out where *list was made NULL (and adding a comment about it where it occurs), I added a test to test_weakref.py . Didn't try to tackle classic classes. -- Comment By: Armin Rigo (arigo) Date: 2006-08-12 04:31 Message: Logged In: YES user_id=4771 The clear_weakref(*list) only clears the first weakref to the object. You need a while loop in your patch. (attached proposed fix) Now we're left with fixing the same bug in old-style classes (surprize surprize), and turning the crasher into a test. -- Comment By: Brett Cannon (bcannon) Date: 2006-06-29 10:36 Message: Logged In: YES user_id=357491 So after staring at this crasher it seemed to me to be that clearing the new weakrefs w/o calling their finalizers after calling the object's finalizer was the best solution. I couldn't think of any other good way to communicate to the new weakrefs that the object they refer to was no longer viable memory without doing clear_weakref() work by hand. Attached is a patch to do this. Michael, can you have a look? -- Comment By: Georg Brandl (birkenfeld) Date: 2006-01-10 11:29 Message: Logged In: YES user_id=1188172 Added to outstanding_crashes.py. -- Comment By: Michael Hudson (mwh) Date: 2006-01-09 03:58 Message: Logged In: YES user_id=6656 Hmm, maybe the referenced mayhem is more to do with clearing __dict__ than calling __del__. What breaks if we do things in this order: 1. call __del__ 2. clear weakrefs 3. clear __dict__ ? -- Comment By: Michael Hudson (mwh) Date: 2006-01-09 03:54 Message: Logged In: YES user_id=6656 Hmm, I was kind of hoping this report would get more attention. The problem is obvious if you read typeobject.c around line 660: the weakref list is cleared before __del__ is called, so any weakrefs added during the execution of __del__ are never informed of the object's death. One fix for this would be to clear the weakref list _after_ calling __del__ but that led to other mayhem in ways I haven't boethered to understand (see SF bug #742911). I guess we could just clear out any weakrefs created in __del__ without calling their callbacks. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1377858&gr
[ python-Bugs-1643150 ] Grammatical Error
Bugs item #1643150, was opened at 2007-01-23 20:22 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1643150&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Documentation Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Steve Miller (stmille) Assigned to: Nobody/Anonymous (nobody) Summary: Grammatical Error Initial Comment: http://docs.python.org/tut/node10.html s/One my also/One may also/ -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1643150&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1643150 ] Grammatical Error
Bugs item #1643150, was opened at 2007-01-24 13:22 Message generated for change (Comment added) made by quiver You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1643150&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Documentation Group: None >Status: Closed Resolution: None Priority: 5 Private: No Submitted By: Steve Miller (stmille) Assigned to: Nobody/Anonymous (nobody) Summary: Grammatical Error Initial Comment: http://docs.python.org/tut/node10.html s/One my also/One may also/ -- >Comment By: George Yoshida (quiver) Date: 2007-01-24 14:46 Message: Logged In: YES user_id=671362 Originator: NO Thank for the report. But this is already fixed in svn trunk. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1643150&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
