[issue3611] invalid exception context

Amaury Forgeot d'Arc <[EMAIL PROTECTED]> added the comment: Committed as r66056. -- resolution: -> fixed status: open -> closed ___ Python tracker <[EMAIL PROTECTED]> ___

[issue3611] invalid exception context

Antoine Pitrou <[EMAIL PROTECTED]> added the comment: > Can this patch go in? I'm ok for it. ___ Python tracker <[EMAIL PROTECTED]> ___ ___ Pyth

[issue3611] invalid exception context

Amaury Forgeot d'Arc <[EMAIL PROTECTED]> added the comment: Some progress: The lines suppressed by the patch at http://bugs.python.org/msg71579 either do nothing (because e.g exc_type is already NULL or None), or happen to be in a case similar to the script "lostcontext2.py" (Most of the time, th

[issue3611] invalid exception context

Benjamin Peterson <[EMAIL PROTECTED]> added the comment: I can't reproduce it either. I suspect that I removed the wrong lines initially and caused other problems. :) Sorry for the false alarm. ___ Python tracker <[EMAIL PROTECTED]>

[issue3611] invalid exception context

Amaury Forgeot d'Arc <[EMAIL PROTECTED]> added the comment: Benjamin, I am testing the patch proposed earlier by Victor in http://bugs.python.org/msg71579 (who might be the correct one, after all), and I don't see the problems you mentioned. I run the test suite without problem, on winXP & Linu

[issue3611] invalid exception context

Antoine Pitrou <[EMAIL PROTECTED]> added the comment: Hmm, answering to myself, I don't think dropping exception normalization would solve the problem, it would just let it occur in a different place... ___ Python tracker <[EMAIL PROTECTED]>

[issue3611] invalid exception context

Antoine Pitrou <[EMAIL PROTECTED]> added the comment: Agreed with Amaury, it's not PyErr_SetObject's job to try to save/restore the tstate->exc_* variables. We'll probably have to live with the small context-losing glitches in 3.0. For 3.1, a radical solution would be to drop the "exception norm

[issue3611] invalid exception context

Amaury Forgeot d'Arc <[EMAIL PROTECTED]> added the comment: Victor, your patch addresses the symptom of the problem, not the cause. The cause is that in some cases, the exception context chain is lost. Of course what could be a minor annoyance becomes dramatic when this precisely happens in code

[issue3611] invalid exception context

Changes by Benjamin Peterson <[EMAIL PROTECTED]>: -- keywords: +needs review ___ Python tracker <[EMAIL PROTECTED]> ___ ___ Python-bugs-

[issue3611] invalid exception context

STINNER Victor <[EMAIL PROTECTED]> added the comment: With my last patch (pyerr_setobject_reentrant-v2.patch) my program works fine! I known that PyErr_SetObject() makes re-entrant calls every ~30 seconds, so the code is tested ;-) ___ Python tracker <[EMAIL

[issue3611] invalid exception context

STINNER Victor <[EMAIL PROTECTED]> added the comment: Ok, it was not enough: exc_{type,value,traceback} have to be saved/restored. So here is a new patch. I doesn't change Python behaviour for previous pitrou snippet (msg71658): it doesn't crash here and display two errors. I think that this

[issue3611] invalid exception context

STINNER Victor <[EMAIL PROTECTED]> added the comment: It's now known that PyErr_SetObject() have to be re-entrant because of the garbage collector interaction. As I wrote in my comments, tstate may be changed during PyEval_CallObject() call. The problem is to known which values have to be prot

[issue3611] invalid exception context

Amaury Forgeot d'Arc <[EMAIL PROTECTED]> added the comment: I found another way to loose the current exception context; see lostcontext2.py Completely removing the block starting with if (why == WHY_EXCEPTION && !throwflag) { corrects the script; but according to Benjamin: """removing th

[issue3611] invalid exception context

Amaury Forgeot d'Arc <[EMAIL PROTECTED]> added the comment: Antoine, your script hangs at the end due to the io.py deadlock (see #3618) - at the end of the script, flush_io() is called - this enters code in io.py - here, garbage collection occurs (thanks to low thresholds) - the Bug() instance i

[issue3611] invalid exception context

STINNER Victor <[EMAIL PROTECTED]> added the comment: @pitrou: Ok, done (issue #3634). We were right, it's a different bug. ___ Python tracker <[EMAIL PROTECTED]> ___ __

[issue3611] invalid exception context

Antoine Pitrou <[EMAIL PROTECTED]> added the comment: Haypo, this is a separate bug I think. Please open a new ticket :) ___ Python tracker <[EMAIL PROTECTED]> ___ _

[issue3611] invalid exception context

STINNER Victor <[EMAIL PROTECTED]> added the comment: Here is a new snippet with strange exception handling: - 8< - from gc import collect import _weakref class FuzzingUserClass: pass obj = _weakref.ref(FuzzingUserClass) # Exception not raised??

[issue3611] invalid exception context

Antoine Pitrou <[EMAIL PROTECTED]> added the comment: On a Windows box, I manage to make the following script reliably hang on a non-debug build of beta3. This can be a good base for further diagnosing. def f(): class Bug: def __del__(self): 1/0 import gc trash

[issue3611] invalid exception context

Changes by Benjamin Peterson <[EMAIL PROTECTED]>: -- resolution: accepted -> status: closed -> open ___ Python tracker <[EMAIL PROTECTED]> ___

[issue3611] invalid exception context

STINNER Victor <[EMAIL PROTECTED]> added the comment: The bug is not closed :-/ With py3k trunk, I still get a crash. So I added a flag to detect inner calls. Here is an example of inner call backtrace: (gdb) where (...) #2 0xb7df4201 in abort () from /lib/tls/i686/cmov/libc.so.6 #3 0x080a23

[issue3611] invalid exception context

Changes by Barry A. Warsaw <[EMAIL PROTECTED]>: -- status: open -> closed ___ Python tracker <[EMAIL PROTECTED]> ___ ___ Python-bugs-lis

[issue3611] invalid exception context

Benjamin Peterson <[EMAIL PROTECTED]> added the comment: Ok. I'll move the test after the betas. ___ Python tracker <[EMAIL PROTECTED]> ___ ___ P

[issue3611] invalid exception context

Benjamin Peterson <[EMAIL PROTECTED]> added the comment: Patch was applied in r65921. ___ Python tracker <[EMAIL PROTECTED]> ___ ___ Python-bugs-

[issue3611] invalid exception context

Amaury Forgeot d'Arc <[EMAIL PROTECTED]> added the comment: Short patch review: __context__ tests are mostly found in test_raise.py (class TestContext) test_exceptions.py seems to only deal with the exception object. ___ Python tracker <[EMAIL PROTECTED]>

[issue3611] invalid exception context

Changes by Barry A. Warsaw <[EMAIL PROTECTED]>: -- resolution: -> accepted ___ Python tracker <[EMAIL PROTECTED]> ___ ___ Python-bugs-l

[issue3611] invalid exception context

Benjamin Peterson <[EMAIL PROTECTED]> added the comment: Here's Amaury's patch and a unittest based on lostcontext.py. -- keywords: +patch Added file: http://bugs.python.org/file11182/3611.patch ___ Python tracker <[EMAIL PROTECTED]>

[issue3611] invalid exception context

Amaury Forgeot d'Arc <[EMAIL PROTECTED]> added the comment: This last proposal does not correct the behaviour of lostcontext.py. ___ Python tracker <[EMAIL PROTECTED]> ___ _

[issue3611] invalid exception context

Antoine Pitrou <[EMAIL PROTECTED]> added the comment: And if it doesn't work, try the following instead: if (why == WHY_EXCEPTION) { PyObject *tmp1, *tmp2, *tmp3; tmp1 = tstate->exc_type; tmp2 = tstate->exc_value; tmp3 = tstate->exc_traceback; tstate->exc_

[issue3611] invalid exception context

Amaury Forgeot d'Arc <[EMAIL PROTECTED]> added the comment: Can you try with the following patch, "throwflag" is specific to a generator being deleted. Index: Python/ceval.c === --- Python/ceval.c (revision 65919) +++ Python/ce

[issue3611] invalid exception context

Benjamin Peterson <[EMAIL PROTECTED]> added the comment: Unfortunately, removing those lines causes a RuntimeError about exceeding the recursion limit for about 1/3 of the tests. -- nosy: +benjamin.peterson ___ Python tracker <[EMAIL PROTECTED]>

[issue3611] invalid exception context

Changes by Barry A. Warsaw <[EMAIL PROTECTED]>: -- priority: -> release blocker ___ Python tracker <[EMAIL PROTECTED]> ___ ___ Python-b

[issue3611] invalid exception context

STINNER Victor <[EMAIL PROTECTED]> added the comment: Great job amaury! So in ceval, here is the block responsible to clear the execution informations: Index: Python/ceval.c === --- Python/ceval.c (révision 65915) +++ Python/c

[issue3611] invalid exception context

STINNER Victor <[EMAIL PROTECTED]> added the comment: It looks that the problem is that PyErr_SetObject() is not re-entrant. The crash occurs when PyErr_SetObject() is called (indirectly) by PyErr_SetObject(): tstate->exc_value value is changed (set to NULL). As noticed by amaury.forgeotdarc, t

[issue3611] invalid exception context

Amaury Forgeot d'Arc <[EMAIL PROTECTED]> added the comment: > Having a snippet deterministically reproducing the problem > would really help in any case. Here you are. Don't ask how I found this. The attached script, when run, prints (KeyError(), ValueError()) (KeyError(), None) The current e

[issue3611] invalid exception context

Antoine Pitrou <[EMAIL PROTECTED]> added the comment: > The problem comes when PyErr_SetObject triggers garbage collection which > runs python code (finalizers...). Mmmh, normally this shouldn't change the value of tstate->exc_value once that Python code returns. That's what exception stacking i

[issue3611] invalid exception context

Amaury Forgeot d'Arc <[EMAIL PROTECTED]> added the comment: The problem comes when PyErr_SetObject triggers garbage collection which runs python code (finalizers...). I could reproduce the crash several times, and each time garbage collection was triggered by the normalization of the exception (

[issue3611] invalid exception context

STINNER Victor <[EMAIL PROTECTED]> added the comment: I'm unable to write a short Python script to reproduce the bug. So download the full program: http://neudorf.hachoir.org/tmp/fusil3000.tar.gz Run in with: $ gdb python3.0 (gdb) run fusil-python --fast __

[issue3611] invalid exception context

Antoine Pitrou <[EMAIL PROTECTED]> added the comment: [...] > > This exception may comes from PyObject_IsSubclass() which calls > PyObject_GetAttr(cls, "__subclasscheck__"). If you look at the code for PyObject_IsSubclass(), there is a pair of PyErr_Fetch / PyErr_Restore calls around PyObject_Ge

[issue3611] invalid exception context

STINNER Victor <[EMAIL PROTECTED]> added the comment: About the PyEval_CallObject() call in errors.c, here is an example: Call exception<0x81dcee0>(args<0x8751dc4>) with exception= object : type: type refcount: 6 address : 0x81dcee0 and args= object : ("type object 'ProcE

[issue3611] invalid exception context

Antoine Pitrou <[EMAIL PROTECTED]> added the comment: I retract what I said above. PyErr_SetObject() and friends only change tstate->curexc_* variables, not tstate->exc_*. The former (tstate->curexc_*) contain the current, pending (uncaught) exception state of the thread. The latter (tstate->exc

[issue3611] invalid exception context

Antoine Pitrou <[EMAIL PROTECTED]> added the comment: Selon STINNER Victor <[EMAIL PROTECTED]>: > > > How can tstate->exc_value become NULL at that point? [error.c:86] > > Could you investigate? > > PyEval_CallObject(exception, args) may calls PyErr_SetObject(). Since > the same thread state is t

[issue3611] invalid exception context

STINNER Victor <[EMAIL PROTECTED]> added the comment: > How can tstate->exc_value become NULL at that point? [error.c:86] > Could you investigate? PyEval_CallObject(exception, args) may calls PyErr_SetObject(). Since the same thread state is the same, tstate->exc_value also changes during this

[issue3611] invalid exception context

Antoine Pitrou <[EMAIL PROTECTED]> added the comment: > I also noticed a crash in PyErr_SetObject(): in block , > tstate->exc_value value may changes and so it's possible that > tstate->exc_value becomes NULL. How can tstate->exc_value become NULL at that point? Could you investigate? --

[issue3611] invalid exception context

STINNER Victor <[EMAIL PROTECTED]> added the comment: I also noticed a crash in PyErr_SetObject(): in block , tstate->exc_value value may changes and so it's possible that tstate->exc_value becomes NULL. I added a test to avoid this crash: Index: Python/errors.c ===

[issue3611] invalid exception context

STINNER Victor <[EMAIL PROTECTED]> added the comment: The crash is on ProcError exception raising (line 21): 20 except IOError as err: 21 raise ProcError("Unable to open %r: %s" % (filename, --- (gdb) where #0 0x0805bcc7 in PyObject_Hash (v=0x8691034) at Objects/object.

[issue3611] invalid exception context

New submission from STINNER Victor <[EMAIL PROTECTED]>: After few seconds (30 sec to 5 minutes), my program (Fusil) crashs at: PyEval_EvalFrameEx (f=0x85b4324, throwflag=0) at Python/ceval.c:2459 Py_CLEAR(tstate->exc_traceback); It crashs because tstate->exc_traceback points to a zombi obje