[ python-Bugs-1579477 ] Use flush() before os.exevp()
Bugs item #1579477, was opened at 2006-10-18 08:04 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1579477&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Documentation Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: Thomas Guettler (guettli) Assigned to: Nobody/Anonymous (nobody) Summary: Use flush() before os.exevp() Initial Comment: Hi, before using one of the os.exec??? functions you should flush all open file descriptors. (E.g. sys.stdout.flush()). If you don't the content in the buffer will get lost. According to Fredrik Lundh this is a feature, not a bug. I think the documentation should include a hint: """ path must contain an appropriate absolute or relative path. NEW The current process gets replaces immediately. Open file descriptors are not flushed. You should flush all open file descriptors (e.g. sys.stdout.flush()) before calling one if the above exec functions. """ -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1579477&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1579796 ] Wrong syntax for PyDateTime_IMPORT in documentation
Bugs item #1579796, was opened at 2006-10-18 17:44 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1579796&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Documentation Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: David Faure (dfaure_kde) Assigned to: Nobody/Anonymous (nobody) Summary: Wrong syntax for PyDateTime_IMPORT in documentation Initial Comment: http://docs.python.org/api/datetime-objects.html says "macro PyDateTime_IMPORT() must be invoked." But this doesn't compile. It's PyDateTime_IMPORT; and not PyDateTime_IMPORT(); Also it would be useful to tell if this should be done once per thread or just once per process; seeing as it modifies a static variable I guess the latter. Cheers, David (still getting crashes in PyDateTime_FromDateAndTime though but that's another topic) -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1579796&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1579931 ] not configured for tk
Bugs item #1579931, was opened at 2006-10-18 11:59 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1579931&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Demos and Tools Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Carl Wenrich (carlwenrich) Assigned to: Nobody/Anonymous (nobody) Summary: not configured for tk Initial Comment: When I try to run the sample tkinter script (hello.py) I get a message saying my python isn't configured for tk. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1579931&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1579370 ] Segfault provoked by generators and exceptions
Bugs item #1579370, was opened at 2006-10-17 19:23
Message generated for change (Comment added) made by mklaas
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1579370&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: Python 2.5
Status: Open
Resolution: None
Priority: 7
Submitted By: Mike Klaas (mklaas)
Assigned to: Nobody/Anonymous (nobody)
Summary: Segfault provoked by generators and exceptions
Initial Comment:
A reproducible segfault when using heavily-nested
generators and exceptions.
Unfortunately, I haven't yet been able to provoke this
behaviour with a standalone python2.5 script. There
are, however, no third-party c extensions running in
the process so I'm fairly confident that it is a
problem in the core.
The gist of the code is a series of nested generators
which leave scope when an exception is raised. This
exception is caught and re-raised in an outer loop.
The old exception was holding on to the frame which was
keeping the generators alive, and the sequence of
generator destruction and new finalization caused the
segfault.
--
>Comment By: Mike Klaas (mklaas)
Date: 2006-10-18 12:37
Message:
Logged In: YES
user_id=1611720
I've produced a simplified traceback with a single generator
. Note the frame being used in the traceback (#0) is the
same frame being dealloc'd (#11).
The relevant call in traceback.c is:
PyTraceBack_Here(PyFrameObject *frame)
{
PyThreadState *tstate = frame->f_tstate;
PyTracebackObject *oldtb = (PyTracebackObject *)
tstate->curexc_traceback;
PyTracebackObject *tb = newtracebackobject(oldtb,
frame);
and I can verify that oldtb contains garbage:
(gdb) print frame
$1 = (PyFrameObject *) 0x8964d94
(gdb) print frame->f_tstate
$2 = (PyThreadState *) 0x895b178
(gdb) print $2->curexc_traceback
$3 = (PyObject *) 0x66
#0 0x080e4296 in PyTraceBack_Here (frame=0x8964d94) at
Python/traceback.c:94
#1 0x080b9ab7 in PyEval_EvalFrameEx (f=0x8964d94,
throwflag=1) at Python/ceval.c:2459
#2 0x08101a40 in gen_send_ex (gen=0xb7cca4ac,
arg=0x81333e0, exc=1) at Objects/genobject.c:82
#3 0x08101c0f in gen_close (gen=0xb7cca4ac, args=0x0) at
Objects/genobject.c:128
#4 0x08101cde in gen_del (self=0xb7cca4ac) at
Objects/genobject.c:163
#5 0x0810195b in gen_dealloc (gen=0xb7cca4ac) at
Objects/genobject.c:31
#6 0x080815b9 in dict_dealloc (mp=0xb7cc913c) at
Objects/dictobject.c:801
#7 0x080927b2 in subtype_dealloc (self=0xb7cca76c) at
Objects/typeobject.c:686
#8 0x0806028d in instancemethod_dealloc (im=0xb7d07f04) at
Objects/classobject.c:2285
#9 0x080815b9 in dict_dealloc (mp=0xb7cc90b4) at
Objects/dictobject.c:801
#10 0x080927b2 in subtype_dealloc (self=0xb7cca86c) at
Objects/typeobject.c:686
#11 0x081028c5 in frame_dealloc (f=0x8964a94) at
Objects/frameobject.c:416
#12 0x080e41b1 in tb_dealloc (tb=0xb7cc1fcc) at
Python/traceback.c:34
#13 0x080e41c2 in tb_dealloc (tb=0xb7cc1f7c) at
Python/traceback.c:33
#14 0x08080dca in insertdict (mp=0xb7f99824, key=0xb7ccd020,
hash=1492466088, value=0xb7ccd054)
at Objects/dictobject.c:394
#15 0x080811a4 in PyDict_SetItem (op=0xb7f99824,
key=0xb7ccd020, value=0xb7ccd054)
at Objects/dictobject.c:619
#16 0x08082dc6 in PyDict_SetItemString (v=0xb7f99824,
key=0x8129284 "exc_traceback",
item=0xb7ccd054) at Objects/dictobject.c:2103
#17 0x080e2837 in PySys_SetObject (name=0x8129284
"exc_traceback", v=0xb7ccd054)
at Python/sysmodule.c:82
#18 0x080bc9e5 in PyEval_EvalFrameEx (f=0x895f934,
throwflag=0) at Python/ceval.c:2954
---Type to continue, or q to quit---
#19 0x080bfda3 in PyEval_EvalCodeEx (co=0xb7f6ade8,
globals=0xb7fafa44, locals=0x0,
args=0xb7cc5ff8, argcount=1, kws=0x0, kwcount=0,
defs=0x0, defcount=0, closure=0x0)
at Python/ceval.c:2833
#20 0x08104083 in function_call (func=0xb7cc7294,
arg=0xb7cc5fec, kw=0x0)
at Objects/funcobject.c:517
#21 0x0805a660 in PyObject_Call (func=0xb7cc7294,
arg=0xb7cc5fec, kw=0x0)
at Objects/abstract.c:1860
--
Comment By: Mike Klaas (mklaas)
Date: 2006-10-17 19:23
Message:
Logged In: YES
user_id=1611720
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread -1208400192 (LWP 26235)]
0x080e4296 in PyTraceBack_Here (frame=0x9c2d7b4) at
Python/traceback.c:94
94 if ((next != NULL &&
!PyTraceBack_Check(next)) ||
(gdb) bt
#0 0x080e4296 in PyTraceBack_Here (frame=0x9c2d7b4) at
Python/traceback.c:94
#1 0x080b9ab7 in PyEval_EvalFrameEx (f=0x9c2d7b4,
throwflag=1) at Python/ceval.c:2459
#2 0x08101a40 in gen_send_ex (gen=0xb64f880c,
arg=0x81333e0, exc=1) at Objects/genobject.c:82
#3 0x08101c0f in gen_close (gen=0xb64f880c, args=0x0) at
Objects/genobject.c:128
#4 0x08101cde in ge
[ python-Bugs-1579370 ] Segfault provoked by generators and exceptions
Bugs item #1579370, was opened at 2006-10-17 19:23
Message generated for change (Comment added) made by mklaas
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1579370&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: Python 2.5
Status: Open
Resolution: None
Priority: 7
Submitted By: Mike Klaas (mklaas)
Assigned to: Nobody/Anonymous (nobody)
Summary: Segfault provoked by generators and exceptions
Initial Comment:
A reproducible segfault when using heavily-nested
generators and exceptions.
Unfortunately, I haven't yet been able to provoke this
behaviour with a standalone python2.5 script. There
are, however, no third-party c extensions running in
the process so I'm fairly confident that it is a
problem in the core.
The gist of the code is a series of nested generators
which leave scope when an exception is raised. This
exception is caught and re-raised in an outer loop.
The old exception was holding on to the frame which was
keeping the generators alive, and the sequence of
generator destruction and new finalization caused the
segfault.
--
>Comment By: Mike Klaas (mklaas)
Date: 2006-10-18 12:47
Message:
Logged In: YES
user_id=1611720
I cannot yet produce an only-python script which reproduces
the problem, but I can give an overview. There is a
generator running in one thread, an exception being raised
in another thread, and as a consequent, the generator in the
first thread is garbage-collected (triggering an exception
due to the new generator cleanup). The problem is extremely
sensitive to timing--often the insertion/removal of print
statements, or reordering the code, causes the problem to
vanish, which is confounding my ability to create a simple
test script.
def getdocs():
def f():
while True:
f()
yield None
#
-
class B(object):
def __init__(self,):
pass
def doit(self):
# must be an instance var to trigger segfault
self.docIter = getdocs()
print self.docIter # this is the generator
referred-to in the traceback
for i, item in enumerate(self.docIter):
if i > 9:
break
print 'exiting generator'
class A(object):
""" Process entry point / main thread """
def __init__(self):
while True:
try:
self.func()
except Exception, e:
print 'right after raise'
def func(self):
b = B()
thread = threading.Thread(target=b.doit)
thread.start()
start_t = time.time()
while True:
try:
if time.time() - start_t > 1:
raise Exception
except Exception:
print 'right before raise'
# SIGSEGV here. If this is changed to
# 'break', no segfault occurs
raise
if __name__ == '__main__':
A()
--
Comment By: Mike Klaas (mklaas)
Date: 2006-10-18 12:37
Message:
Logged In: YES
user_id=1611720
I've produced a simplified traceback with a single generator
. Note the frame being used in the traceback (#0) is the
same frame being dealloc'd (#11).
The relevant call in traceback.c is:
PyTraceBack_Here(PyFrameObject *frame)
{
PyThreadState *tstate = frame->f_tstate;
PyTracebackObject *oldtb = (PyTracebackObject *)
tstate->curexc_traceback;
PyTracebackObject *tb = newtracebackobject(oldtb,
frame);
and I can verify that oldtb contains garbage:
(gdb) print frame
$1 = (PyFrameObject *) 0x8964d94
(gdb) print frame->f_tstate
$2 = (PyThreadState *) 0x895b178
(gdb) print $2->curexc_traceback
$3 = (PyObject *) 0x66
#0 0x080e4296 in PyTraceBack_Here (frame=0x8964d94) at
Python/traceback.c:94
#1 0x080b9ab7 in PyEval_EvalFrameEx (f=0x8964d94,
throwflag=1) at Python/ceval.c:2459
#2 0x08101a40 in gen_send_ex (gen=0xb7cca4ac,
arg=0x81333e0, exc=1) at Objects/genobject.c:82
#3 0x08101c0f in gen_close (gen=0xb7cca4ac, args=0x0) at
Objects/genobject.c:128
#4 0x08101cde in gen_del (self=0xb7cca4ac) at
Objects/genobject.c:163
#5 0x0810195b in gen_dealloc (gen=0xb7cca4ac) at
Objects/genobject.c:31
#6 0x080815b9 in dict_dealloc (mp=0xb7cc913c) at
Objects/dictobject.c:801
#7 0x080927b2 in subtype_dealloc (self=0xb7cca76c) at
Objects/typeobject.c:686
#8 0x0806028d in instancemethod_dealloc (im=0xb7d07f04) at
Objects/classobject.c:2285
#9 0x080815b9 in dict_dealloc (mp=0xb7cc90b4) at
Objects/dictobject.c:801
#10 0x080927b2 in subtype_dealloc (self=0xb7cca86c) at
Objects/typeobject.c:686
#11 0x081028c5 in
[ python-Bugs-1579370 ] Segfault provoked by generators and exceptions
Bugs item #1579370, was opened at 2006-10-18 04:23
Message generated for change (Comment added) made by loewis
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1579370&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: Python 2.5
Status: Open
Resolution: None
Priority: 7
Submitted By: Mike Klaas (mklaas)
Assigned to: Nobody/Anonymous (nobody)
Summary: Segfault provoked by generators and exceptions
Initial Comment:
A reproducible segfault when using heavily-nested
generators and exceptions.
Unfortunately, I haven't yet been able to provoke this
behaviour with a standalone python2.5 script. There
are, however, no third-party c extensions running in
the process so I'm fairly confident that it is a
problem in the core.
The gist of the code is a series of nested generators
which leave scope when an exception is raised. This
exception is caught and re-raised in an outer loop.
The old exception was holding on to the frame which was
keeping the generators alive, and the sequence of
generator destruction and new finalization caused the
segfault.
--
>Comment By: Martin v. Löwis (loewis)
Date: 2006-10-18 23:05
Message:
Logged In: YES
user_id=21627
Can you please review/try attached patch? Can anybody tell
why gi_frame *isn't* incref'ed when the generator is created?
--
Comment By: Mike Klaas (mklaas)
Date: 2006-10-18 21:47
Message:
Logged In: YES
user_id=1611720
I cannot yet produce an only-python script which reproduces
the problem, but I can give an overview. There is a
generator running in one thread, an exception being raised
in another thread, and as a consequent, the generator in the
first thread is garbage-collected (triggering an exception
due to the new generator cleanup). The problem is extremely
sensitive to timing--often the insertion/removal of print
statements, or reordering the code, causes the problem to
vanish, which is confounding my ability to create a simple
test script.
def getdocs():
def f():
while True:
f()
yield None
#
-
class B(object):
def __init__(self,):
pass
def doit(self):
# must be an instance var to trigger segfault
self.docIter = getdocs()
print self.docIter # this is the generator
referred-to in the traceback
for i, item in enumerate(self.docIter):
if i > 9:
break
print 'exiting generator'
class A(object):
""" Process entry point / main thread """
def __init__(self):
while True:
try:
self.func()
except Exception, e:
print 'right after raise'
def func(self):
b = B()
thread = threading.Thread(target=b.doit)
thread.start()
start_t = time.time()
while True:
try:
if time.time() - start_t > 1:
raise Exception
except Exception:
print 'right before raise'
# SIGSEGV here. If this is changed to
# 'break', no segfault occurs
raise
if __name__ == '__main__':
A()
--
Comment By: Mike Klaas (mklaas)
Date: 2006-10-18 21:37
Message:
Logged In: YES
user_id=1611720
I've produced a simplified traceback with a single generator
. Note the frame being used in the traceback (#0) is the
same frame being dealloc'd (#11).
The relevant call in traceback.c is:
PyTraceBack_Here(PyFrameObject *frame)
{
PyThreadState *tstate = frame->f_tstate;
PyTracebackObject *oldtb = (PyTracebackObject *)
tstate->curexc_traceback;
PyTracebackObject *tb = newtracebackobject(oldtb,
frame);
and I can verify that oldtb contains garbage:
(gdb) print frame
$1 = (PyFrameObject *) 0x8964d94
(gdb) print frame->f_tstate
$2 = (PyThreadState *) 0x895b178
(gdb) print $2->curexc_traceback
$3 = (PyObject *) 0x66
#0 0x080e4296 in PyTraceBack_Here (frame=0x8964d94) at
Python/traceback.c:94
#1 0x080b9ab7 in PyEval_EvalFrameEx (f=0x8964d94,
throwflag=1) at Python/ceval.c:2459
#2 0x08101a40 in gen_send_ex (gen=0xb7cca4ac,
arg=0x81333e0, exc=1) at Objects/genobject.c:82
#3 0x08101c0f in gen_close (gen=0xb7cca4ac, args=0x0) at
Objects/genobject.c:128
#4 0x08101cde in gen_del (self=0xb7cca4ac) at
Objects/genobject.c:163
#5 0x0810195b in gen_dealloc (gen=0xb7cca4ac) at
Objects/genobject.c:31
#6 0x080815b9 in dict_dealloc (mp=0xb7cc913c) at
Objects/dictobject.c:801
#7 0x080927b2 in subtype_dealloc (self=0xb7
[ python-Bugs-1579931 ] not configured for tk
Bugs item #1579931, was opened at 2006-10-18 20:59 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1579931&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Demos and Tools Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Carl Wenrich (carlwenrich) Assigned to: Nobody/Anonymous (nobody) Summary: not configured for tk Initial Comment: When I try to run the sample tkinter script (hello.py) I get a message saying my python isn't configured for tk. -- >Comment By: Martin v. Löwis (loewis) Date: 2006-10-18 23:06 Message: Logged In: YES user_id=21627 Why do you think this is a bug? What operating system are you using, and where do your Python binaries come from? -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1579931&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1579370 ] Segfault provoked by generators and exceptions
Bugs item #1579370, was opened at 2006-10-17 22:23
Message generated for change (Comment added) made by tim_one
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1579370&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: Python 2.5
Status: Open
Resolution: None
Priority: 7
Submitted By: Mike Klaas (mklaas)
Assigned to: Nobody/Anonymous (nobody)
Summary: Segfault provoked by generators and exceptions
Initial Comment:
A reproducible segfault when using heavily-nested
generators and exceptions.
Unfortunately, I haven't yet been able to provoke this
behaviour with a standalone python2.5 script. There
are, however, no third-party c extensions running in
the process so I'm fairly confident that it is a
problem in the core.
The gist of the code is a series of nested generators
which leave scope when an exception is raised. This
exception is caught and re-raised in an outer loop.
The old exception was holding on to the frame which was
keeping the generators alive, and the sequence of
generator destruction and new finalization caused the
segfault.
--
>Comment By: Tim Peters (tim_one)
Date: 2006-10-18 17:57
Message:
Logged In: YES
user_id=31435
> Can anybody tell why gi_frame *isn't* incref'ed when
> the generator is created?
As documented (in concrete.tex), PyGen_New(f) steals a
reference to the frame passed to it. Its only call site
(well, in the core) is in ceval.c, which returns immediately
after PyGen_New takes over ownership of the frame the caller
created:
"""
/* Create a new generator that owns the ready to run frame
* and return that as the value. */
return PyGen_New(f);
"""
In short, that PyGen_New() doesn't incref the frame passed
to it is intentional.
It's possible that the intent is flawed ;-), but offhand I
don't see how.
--
Comment By: Martin v. Löwis (loewis)
Date: 2006-10-18 17:05
Message:
Logged In: YES
user_id=21627
Can you please review/try attached patch? Can anybody tell
why gi_frame *isn't* incref'ed when the generator is created?
--
Comment By: Mike Klaas (mklaas)
Date: 2006-10-18 15:47
Message:
Logged In: YES
user_id=1611720
I cannot yet produce an only-python script which reproduces
the problem, but I can give an overview. There is a
generator running in one thread, an exception being raised
in another thread, and as a consequent, the generator in the
first thread is garbage-collected (triggering an exception
due to the new generator cleanup). The problem is extremely
sensitive to timing--often the insertion/removal of print
statements, or reordering the code, causes the problem to
vanish, which is confounding my ability to create a simple
test script.
def getdocs():
def f():
while True:
f()
yield None
#
-
class B(object):
def __init__(self,):
pass
def doit(self):
# must be an instance var to trigger segfault
self.docIter = getdocs()
print self.docIter # this is the generator
referred-to in the traceback
for i, item in enumerate(self.docIter):
if i > 9:
break
print 'exiting generator'
class A(object):
""" Process entry point / main thread """
def __init__(self):
while True:
try:
self.func()
except Exception, e:
print 'right after raise'
def func(self):
b = B()
thread = threading.Thread(target=b.doit)
thread.start()
start_t = time.time()
while True:
try:
if time.time() - start_t > 1:
raise Exception
except Exception:
print 'right before raise'
# SIGSEGV here. If this is changed to
# 'break', no segfault occurs
raise
if __name__ == '__main__':
A()
--
Comment By: Mike Klaas (mklaas)
Date: 2006-10-18 15:37
Message:
Logged In: YES
user_id=1611720
I've produced a simplified traceback with a single generator
. Note the frame being used in the traceback (#0) is the
same frame being dealloc'd (#11).
The relevant call in traceback.c is:
PyTraceBack_Here(PyFrameObject *frame)
{
PyThreadState *tstate = frame->f_tstate;
PyTracebackObject *oldtb = (PyTracebackObject *)
tstate->curexc_traceback;
PyTracebackObject *tb = newtracebackobject(oldtb,
frame);
and I can verify that oldtb contains garba
[ python-Bugs-1579370 ] Segfault provoked by generators and exceptions
Bugs item #1579370, was opened at 2006-10-17 19:23 Message generated for change (Comment added) made by mklaas You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1579370&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Interpreter Core Group: Python 2.5 Status: Open Resolution: None Priority: 7 Submitted By: Mike Klaas (mklaas) Assigned to: Nobody/Anonymous (nobody) Summary: Segfault provoked by generators and exceptions Initial Comment: A reproducible segfault when using heavily-nested generators and exceptions. Unfortunately, I haven't yet been able to provoke this behaviour with a standalone python2.5 script. There are, however, no third-party c extensions running in the process so I'm fairly confident that it is a problem in the core. The gist of the code is a series of nested generators which leave scope when an exception is raised. This exception is caught and re-raised in an outer loop. The old exception was holding on to the frame which was keeping the generators alive, and the sequence of generator destruction and new finalization caused the segfault. -- >Comment By: Mike Klaas (mklaas) Date: 2006-10-18 17:12 Message: Logged In: YES user_id=1611720 Despite Tim's reassurrance, I'm afraid that Martin's patch does infact prevent the segfault. Sounds like it also introduces a memleak. -- Comment By: Tim Peters (tim_one) Date: 2006-10-18 14:57 Message: Logged In: YES user_id=31435 > Can anybody tell why gi_frame *isn't* incref'ed when > the generator is created? As documented (in concrete.tex), PyGen_New(f) steals a reference to the frame passed to it. Its only call site (well, in the core) is in ceval.c, which returns immediately after PyGen_New takes over ownership of the frame the caller created: """ /* Create a new generator that owns the ready to run frame * and return that as the value. */ return PyGen_New(f); """ In short, that PyGen_New() doesn't incref the frame passed to it is intentional. It's possible that the intent is flawed ;-), but offhand I don't see how. -- Comment By: Martin v. Löwis (loewis) Date: 2006-10-18 14:05 Message: Logged In: YES user_id=21627 Can you please review/try attached patch? Can anybody tell why gi_frame *isn't* incref'ed when the generator is created? -- Comment By: Mike Klaas (mklaas) Date: 2006-10-18 12:47 Message: Logged In: YES user_id=1611720 I cannot yet produce an only-python script which reproduces the problem, but I can give an overview. There is a generator running in one thread, an exception being raised in another thread, and as a consequent, the generator in the first thread is garbage-collected (triggering an exception due to the new generator cleanup). The problem is extremely sensitive to timing--often the insertion/removal of print statements, or reordering the code, causes the problem to vanish, which is confounding my ability to create a simple test script. def getdocs(): def f(): while True: f() yield None # - class B(object): def __init__(self,): pass def doit(self): # must be an instance var to trigger segfault self.docIter = getdocs() print self.docIter # this is the generator referred-to in the traceback for i, item in enumerate(self.docIter): if i > 9: break print 'exiting generator' class A(object): """ Process entry point / main thread """ def __init__(self): while True: try: self.func() except Exception, e: print 'right after raise' def func(self): b = B() thread = threading.Thread(target=b.doit) thread.start() start_t = time.time() while True: try: if time.time() - start_t > 1: raise Exception except Exception: print 'right before raise' # SIGSEGV here. If this is changed to # 'break', no segfault occurs raise if __name__ == '__main__': A() -- Comment By: Mike Klaas (mklaas) Date: 2006-10-18 12:37 Message: Logged In: YES user_id=1611720 I've produced a simplified traceback with a single generator . Note the frame being used in the traceback (#0) is the same frame being dealloc'd (#11). The relevant
[ python-Bugs-1579370 ] Segfault provoked by generators and exceptions
Bugs item #1579370, was opened at 2006-10-17 22:23 Message generated for change (Comment added) made by tim_one You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1579370&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Interpreter Core Group: Python 2.5 Status: Open Resolution: None Priority: 7 Submitted By: Mike Klaas (mklaas) Assigned to: Nobody/Anonymous (nobody) Summary: Segfault provoked by generators and exceptions Initial Comment: A reproducible segfault when using heavily-nested generators and exceptions. Unfortunately, I haven't yet been able to provoke this behaviour with a standalone python2.5 script. There are, however, no third-party c extensions running in the process so I'm fairly confident that it is a problem in the core. The gist of the code is a series of nested generators which leave scope when an exception is raised. This exception is caught and re-raised in an outer loop. The old exception was holding on to the frame which was keeping the generators alive, and the sequence of generator destruction and new finalization caused the segfault. -- >Comment By: Tim Peters (tim_one) Date: 2006-10-18 20:38 Message: Logged In: YES user_id=31435 I've attached a much simplified pure-Python script (hope.py) that reproduces a problem very quickly, on Windows, in a /debug/ build of current trunk. It typically prints: exiting generator joined thread at most twice before crapping out. At the time, the `next` argument to newtracebackobject() is 0x, and tracing back a level shows that, in PyTraceBack_Here(), frame->tstate is entirely filled with 0xdd bytes. Note that this is not a debug-build obmalloc gimmick! This is Microsoft's similar debug-build gimmick for their malloc, and for some reason Python uses the system malloc directly to obtain memory for thread states. The Microsoft debug free() fills newly-freed memory with 0xdd, which has the same meaning as the debug-build obmalloc's DEADBYTE (0xdb). So somebody is accessing a thread state here after it's been freed. Best guess is that the generator is getting "cleaned up" after the thread that created it has gone away, so the generator's frame's f_tstate is trash. Note that a PyThreadState (a frame's f_tstate) is /not/ a Python object -- it's just a raw C struct, and its lifetime isn't controlled by refcounts. -- Comment By: Mike Klaas (mklaas) Date: 2006-10-18 20:12 Message: Logged In: YES user_id=1611720 Despite Tim's reassurrance, I'm afraid that Martin's patch does infact prevent the segfault. Sounds like it also introduces a memleak. -- Comment By: Tim Peters (tim_one) Date: 2006-10-18 17:57 Message: Logged In: YES user_id=31435 > Can anybody tell why gi_frame *isn't* incref'ed when > the generator is created? As documented (in concrete.tex), PyGen_New(f) steals a reference to the frame passed to it. Its only call site (well, in the core) is in ceval.c, which returns immediately after PyGen_New takes over ownership of the frame the caller created: """ /* Create a new generator that owns the ready to run frame * and return that as the value. */ return PyGen_New(f); """ In short, that PyGen_New() doesn't incref the frame passed to it is intentional. It's possible that the intent is flawed ;-), but offhand I don't see how. -- Comment By: Martin v. Löwis (loewis) Date: 2006-10-18 17:05 Message: Logged In: YES user_id=21627 Can you please review/try attached patch? Can anybody tell why gi_frame *isn't* incref'ed when the generator is created? -- Comment By: Mike Klaas (mklaas) Date: 2006-10-18 15:47 Message: Logged In: YES user_id=1611720 I cannot yet produce an only-python script which reproduces the problem, but I can give an overview. There is a generator running in one thread, an exception being raised in another thread, and as a consequent, the generator in the first thread is garbage-collected (triggering an exception due to the new generator cleanup). The problem is extremely sensitive to timing--often the insertion/removal of print statements, or reordering the code, causes the problem to vanish, which is confounding my ability to create a simple test script. def getdocs(): def f(): while True: f() yield None # - class B(object): def __init__(self,): pass def doit(self): # must be an instance var to trigger segfault self.do
