Bugs item #931877, was opened at 2004-04-08 19:46
Message generated for change (Comment added) made by zseil
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=931877&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Tim Peters (tim_one)
Assigned to: Nobody/Anonymous (nobody)
Summary: Segfault in object_reduce_ex
Initial Comment:
Shane Hathaway bumped into this, unbounded recursion
in typeobject.c's object_reduce_ex(). This occurs in
Python 2.3.3 and current CVS.
Assigned to Guido, to ponder whether object_reduce_ex
is doing what it should; if it is (which seems likely to
me), I suppose we need to inject a recursion counter to
prevent the segfault.
The failing case is short, but I'll attach it (temp99.py) to
avoid SF line mangling. While the test uses pickle, same
symptom if it's changed to use cPickle instead.
Jim Fulton's analysis:
"""
This is a very clever infinite loop. The proxy doesn't
actually proxy, but it does manage to confuse reduce
about what's going on.
reduce tries to figure out if it has been overridden by
asking whether the class's reduce is the same as
object.__reduce__. It doesn't expect to be lied to
about the class. Things wouldn't have been so bad if
the proxy had proxied __reduce__ as well as __class__.
"""
The priority hasn't been bumped, because "the real
code" from which this was whittled down wasn't doing
what it needed to do anyway, and the recursion went
away when the real code was repaired.
----------------------------------------------------------------------
Comment By: iga Seilnacht (zseil)
Date: 2006-04-01 03:43
Message:
Logged In: YES
user_id=1326842
See patch #1462488. If that patch is accepted, this bug
should be closed as fixed.
----------------------------------------------------------------------
Comment By: Guido van Rossum (gvanrossum)
Date: 2006-03-15 06:05
Message:
Logged In: YES
user_id=6380
Unassigning. I need to concentrate on Python 3000.
----------------------------------------------------------------------
Comment By: Georg Brandl (birkenfeld)
Date: 2006-01-10 22:58
Message:
Logged In: YES
user_id=1188172
Still crashing with 2.5...
----------------------------------------------------------------------
Comment By: Nathan Srebro (nati)
Date: 2004-11-30 02:26
Message:
Logged In: YES
user_id=63133
This infinite recursion also occurs in another place, that
got me stumped for a couple of days when old code that
worked with Python 2.2 stopped working. If __class__ is not
fidgeted with (as in original bug report), but a descriptor
returns a custom reduce for the class, but not for its
objects, reduce enters an infinite loop on the object:
"""
class descriptor_for_reduce(object):
def __get__(self,obj,tp=None):
if obj is not None: return
super(ASpecialClass,obj).__reduce__
return self.reducer
def reducer(self,proto=None):
return "VerySpecial"
class ASpecialClass(object):
__reduce__ = descriptor_for_reduce()
copy.copy(ASpecialClass())
"""
ASpecialClass().__reduce__ is object.__reduce__, which is
implemented by typeobject.c:object_reduce_ex. This function
(that doesn't know if its called as the __reduce__ or the
__reduce_ex__ method) tries to detect if the object's
__reduce__ is overridden. It does so by checking if the
object's class's __reduce__ is overridden, and in fact it
is. It then assumes that the object's __reduce__ is
overridden, and calls it. But the object's __reduce__ is
the same function, causing the infinite loop.
If __reduce_ex__ is used instead of __reduce__, the problem
goes away, ASpecialClass().__reduce_ex__() return the usual
tuple, and ASpecialClass.__reduce_ex__() return
"VerySpecial". But when __reduce__ is overridden,
ASpecialClass().__reduce__() enters an infinite loop.
I believe this is a legitimate example that should behave
just as when __reduce_ex__ is overridden. The example
doesn't lie about __class__, and it is certainly legitimate
for define a property that behaves differently for the class
and for its objects.
Where did this come up and why would I ever care about a
class's __reduce__? The __reduce__ attribute of a class is
never used by (the standard) pickle or copy, since
save_global() is called instead. However, I have a custom
pickler, implemented as a subclass of pickle.Pickler, which
falls back on the class's __reduce__ when save_global()
fails. This way, I can pickle certain classes that are
created at run-time (and can be easily recreated, e.g. from
their bases and dictionaries).
----------------------------------------------------------------------
Comment By: Tim Peters (tim_one)
Date: 2004-04-08 19:51
Message:
Logged In: YES
user_id=31435
Hmm! The temp99.py download link doesn't work for me.
Here's the content in case it doesn't work for others either:
"""
import cPickle as pickle
from pickle import dumps
class SimpleItem:
def __reduce__(self):
return (self.__class__, None, {})
class Proxy(object):
__class__ = property(lambda self: self.__target.__class__)
def __init__(self, target):
self.__target = target
def __getstate__(self):
raise RuntimeError("No don't pickle me! Aaarrgghh!")
p = Proxy(SimpleItem())
dumps(p)
"""
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=931877&group_id=5470
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
