duncan smith wrote:
> On 06/03/2019 16:14, duncan smith wrote:
>> Hello,
>> I've been trying to figure out why one of my classes can be
>> pickled but not unpickled. (I realise the problem is probably with the
>> pickling, but I get the error when I attempt to unpickle.)
>>
>> A relatively minimal example is pasted below.
>>
>>
>>>>> import pickle
>>>>> class test(dict):
>> def __init__(self, keys, shape=None):
>> self.shape = shape
>> for key in keys:
>> self[key] = None
>>
>> def __setitem__(self, key, val):
>> print (self.shape)
>> dict.__setitem__(self, key, val)
>>
>>
>>>>> x = test([1,2,3])
>> None
>> None
>> None
>>>>> s = pickle.dumps(x)
>>>>> y = pickle.loads(s)
>> Traceback (most recent call last):
>> File "<pyshell#114>", line 1, in <module>
>> y = pickle.loads(s)
>> File "<pyshell#111>", line 8, in __setitem__
>> print (self.shape)
>> AttributeError: 'test' object has no attribute 'shape'
>>
>>
>> I have DUCkDuckGo'ed the issue and have tinkered with __getnewargs__ and
>> __getnewargs_ex__ without being able to figure out exactly what's going
>> on. If I comment out the print call, then it seems to be fine. I'd
>> appreciate any pointers to the underlying problem. I have one or two
>> other things I can do to try to isolate the issue further, but I think
>> the example is perhaps small enough that someone in the know could spot
>> the problem at a glance. Cheers.
>>
>> Duncan
>>
>
> OK, this seems to be a "won't fix" bug dating back to 2003
> (https://bugs.python.org/issue826897). The workaround,
>
>
> class DictPlus(dict):
> def __init__(self, *args, **kwargs):
> self.extra_thing = ExtraThingClass()
> dict.__init__(self, *args, **kwargs)
> def __setitem__(self, k, v):
> try:
> do_something_with(self.extra_thing, k, v)
> except AttributeError:
> self.extra_thing = ExtraThingClass()
> do_something_with(self.extra_thing, k, v)
> dict.__setitem__(self, k, v)
> def __setstate__(self, adict):
> pass
>
>
> doesn't work around the problem for me because I need the actual value
> of self.shape from the original instance. But I only need it for sanity
> checking, and under the assumption that the original instance was valid,
> I don't need to do this when unpickling. I haven't managed to find a
> workaround that exploits that (yet?). Cheers.
I've been playing around with __getnewargs__(), and it looks like you can
get it to work with a custom __new__(). Just set the shape attribute there
rather than in __init__():
$ cat pickle_dict_subclass.py
import pickle
class A(dict):
def __new__(cls, keys=(), shape=None):
obj = dict.__new__(cls)
obj.shape = shape
return obj
def __init__(self, keys=(), shape=None):
print("INIT")
for key in keys:
self[key] = None
print("EXIT")
def __setitem__(self, key, val):
print(self.shape, ": ", key, " <-- ", val, sep="")
super().__setitem__(key, val)
def __getnewargs__(self):
print("GETNEWARGS")
return ("xyz", self.shape)
x = A([1, 2, 3], shape="SHAPE")
x["foo"] = "bar"
print("pickling:")
s = pickle.dumps(x)
print("unpickling:")
y = pickle.loads(s)
print(y)
$ python3 pickle_dict_subclass.py
INIT
SHAPE: 1 <-- None
SHAPE: 2 <-- None
SHAPE: 3 <-- None
EXIT
SHAPE: foo <-- bar
pickling:
GETNEWARGS
unpickling:
SHAPE: 1 <-- None
SHAPE: 2 <-- None
SHAPE: 3 <-- None
SHAPE: foo <-- bar
{1: None, 2: None, 3: None, 'foo': 'bar'}
It's not clear to me how the dict items survive when they are not included
in the __getnewargs__() result, but apparently they do.
--
https://mail.python.org/mailman/listinfo/python-list
