https://docs.python.org/3.7/library/pickle.html#pickling-class-instances
includes 2 notes, which I think are the relevant ones:
When a class instance is unpickled, its __init__() method is usually not
invoked. The default behaviour first creates an uninitialized instance and then
restores the saved attributes.
At unpickling time, some methods like __getattr__(), __getattribute__(), or
__setattr__() may be called upon the instance. In case those methods rely on
some internal invariant being true, the type should implement __getnewargs__()
or __getnewargs_ex__() to establish such an invariant; otherwise, neither
__new__() nor __init__() will be called.
So I think that without using the various dunders you can't rely on the bits
being restored in a specific order.
So at the point in the restoration where you're getting the exception,
self.shape hasn't been restored yet while it's trying to set self[1], which is
why trying to print self.shape is failing. Here's a modified version with more
prints to show some of that:
import pickle
class test(dict):
def __init__(self, keys, shape = None):
print("__init__ is running")
print("keys:", keys)
print("shape:", shape)
self.shape = shape
for key in keys:
self[key] = None
def __setitem__(self, key, val):
print("__setitem__ is running")
print("self:", self)
print("key:", key)
print("val:", val)
try:
print("self.shape:", self.shape)
except AttributeError as err:
print("AttributeError:", err)
dict.__setitem__(self, key, val)
x = test([1, 2, 3])
print(x, x.shape)
s = pickle.dumps(x)
print("\n---About to load it---\n")
y = pickle.loads(s)
print(".loads() complete")
print(y, y.shape)
This is what that outputs:
__init__ is running
keys: [1, 2, 3]
shape: None
__setitem__ is running
self: {}
key: 1
val: None
self.shape: None
__setitem__ is running
self: {1: None}
key: 2
val: None
self.shape: None
__setitem__ is running
self: {1: None, 2: None}
key: 3
val: None
self.shape: None
{1: None, 2: None, 3: None} None
---About to load it---
__setitem__ is running
self: {}
key: 1
val: None
AttributeError: 'test' object has no attribute 'shape'
__setitem__ is running
self: {1: None}
key: 2
val: None
AttributeError: 'test' object has no attribute 'shape'
__setitem__ is running
self: {1: None, 2: None}
key: 3
val: None
AttributeError: 'test' object has no attribute 'shape'
.loads() complete
{1: None, 2: None, 3: None} None
Alas, I can't offer any help with how to use __getnewargs__() or the other
dunders to properly handle it.
-----Original Message-----
From: Python-list
[mailto:python-list-bounces+david.raymond=tomtom....@python.org] On Behalf Of
duncan smith
Sent: Wednesday, March 06, 2019 11:14 AM
To: python-list@python.org
Subject: in a pickle
Hello,
I've been trying to figure out why one of my classes can be
pickled but not unpickled. (I realise the problem is probably with the
pickling, but I get the error when I attempt to unpickle.)
A relatively minimal example is pasted below.
>>> import pickle
>>> class test(dict):
def __init__(self, keys, shape=None):
self.shape = shape
for key in keys:
self[key] = None
def __setitem__(self, key, val):
print (self.shape)
dict.__setitem__(self, key, val)
>>> x = test([1,2,3])
None
None
None
>>> s = pickle.dumps(x)
>>> y = pickle.loads(s)
Traceback (most recent call last):
File "<pyshell#114>", line 1, in <module>
y = pickle.loads(s)
File "<pyshell#111>", line 8, in __setitem__
print (self.shape)
AttributeError: 'test' object has no attribute 'shape'
I have DUCkDuckGo'ed the issue and have tinkered with __getnewargs__ and
__getnewargs_ex__ without being able to figure out exactly what's going
on. If I comment out the print call, then it seems to be fine. I'd
appreciate any pointers to the underlying problem. I have one or two
other things I can do to try to isolate the issue further, but I think
the example is perhaps small enough that someone in the know could spot
the problem at a glance. Cheers.
Duncan
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
