On Dec 29, 8:55 pm, Ian Kelly <ian.g.ke...@gmail.com> wrote:
> On Thu, Dec 29, 2011 at 2:55 AM, lars van gemerden <l...@rational-it.com>
> wrote:
>
> > Hello,
>
> > Can someone help me with the following:
>
> > I am using metaclasses to make classes and these classes to make
> > instances. Now I want to use multiprocessing, which needs to pickle
> > these instances.
>
> > Pickle cannot find the class definitions of the instances. I am trying
> > to add a line to the __new__ of the metaclass to add the new class
> > under the right name in the right module/place, so pickle can find
> > it.
>
> > Is this the right approach? Can anyone explain to me where/how to add
> > these classes for pickle to find and maybe why?
>
> It sounds like you're trying to do something like this?
>
> >>> class MetaClass(type):
>
> ... pass
> ...>>> instance = MetaClass('<Anonymous>', (object,), {})()
> >>> instance
>
> <__main__.<Anonymous> object at 0x00CC00F0>>>> import pickle
> >>> pickle.dumps(instance)
>
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> File "c:\python27\lib\pickle.py", line 1374, in dumps
> Pickler(file, protocol).dump(obj)
> File "c:\python27\lib\pickle.py", line 224, in dump
> self.save(obj)
> File "c:\python27\lib\pickle.py", line 331, in save
> self.save_reduce(obj=obj, *rv)
> File "c:\python27\lib\pickle.py", line 401, in save_reduce
> save(args)
> File "c:\python27\lib\pickle.py", line 286, in save
> f(self, obj) # Call unbound method with explicit self
> File "c:\python27\lib\pickle.py", line 562, in save_tuple
> save(element)
> File "c:\python27\lib\pickle.py", line 295, in save
> self.save_global(obj)
> File "c:\python27\lib\pickle.py", line 748, in save_global
> (obj, module, name))
> pickle.PicklingError: Can't pickle <class '__main__.<Anonymous>'>:
> it's not found as __main__.<Anonymous>
>
> Yeah, pickle's not going to work with anonymous classes. As you
> suggest, you could dynamically add the classes to the module namespace
> so that pickle.dumps will find them, but bear in mind that they will
> also have to exist when calling pickle.loads, so you will need to be
> able to reconstruct the same anonymous classes before unpickling later
> on.
>
> Cheers,
> Ian
Ian also wrote:
'''
Actually, I was wrong, you probably don't need to do that. I suggest
going with Robert Kern's suggestion to either register the class with
the copy_reg module, or (perhaps better since it won't leak
registrations) implement a __reduce__ method on the class. For
example, this seems to work:
>>> def reconstructor(*metaclass_args):
... cls = MetaClass.build_class(*metaclass_args)
... self = cls.__new__(cls)
... return self
...
>>> class MetaClass(type):
... @classmethod
... def build_class(mcs, arg1, arg2, arg3):
... # Do something useful with the args...
... class _AnonymousClass(object):
... __metaclass__ = mcs
... def __reduce__(self):
... return (reconstructor, ('foo', 'bar', 'baz'),
self.__dict__)
... return _AnonymousClass
...
>>> instance = MetaClass.build_class('foo', 'bar', 'baz')()
>>> instance
<__main__._AnonymousClass object at 0x011DB410>
>>> instance.banana = 42
>>> import pickle
>>> s = pickle.dumps(instance)
>>> s
"c__main__\nreconstructor
\np0\n(S'foo'\np1\nS'bar'\np2\nS'baz'\np3\ntp4\nRp5\n(dp6\nS'banana'\np7\nI42\nsb."
>>> inst2 = pickle.loads(s)
>>> inst2
<__main__._AnonymousClass object at 0x011DBE90>
>>> inst2.banana
42
>>> inst2.__class__ is instance.__class__
False
Cheers,
Ian
'''
--
http://mail.python.org/mailman/listinfo/python-list
