On Dec 30, 12:16 pm, lars van gemerden <l...@rational-it.com> wrote:
> On Dec 29, 8:55 pm, Ian Kelly <ian.g.ke...@gmail.com> wrote:
>
>
>
>
>
>
>
>
>
> > On Thu, Dec 29, 2011 at 2:55 AM, lars van gemerden <l...@rational-it.com>
> > wrote:
>
> > > Hello,
>
> > > Can someone help me with the following:
>
> > > I am using metaclasses to make classes and these classes to make
> > > instances. Now I want to use multiprocessing, which needs to pickle
> > > these instances.
>
> > > Pickle cannot find the class definitions of the instances. I am trying
> > > to add a line to the __new__ of the metaclass to add the new class
> > > under the right name in the right module/place, so pickle can find
> > > it.
>
> > > Is this the right approach? Can anyone explain to me where/how to add
> > > these classes for pickle to find and maybe why?
>
> > It sounds like you're trying to do something like this?
>
> > >>> class MetaClass(type):
>
> > ... pass
> > ...>>> instance = MetaClass('<Anonymous>', (object,), {})()
> > >>> instance
>
> > <__main__.<Anonymous> object at 0x00CC00F0>>>> import pickle
> > >>> pickle.dumps(instance)
>
> > Traceback (most recent call last):
> > File "<stdin>", line 1, in <module>
> > File "c:\python27\lib\pickle.py", line 1374, in dumps
> > Pickler(file, protocol).dump(obj)
> > File "c:\python27\lib\pickle.py", line 224, in dump
> > self.save(obj)
> > File "c:\python27\lib\pickle.py", line 331, in save
> > self.save_reduce(obj=obj, *rv)
> > File "c:\python27\lib\pickle.py", line 401, in save_reduce
> > save(args)
> > File "c:\python27\lib\pickle.py", line 286, in save
> > f(self, obj) # Call unbound method with explicit self
> > File "c:\python27\lib\pickle.py", line 562, in save_tuple
> > save(element)
> > File "c:\python27\lib\pickle.py", line 295, in save
> > self.save_global(obj)
> > File "c:\python27\lib\pickle.py", line 748, in save_global
> > (obj, module, name))
> > pickle.PicklingError: Can't pickle <class '__main__.<Anonymous>'>:
> > it's not found as __main__.<Anonymous>
>
> > Yeah, pickle's not going to work with anonymous classes. As you
> > suggest, you could dynamically add the classes to the module namespace
> > so that pickle.dumps will find them, but bear in mind that they will
> > also have to exist when calling pickle.loads, so you will need to be
> > able to reconstruct the same anonymous classes before unpickling later
> > on.
>
> > Cheers,
> > Ian
> Ian also wrote:
>
> '''
> Actually, I was wrong, you probably don't need to do that. I suggest
> going with Robert Kern's suggestion to either register the class with
> the copy_reg module, or (perhaps better since it won't leak
> registrations) implement a __reduce__ method on the class. For
> example, this seems to work:
>
> >>> def reconstructor(*metaclass_args):
>
> ... cls = MetaClass.build_class(*metaclass_args)
> ... self = cls.__new__(cls)
> ... return self
> ...>>> class MetaClass(type):
>
> ... @classmethod
> ... def build_class(mcs, arg1, arg2, arg3):
> ... # Do something useful with the args...
> ... class _AnonymousClass(object):
> ... __metaclass__ = mcs
> ... def __reduce__(self):
> ... return (reconstructor, ('foo', 'bar', 'baz'),
> self.__dict__)
> ... return _AnonymousClass
> ...>>> instance = MetaClass.build_class('foo', 'bar', 'baz')()
> >>> instance
>
> <__main__._AnonymousClass object at 0x011DB410>>>> instance.banana = 42
> >>> import pickle
> >>> s = pickle.dumps(instance)
> >>> s
>
> "c__main__\nreconstructor
> \np0\n(S'foo'\np1\nS'bar'\np2\nS'baz'\np3\ntp4\nRp5\n(dp6\nS'banana'\np7\nI
> 42\nsb.">>> inst2 = pickle.loads(s)
> >>> inst2
>
> <__main__._AnonymousClass object at 0x011DBE90>>>> inst2.banana
> 42
> >>> inst2.__class__ is instance.__class__
>
> False
>
> Cheers,
> Ian
>
> '''
Interesting, though I cannot say I completely understand this solution
(looked up __reduce__, but still). I am trying to adapt this example
to a situation where the metaclass generated classes are named at
runtime (not anonymous), but cannot figure it out.
Cheers, Lars
--
http://mail.python.org/mailman/listinfo/python-list
