[issue9276] pickle should support methods

Alexandre Vassalotti added the comment: As part of the implementation of PEP 3154 (Pickle protocol 4), I've introduced support for pickling methods for all pickle protocols (and not just for the new protocol 4). This was implemented by adding the appropriate __reduce__ method on built-in funct

[issue9276] pickle should support methods

Changes by Alexandre Vassalotti : -- dependencies: +Implement PEP 3154 (pickle protocol 4) ___ Python tracker ___ ___ Python-bugs-list

[issue9276] pickle should support methods

Antoine Pitrou added the comment: PEP 3155 is now implemented (the __qualname__ attribute), making many more things potentially picklable. See PEP 3154 for a hypothetical new pickle protocol, and issue 13520 for a patch proposal. -- ___ Python trac

[issue9276] pickle should support methods

Ram Rachum added the comment: I don't have the time and the ability to write the patch that implements this. I'll be happy to write tests if you think this will help. -- ___ Python tracker

[issue9276] pickle should support methods

Amaury Forgeot d'Arc added the comment: OK, let's go back to the "__namespace__" idea, then. A long time ago I had the idea that the ast compiler could remember the list of "named blocks" (classes, functions) with their line numbers; for example, the statement "class C" spans from line 20 to

[issue9276] pickle should support methods

Antoine Pitrou added the comment: > Amaury: > > Your example succeeds on Linux but fails on Windows: > > $ python3.2 main.py I think the difference has to do with Python 3 vs. Python 2. In Python 3 unbound methods are not wrapped in a specific object, and so ForkingPickler does not find

[issue9276] pickle should support methods

Ram Rachum added the comment: Amaury: Your example succeeds on Linux but fails on Windows: $ python3.2 main.py CALLED Traceback (most recent call last): File "C:\Python32\Lib\pickle.py", line 679, in save_global klass = getattr(mod, name)

[issue9276] pickle should support methods

Amaury Forgeot d'Arc added the comment: did you see my example above? it passes methods defined in user code. -- ___ Python tracker ___ __

[issue9276] pickle should support methods

Ram Rachum added the comment: Amaury: I don't think ForkingPickler works for unbound methods defined in user code, which are implemented as functions. I think it only works for method-descriptors and wrapper-descriptors. -- ___ Python tracker

[issue9276] pickle should support methods

Amaury Forgeot d'Arc added the comment: > Being able to pickle unbound methods is important. In my project I have > objects that refer to unbound methods. Now these objects are > unpickleable. I can't save them to disk and I can't use the > multiprocessing module on them. That's a big problem.

[issue9276] pickle should support methods

Antoine Pitrou added the comment: > To have a non-hackish patch we need a non-hackish idea. The > `.__parent_class__` idea looks hackish to me, but now that I think > about it, how is it more hackish than a function's `.__module__` > attribute? Not much, agreed. You might want to call it `__nam

[issue9276] pickle should support methods

Changes by Daniel Urban : -- nosy: +durban ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.or

[issue9276] pickle should support methods

Ram Rachum added the comment: > [...] try to whip up a patch and upload it if it ends up not too hackish. To have a non-hackish patch we need a non-hackish idea. The `.__parent_class__` idea looks hackish to me, but now that I think about it, how is it more hackish than a function's `.__modu

[issue9276] pickle should support methods

Antoine Pitrou added the comment: > Being able to pickle unbound methods is important. In my project I have > objects that refer to unbound methods. Now these objects are > unpickleable. I can't save them to disk and I can't use the > multiprocessing module on them. That's a big problem. Judgi

[issue9276] pickle should support methods

Ram Rachum added the comment: > This isn't worth introducing poorly thought out hacks. Being able to pickle unbound methods is important. In my project I have objects that refer to unbound methods. Now these objects are unpickleable. I can't save them to disk and I can't use the multiprocessi

[issue9276] pickle should support methods

Raymond Hettinger added the comment: This isn't worth introducing poorly thought out hacks. -- ___ Python tracker ___ ___ Python-bugs-

[issue9276] pickle should support methods

Ram Rachum added the comment: Raymond: I don't think this matters. We don't need a canonical `.__parent_class__`, we just need to know where that function is defined so we could find it when unpickling. In the example that you gave, `f` would have a `.__parent_class__` of `None`, and it would

[issue9276] pickle should support methods

Raymond Hettinger added the comment: > how about we give every function a `.__parent_class__` > attribute saying which class it is a method of? Won't work. The same function can be used in multiple classes. The function object is independent of the class. This is conceptually no different

[issue9276] pickle should support methods

Ram Rachum added the comment: Okay, as an initial suggestion, how about we give every function a `.__parent_class__` attribute saying which class it is a method of? I feel this is a bit of a duct tape solution, but I don't see any other alternative. -- ___

[issue9276] pickle should support methods

Changes by Ram Rachum : -- versions: +Python 3.3 -Python 3.2 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: htt

[issue9276] pickle should support methods

Ram Rachum added the comment: I also miss being able to pickle unbound methods on Python 3. I don't think there's an interest in pickling the actual code objects. In my opinion, unbound methods should be pickled exactly like all the other Python definitions, such as bound methods, top-level f

[issue9276] pickle should support methods

Alexandre Vassalotti added the comment: The security issue mentioned previously has been known for years. And, it is easy to protect against. See http://docs.python.org/py3k/library/pickle.html#restricting-globals Also I am against adding pickling support to code objects. Code objects have no

[issue9276] pickle should support methods

Jean-Paul Calderone added the comment: > Note, however that since unbound methods have been removed in 3.x, it is not > trivial to find a fully qualified name of a method anymore. This is a rather sad loss of functionality. -- ___ Python tracker <

[issue9276] pickle should support methods

Changes by Antoine Pitrou : -- nosy: +hinsen, loewis, obamausa8, rhettinger ___ Python tracker ___ ___ Python-bugs-list mailing list Un

[issue9276] pickle should support methods

Antoine Pitrou added the comment: > Yes, I think we have a consensus on this point. Note, however that > since unbound methods have been removed in 3.x, it is not trivial to > find a fully qualified name of a method anymore. I suppose only bound methods should be pickleable: >>> class C: ...

[issue9276] pickle should support methods

Changes by Éric Araujo : -- nosy: +merwok ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org

[issue9276] pickle should support methods

Alexander Belopolsky added the comment: On Mon, Aug 2, 2010 at 10:32 AM, Jean-Paul Calderone wrote: > > Jean-Paul Calderone added the comment: > >> I also like Antoine's idea of pickling the function/method name instead of >> the whole code object. > > I like it too.  That's why I suggested i

[issue9276] pickle should support methods

Antoine Pitrou added the comment: There's already issue558238 on the same topic. -- ___ Python tracker ___ ___ Python-bugs-list mailin

[issue9276] pickle should support methods

Jean-Paul Calderone added the comment: > I also like Antoine's idea of pickling the function/method name instead of > the whole code object. I like it too. That's why I suggested it in the first comment on the ticket (read the linked code). I guess Alexander likes it too, since he basically

[issue9276] pickle should support methods

Marc-Andre Lemburg added the comment: Alexander Belopolsky wrote: > > Alexander Belopolsky added the comment: > > On Mon, Aug 2, 2010 at 10:11 AM, Marc-Andre Lemburg > wrote: > .. >> Hmm, I just tried the code and it seems that you're right: >> >> The pickle string does not contain a referen

[issue9276] pickle should support methods

Alexander Belopolsky added the comment: On Mon, Aug 2, 2010 at 10:11 AM, Marc-Andre Lemburg wrote: .. > Hmm, I just tried the code and it seems that you're right: > > The pickle string does not contain a reference to class x, > but only the name of the function to call. Wow, that's a huge > hol

[issue9276] pickle should support methods

Marc-Andre Lemburg added the comment: M.-A. Lemburg wrote: > Jean-Paul Calderone wrote: >> >> Jean-Paul Calderone added the comment: >> >> For example: >> >> exar...@boson:~$ python >> Python 2.6.4 (r264:75706, Dec 7 2009, 18:45:15) >> [GCC 4.4.1] on linux2 >> Type "help", "copyright", "credi

[issue9276] pickle should support methods

Antoine Pitrou added the comment: I think methods should be picklable just like global functions are, that is, by pickling a tuple of the fully-qualified class name ("mymodule.myclass"), method name ("mymethod"), and self. -- ___ Python tracker

[issue9276] pickle should support methods

Alexander Belopolsky added the comment: On Mon, Aug 2, 2010 at 10:05 AM, Marc-Andre Lemburg wrote: .. > Without the definition of class x on the receiving side, there > would be no exploit. You are mistaken. Try adding del x (or del evil in my example) between dumps and loads and see it worki

[issue9276] pickle should support methods

Marc-Andre Lemburg added the comment: Jean-Paul Calderone wrote: > > Jean-Paul Calderone added the comment: > > For example: > > exar...@boson:~$ python > Python 2.6.4 (r264:75706, Dec 7 2009, 18:45:15) > [GCC 4.4.1] on linux2 > Type "help", "copyright", "credits" or "license" for more inf

[issue9276] pickle should support methods

Changes by Alexander Belopolsky : -- nosy: +alexandre.vassalotti, pitrou ___ Python tracker ___ ___ Python-bugs-list mailing list Unsub

[issue9276] pickle should support methods

Alexander Belopolsky added the comment: On Mon, Aug 2, 2010 at 9:25 AM, Marc-Andre Lemburg wrote: .. >> You can *already* trick unpickling code into executing serialized code.  You >> don't need > this feature in order to be able to do it. > > How ? > >>> from pickle import * >>> class evil: .

[issue9276] pickle should support methods

Jean-Paul Calderone added the comment: For example: exar...@boson:~$ python Python 2.6.4 (r264:75706, Dec 7 2009, 18:45:15) [GCC 4.4.1] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> class x(object): ... def __reduce__(self): ... import os ...

[issue9276] pickle should support methods

Marc-Andre Lemburg added the comment: Jean-Paul Calderone wrote: > > Jean-Paul Calderone added the comment: > >> By adding default support for unpickling code objects, you can trick > the unpickling code into executing serialized code: > > This doesn't sound correct to me. > > You can *alre

[issue9276] pickle should support methods

Jean-Paul Calderone added the comment: > By adding default support for unpickling code objects, you can trick the unpickling code into executing serialized code: This doesn't sound correct to me. You can *already* trick unpickling code into executing serialized code. You don't need this feat

[issue9276] pickle should support methods

Marc-Andre Lemburg added the comment: Jean-Paul Calderone wrote: > > Jean-Paul Calderone added the comment: > >> This is a security feature and should not be broken ! > > Can you explain this? > > I don't think I agree, since an attacker can always serialize whatever they > feel like. It'

[issue9276] pickle should support methods

Jean-Paul Calderone added the comment: > This is a security feature and should not be broken ! Can you explain this? I don't think I agree, since an attacker can always serialize whatever they feel like. It's the person doing the deserialization that has to be careful. -- _

[issue9276] pickle should support methods

Marc-Andre Lemburg added the comment: Note that pickle deliberately does not support serializing code objects. This is a security feature and should not be broken ! If you need to pickle such objects, you can easily register handlers that take care of this. -- nosy: +lemburg __

[issue9276] pickle should support methods

Alexander Belopolsky added the comment: Not a proposed solution, but food for thought. Methods do have __reduce_ex__ method which works with protocol 3: >>> class X: ... def f(self): ... pass >>> X.f.__reduce_ex__(3) (, (,), {}, None, None) This result is useless for several reasons:

[issue9276] pickle should support methods

New submission from Jean-Paul Calderone : pickle doesn't support methods: >>> class x: ... def y(self): ... pass ... >>> import pickle >>> pickle.dumps(x.y) Traceback (most recent call last): File "", line 1, in File "/home/exarkun/Projects/python/branches/py3k/Lib/pickle.