On 2011-05-03 20:18:33 -0400, Catherine Moroney said:
Hello,
I have an object of class X that I am writing to a pickled file. The
pickling part goes fine, but I am having some problems reading the
object back out, as I get complaints about "unable to import module X".
The only way I have found around it is to run the read-file code out of
the same directory that contains the X.py file, but this is obviously
not a portable way of doing things.
Even when I put statements into the code such as "from Y.X import X"
where Y is the name of the python package that contains the X,py file,
the import statement works, but I am still unable to read the object
from the pickled file, running into the same "unable to import module X"
error.
Am I explaining myself properly? Why doesn't the code that loads the
object from the pickled file work unless I am sitting in the same directory?
That's not the actual requirement.
The pickled representation of an instance of a user-defined class contains:
* the name of the class
* the name of the class's module
* the object's attributes, pickled recursively
You can have a look at the representation using protocol zero, which is
printable and not terribly hard to read. Here's an example, lightly
annotated:
>>> import pickle
>>> class Example(object):
... def __init__(self, x):
... self.x = x
...
>>> one = Example(1)
>>> print pickle.dumps(one, 0)
ccopy_reg
_reconstructor
p0
module name (c__main__
class name Example
p1
c__builtin__
object
p2
Ntp3
Rp4
(dp5
field name S'x'
p6
field value I1
sb.
>>>
Obviously, since the pickled representation only contains the *name and
location* of the class and not the class itself, you need to have the
definition of the class handy when unpickling an object. This means
that the module that defines the class must be on the module search
path - either because it's in the current directory (or a package in
the current directory, or..), or because it's in PYTHONPATH, or because
it's somewhere else on sys.path.
The code that writes the pickled file has the statement
"from Y.X import X" statement" at the top, as does the reading code, but
even though that import statement succeeds, the read still fails with
the import error.
The unpickle code uses something morally equivalent to __import__ to
convert the module name in the pickled data stream into a module
object. The symbols defined in the module you called pickle.load from
don't matter at all, just like any other function defined in a separate
module.
Pickle isn't meant for transmitting data between two unrelated
programs. It's designed to be a low-development-effort way to preserve
and restore objects for a single program or a group of related programs
that share libraries. If you want to share data between unrelated
programs, use something a little more interoperable - json, xml, or
some other program-agnostic representation.
-o
(And if you were thinking of using pickle over the internet, forget it:
the pickle encoding is a small stack-based programming language in its
own right, and can be used to invoke semi-arbitrary module-scoped
funtions.)
--
http://mail.python.org/mailman/listinfo/python-list
