pickle unable to load collection

[paragk]

Hi,

I am unable to figure out the cause of python pickle unable to find
the collection module.

I am getting

__import__(module)
ImportError: No module named collections

when I try to load a pickled object.

Details:

Python version:

Python 2.6 (r26:66721, Oct  2 2008, 11:35:03) [MSC v.1500 32 bit
(Intel)] on win32


[CODE]

import sys
import glob
import collections
import pickle

...

SomeListDict = collections.defaultdict(list)

... # Populate SomeListDict

DictFile = open (options.saveDict, 'w')
pickle.dump(SomeListDict , DictFile, -1)
DictFile.close()

...

# Try loading the dictionary

LoadDictFile = open(options.loadDict, 'rb')
SomeListDict = pickle.load(LoadDictFile)
LoadDictFile.close()

...
 [CODE END]

The pickle.dump(CallGraphDict, DictFile, -1) works fine. The
dictionary is saved.

The interesting thing is, the dump function is able to find the
collections module.
try:
__import__(module)
mod = sys.modules[module]
klass = getattr(mod, name)

The above try block succeeds.

module
str: collections

mod
module: 

klass
type: 


The SomeListDict = pickle.load(LoadDictFile) fails with

__import__(module)
ImportError: No module named collections


The sequence of calls is:

def load_global(self):
module = self.readline()[:-1]
name = self.readline()[:-1]
klass = self.find_class(module, name)
self.append(klass)
dispatch[GLOBAL] = load_global

Variables:

module
str: collections

name
str: defaultdict


The code where the exception occurs is:

def find_class(self, module, name):
# Subclasses may override this
__import__(module)
mod = sys.modules[module]
klass = getattr(mod, name)
return klass


>From the above analysis, clearly the collection module exists, since
the dump works.

What am I missing?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pickle unable to load collection

[paragk]

On May 18, 12:10 am, Peter Otten <__pete...@web.de> wrote:
> paragk wrote:
> > Hi,
>
> > I am unable to figure out the cause of python pickle unable to find
> > the collection module.
>
> > I am getting
>
> >     __import__(module)
> > ImportError: No module named collections
>
> > when I try to load a pickled object.
>
> > Details:
>
> > Python version:
>
> > Python 2.6 (r26:66721, Oct  2 2008, 11:35:03) [MSC v.1500 32 bit
> > (Intel)] on win32
>
> > [CODE]
>
> > import sys
> > import glob
> > import collections
> > import pickle
>
> > ...
>
> > SomeListDict = collections.defaultdict(list)
>
> > ... # Populate SomeListDict
>
> > DictFile = open (options.saveDict, 'w')
> > pickle.dump(SomeListDict , DictFile, -1)
> > DictFile.close()
>
> [...]
>
> > From the above analysis, clearly the collection module exists, since
> > the dump works.
>
> > What am I missing?
>
> You have to open the file in binary mode "wb".
>
> >>> import collections
> >>> import pickle
> >>> data = pickle.dumps(collections.defaultdict())
> >>> pickle.loads(data)
>
> defaultdict(None, {})
>
> Now simulate the effect of writing in text mode and reading in binary mode:
>
> >>> garbled_data = data.replace("\n", "\r\n")
> >>> pickle.loads(garbled_data)
>
> Traceback (most recent call last):
>   File "", line 1, in 
>   File "/usr/lib/python2.6/pickle.py", line 1374, in loads
>     return Unpickler(file).load()
>   File "/usr/lib/python2.6/pickle.py", line 858, in load
>     dispatch[key](self)
>   File "/usr/lib/python2.6/pickle.py", line 1090, in load_global
>     klass = self.find_class(module, name)
>   File "/usr/lib/python2.6/pickle.py", line 1124, in find_class
>     __import__(module)
> ImportError: No module named collections
>
> >>> try: pickle.loads(garbled_data)
>
> ... except ImportError as e:
> ...     e
> ...
> ImportError('No module named collections\r',)
>
> So as Steven suspected there was a whitespace char in the module name:
> pickle.load() was looking for the "collections\r" module.
>
> Peter

Thank you for the suggestions. Opening the file in 'wb' mode worked.

Thanks,
Parag
-- 
http://mail.python.org/mailman/listinfo/python-list