Virgil Stokes wrote:
> Here is snippet of Python (vers. 2.7.10) code that bothers me.
>
> import cPickle as pickle
>
> print "Pickle lists:"
> dogs = ['Rover','King','Spot','Rufus']
> cats = ['Mimi','Misty','Sasha']
>
> with open('pickle.dat', 'wb') as pfile:
> pickle.dump(dogs, pfile)
> pickle.dump(cats,pfile)
>
> del(dogs); del(cats)
> with open('pickle.dat', 'rb') as pfile:
> dogs = pickle.load(pfile)
> cats = pickle.load(pfile)
> print dogs, '\n', cats, '\n'
>
> import shelve
>
> # Note! __exit__ attribute undefined for shelve
> sfile = shelve.open('shelve.dat')
> sfile['dogs'] = dogs
> sfile['cats'] = cats
> sfile.close()
>
> print "Shelve entries:"
> del(cats); del(dogs)
> sfile = shelve.open('shelve.dat')
> #print sfile
> for key in sfile.keys():
> print key,' - ',sfile[key]
> sfile.close()
>
> 1) Which (the pickle or shelve code) takes less total RAM, if dogs and
> cats were very large?
I think this is the wrong question. shelve is built on top of a key-value
store. It can hold more cats and dogs than fit into memory, and lookup of a
cat or dog should be efficient when you store one animal per key:
db = shelve.open(...)
db["Rover"] = Dog(...)
db["King"] = Dog(...)
db["Misty"] = Cat(...)
If you put all dogs in a big list to store them in the shelve and there are
only a few such lists there is no advantage over plain pickle.
> 2) When the last shelve.open is given, is the entire contents of
> shelve.data
> transferred to RAM? Note, if the print sfile is uncommented then the
> entire contents of shelve.data is printed out.
That is due to a careless implementation. Lookup is good, but everything
else will only work for debugging, with small data samples.
> I was under the impression that the entire contents of a shelved file was
> not transferred to RAM when it was opened.
They aren't. The details vary depending on what database is used.
--
https://mail.python.org/mailman/listinfo/python-list
