If you are getting to the point where your data is large enough to
really care about the speed of cPickle, then maybe its time you moved
past pickles for your storage format? 2.5 includes sqlite, so you
could persist them in a nice, indexed table or something. Just a
suggestion.
On Jun 12, 2008, at 2:25 PM, Eric Jonas wrote:
Hello,
I've done some benchmarking while attempting to serialize my (large)
graph data structure with cPickle; I'm seeing superlinear performance
(plotting it seems to suggest n^2 where n is the number of nodes of my
graph), in the duration of the pickle.dump calls and I can't quite
figure out why. The connectivity of the graph is such that the
number of
nodes is ~ number of edges, so I don't think this is a problem of edge
count secretly growing > O(n). Is cPickle's behavior known to be O
(n^2)?
Does anyone have any generic tips for speeding up cPickle?
Thanks,
...Eric
--
http://mail.python.org/mailman/listinfo/python-list
--
http://mail.python.org/mailman/listinfo/python-list
