On Thu, 2008-06-12 at 20:57 +0200, Hrvoje Niksic wrote:
> Eric Jonas <[EMAIL PROTECTED]> writes:
>
> > I've done some benchmarking while attempting to serialize my (large)
> > graph data structure with cPickle; I'm seeing superlinear performance
> > (plotting it seems to suggest n^2 where n is the number of nodes of my
> > graph), in the duration of the pickle.dump calls and I can't quite
> > figure out why.
>
> Try gc.disable() before loading the pickle, and gc.enable() after.
>
> > Is cPickle's behavior known to be O(n^2)?
>
> No, but the garbage collector's sometimes is.
Wow, that totally fixed it -- we went from 1200 seconds to 60 seconds.
I'm somewhat shocked -- is the occasionally-abysmal behavior of the GC
documented anywhere?
...Eric
--
http://mail.python.org/mailman/listinfo/python-list
