On Feb 10, 9:30 am, "Charles Fox (Sheffield)" <charles....@gmail.com> wrote: > Hi guys, > I'm working on debugging a large python simulation which begins by > preloading a huge cache of data. I want to step through code on many > runs to do the debugging. Problem is that it takes 20 seconds to > load the cache at each launch. (Cache is a dict in a 200Mb cPickle > binary file). > > So speed up the compile-test cycle I'm thinking about running a > completely separate process (not a fork, but a processed launched form > a different terminal)
Why _not_ fork? Load up your data, then go into a loop forking and loading/ running the rest of your code in the child. This should be really easy to implement compared to doing something with shared memory, and solves the problem you're trying to solve of long startup time just as well. It also protects you from possible bugs where the data gets corrupted by the code that operates on it, since there's only one copy shared amongst all your tests. Is there some other benefit that the shared memory approach gives you? Of course, adding unit tests that exercise your code on a smaller data set might also be a way to speed up development. Jean-Paul -- http://mail.python.org/mailman/listinfo/python-list
