On 3 Aug, 09:36, ma3mju <matt.u...@googlemail.com> wrote: > On 2 Aug, 21:49, Piet van Oostrum <p...@cs.uu.nl> wrote: > > > >>>>> MRAB <pyt...@mrabarnett.plus.com> (M) wrote: > > >M> I wonder whether one of the workers is raising an exception, perhaps due > > >M> to lack of memory, when there are large number of jobs to process. > > > But that wouldn't prevent the join. And you would probably get an > > exception traceback printed. > > > I wonder if something fishy is happening in the multiprocessing > > infrastructure. Or maybe the Fortran code goes wrong because it has no > > protection against buffer overruns and similar problems, I think. > > -- > > Piet van Oostrum <p...@cs.uu.nl> > > URL:http://pietvanoostrum.com[PGP8DAE142BE17999C4] > > Private email: p...@vanoostrum.org > > I don't think it's a memory problem, the reason for the hard and easy > queue is because for larger examples it uses far more RAM. If I run > all of workers with harder problems I do begin to run out of RAM and > end up spending all my time switching in and out of swap so I limit > the number of harder problems I run at the same time. I've watched it > run to the end (a very boring couple of hours) and it stays out of my > swap space and everything appears to be staying in RAM. Just hangs > after all "poison" has been printed for each process. > > The other thing is that I get the message "here" telling me I broke > out of the loop after seeing the poison pill in the process and I get > all the things queued listed as output surely if I were to run out of > memory I wouldn't expect all of the jobs to be listed as output. > > I have a serial script that works fine so I know individually for each > example the fortran code works. > > Thanks > > Matt
Any ideas? -- http://mail.python.org/mailman/listinfo/python-list
