El Lunes Diciembre 5 2011, b...@mail.python.org escribió: > On Mon, 5 Dec 2011 09:02:08 +0100, DPalao <dpalao.pyt...@gmail.com> > > wrote: > >El Martes Noviembre 29 2011, DPalao escribió: > >> Hello, > >> I'm trying to use multiprocessing to parallelize a code. There is a > >> number of tasks (usually 12) that can be run independently. Each task > >> produces a numpy array, and at the end, those arrays must be combined. > >> I implemented this using Queues (multiprocessing.Queue): one for input > >> and another for output. > >> But the code blocks. And it must be related to the size of the item I > >> put on the Queue: if I put a small array, the code works well; if the > >> array is realistically large (in my case if can vary from 160kB to > >> 1MB), the code blocks apparently forever. > >> I have tried this: > >> http://www.bryceboe.com/2011/01/28/the-python-multiprocessing-queue-and- > >> lar ge- objects/ > >> but it didn't work (especifically I put a None sentinel at the end for > >> each worker). > >> > >> Before I change the implementation, > >> is there a way to bypass this problem with multiprocessing.Queue? > >> Should I post the code (or a sketchy version of it)? > >> > >> TIA, > >> > >> David > > > >Just for reference. The other day I found the explanation by "ryles" on > >his/her mail of 27th aug 2009, with title "Re: Q: multiprocessing.Queue > >size limitations or bug...". It is very clarifying. > >After having read that I arranged the program such that the main process > >did not need to know when the others have finished, so I changed the > >process join call with a queue get call, until a None (one per process) > >is returned. > > > >Best, > > > >David > > Why do people add character like [* SPAM *] to their subject > lines ?? Is it supposed to do something ?? I figured since > programmers hang out here, maybe one of you know this. > > Thanks, > boB
Obviously it was not me who added the disgusting "70% [* SPAM *]" string to the subject. And I'd like to know the answer too. David -- http://mail.python.org/mailman/listinfo/python-list
