Hey, this pretty easy hack appears to work!
[code]
from multiprocessing.pool import Pool, RUN, MapResult, mapstar
class PriorityPool(Pool):
def map_async_nowait(self, func, iterable, chunksize=None, \
callback=None):
"""
Same as map_async(), except uses put_nowait()
On Sep 15, 1:52 pm, John Ladasky wrote:
> I've been snooping around inside Pool, and I would guess that what I
> want to do is to manipulate Pool._inqueue, which is a
> multiprocessing.queues.SimpleQueue object. I haven't found any
> documentation for SimpleQueue. It appears to have only the mos
On Fri, Sep 16, 2011 at 8:32 AM, John Ladasky wrote:
> Ah. Now, see? Having the right vocabulary helps. Searching for
> "priority queue" brings up this discussion from back in January:
>
> http://groups.google.com/group/comp.lang.python/browse_frm/thread/b69aeced28634898
>
> Now, this discussion
Ah. Now, see? Having the right vocabulary helps. Searching for
"priority queue" brings up this discussion from back in January:
http://groups.google.com/group/comp.lang.python/browse_frm/thread/b69aeced28634898
Now, this discussion refers to a PriorityPool class which doesn't
appear to be a sta
On Fri, Sep 16, 2011 at 8:25 AM, John Ladasky wrote:
> Starting 50 milliseconds late would be close enough
> to "real time" for my purposes.
> ...
>
> If that's not putting T2 at the head of the queue, I guess I don't
> know a better way to describe it.
Yep, your terms are correct, with that cave
On Sep 15, 3:14 pm, Chris Angelico wrote:
> On Fri, Sep 16, 2011 at 6:52 AM, John Ladasky wrote:
> > Suppose that I have a second, parallelizable, long-running task T2
> > that I want to address in REAL TIME when the need arises. Using Pool,
> > is there a way for me to insert the chunks of T2 a
On Fri, Sep 16, 2011 at 6:52 AM, John Ladasky wrote:
> Suppose that I have a second, parallelizable, long-running task T2
> that I want to address in REAL TIME when the need arises. Using Pool,
> is there a way for me to insert the chunks of T2 at the HEAD of the
> task queue, instead of at its T
Suppose that I have a multi-core computer with N CPU's, and I create a
multiprocessing.Pool in Python 2.6 with N-1 Processes.
(My rationale for choosing N-1 Processes was discussed here:
http://groups.google.com/group/comp.lang.python/browse_frm/thread/65ba3ccd4be8228c)
Then I use Pool.map_async(
