Re: [Twisted-Python] A resizable cooperator class for queuing and dispatching jobs

2009-12-21 Thread Terry Jones
I forgot to mention that I put the code for this class onto Launchpad: https://launchpad.net/txrdq Terry ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] A resizable cooperator class for queuing and dispatching jobs

2009-12-11 Thread sstein...@gmail.com
On Dec 11, 2009, at 6:44 AM, Terry Jones wrote: > I'm not sure the details of all this are of interest to many people on the > list. I'll find time to put the code up somewhere more useful. It would be > nice to have a runnable demo/example too. Not sure I'll get there. Cool, if you'll put up a li

Re: [Twisted-Python] A resizable cooperator class for queuing and dispatching jobs

2009-12-11 Thread Terry Jones
> "Dave" == Dave Peticolas writes: Dave> Cool. What about being able to get the set of underway objects, in Dave> addition to the set of pending objects? Then you can get the whole Dave> state of the queue at that point. I added an underway() method that gets you a set of underway jobs, each

Re: [Twisted-Python] A resizable cooperator class for queuing and dispatching jobs

2009-12-10 Thread Dave Peticolas
Terry Jones wrote: >> "Dave" == Dave Peticolas writes: > Dave> This is really nifty. I know I could use this. > > Great. Me too :-) > >>> There's also the issue about what to do when the dispatch function hits >>> an error. An option could be added to re-queue the job, but it's >>> perhaps

Re: [Twisted-Python] A resizable cooperator class for queuing and dispatching jobs

2009-12-10 Thread sstein...@gmail.com
On Dec 10, 2009, at 1:46 AM, Terry Jones wrote: > glyph> Heh. For something in a pastebin, that's all I got :). Stick it in > glyph> version control somewhere! > > Will do. I'm just adding pause/resume, following the suggestion of Dave > Peticolas (thanks Dave). Cool. I could put this to go

Re: [Twisted-Python] A resizable cooperator class for queuing and dispatching jobs

2009-12-10 Thread Terry Jones
> "Dave" == Dave Peticolas writes: Dave> This is really nifty. I know I could use this. Great. Me too :-) >> There's also the issue about what to do when the dispatch function hits >> an error. An option could be added to re-queue the job, but it's >> perhaps better to let the dispatch func

Re: [Twisted-Python] A resizable cooperator class for queuing and dispatching jobs

2009-12-09 Thread Terry Jones
glyph> Heh. For something in a pastebin, that's all I got :). Stick it in glyph> version control somewhere! Will do. I'm just adding pause/resume, following the suggestion of Dave Peticolas (thanks Dave). Terry ___ Twisted-Python mailing list Twiste

Re: [Twisted-Python] A resizable cooperator class for queuing and dispatching jobs

2009-12-09 Thread Glyph Lefkowitz
On Dec 9, 2009, at 4:44 PM, Terry Jones wrote: > But the code Glyph, the code? Or have I graduated to the point where I no > longer need a laconic garotting? I think the combo of a DeferredQueue and > a task.Cooperator is pretty deadly. Heh. For something in a pastebin, that's all I got :). St

Re: [Twisted-Python] A resizable cooperator class for queuing and dispatching jobs

2009-12-09 Thread Dave Peticolas
Terry Jones wrote: > I just wrote a fun class that lets you > >- submit jobs to be dispatched to a queue >- manage how many tasks are in progress at once >- dynamically adjust that number >- shut down cleanly, including >- recovering jobs that were queued but hadn't been dispat

Re: [Twisted-Python] A resizable cooperator class for queuing and dispatching jobs

2009-12-09 Thread Terry Jones
> "Glyph" == Glyph Lefkowitz writes: Glyph> On Dec 8, 2009, at 11:09 PM, Terry Jones wrote: >> I just wrote a fun class that lets you >> >> - submit jobs to be dispatched to a queue >> - manage how many tasks are in progress at once >> - dynamically adjust that number >> - shut down cleanly,

Re: [Twisted-Python] A resizable cooperator class for queuing and dispatching jobs

2009-12-09 Thread Glyph Lefkowitz
On Dec 8, 2009, at 11:09 PM, Terry Jones wrote: > I just wrote a fun class that lets you > > - submit jobs to be dispatched to a queue > - manage how many tasks are in progress at once > - dynamically adjust that number > - shut down cleanly, including > - recovering jobs that were que

[Twisted-Python] A resizable cooperator class for queuing and dispatching jobs

2009-12-08 Thread Terry Jones
I just wrote a fun class that lets you - submit jobs to be dispatched to a queue - manage how many tasks are in progress at once - dynamically adjust that number - shut down cleanly, including - recovering jobs that were queued but hadn't been dispatched This uses a combination of