I am wondering a couple things

 

First, are there any good, open source, perl-based queue systems out there?  I think I would like a stable, centralized daemon, which likely rules out many systems.

 

Second, is anyone interesting in working on a queue system with mod_perl2/apache2 as it's basis, using the mod_echo, arbitrary protocol stuff?

 

Basically, you would register servers and jobs with the daemon, and then the daemon would give jobs out, perhaps wait for them to finish and then return appropriate output.  The system would also handle job dependencies.

I would like to be pretty open, and would hope to easily handle some basic job types, like

 

  1. for each row returned from a db query, do something with it
  2. watch a filesystem for changes to a directory or file and act accordingly
  3. be fast enough to serve web hits through it, like maybe 1/100 second system overhead
  4. have a flat file and do something for each line in it
  5. be able to rather arbitrarily add little jobs, as in a web crawler running in parallel

 

I think using mod_perl2/apache2 would be ideal.  Could use apache2 to give jobs to non-mod_perl processes if desired.  I think that if apache2 could manage forking, children and the like that would save many headaches.

 

I think I even have a name, PDQ, which could stand for Perl Dynamic Queue, or yeah, Pretty Darn Quick.

 

Feed back welcome.

 

Thanks,

Earl

Reply via email to