-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 01/03/07 00:34, Mark Harrison wrote: > I have a cluster of CPUs generating thumbnails for > a render farm. I would like to place thumbnail > requests on a queue, and have the cluster of client > dequeue the requests and process them. > > Of course, each request should be only dequeued once... > if a thumbnail is being processed by one CPU, it > shouldn't be processed by another CPU. > > Does the following sound like a reasonable approach? > If not, what's a good way to go? > > The processes generating the requests will insert into > a queue table. They may add a priority and timestamp. > > The several processes servicing the requests will do a > SELECT FOR UPDATE where ... limit 1, generate thumbnail, > delete the record and commit. > > Comments and suggestions welcome,
That's not what relational tables are good at. Instead, use a message queuing library with a file backing-store (so that if the machine goes down for any reason, the messages are still in the queue). In a message-passing system, a network-aware daemon manages a set of named FIFO queues. Some processes call in_q(), and other processes (same machine, or not) call de_q(). If nothing is calling de_q(), messages just pile up in the queue until such time as something *does* start calling de_q(). -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFFm504S9HxQb37XmcRAmj6AKDWa7Sx15wygoTc+/wOfLZIpqi4awCg34SZ rkq1IEjdqu1zx0B5QyFW/n0= =V0oF -----END PGP SIGNATURE----- ---------------------------(end of broadcast)--------------------------- TIP 6: explain analyze is your friend
