chris van wrote: > > Date: Tue, 3 Mar 2009 12:57:39 -0800 > > Subject: Pylons, HTTP 201 Accepted, Task Queues and Background threads > > From: [email protected] > > To: [email protected] > > > > Hi everyone, > > > > I'm trying to find some docs or perhaps old discussions about > > implementing a task queue within a pylons application. The scenario > > I'm trying to support involves a request coming into the app server to > > perform an action which takes a long time to complete, such as > > rebuilding an index or updating a value across hundreds of thousands > > of objects. > > > > My thought was to create a processing thread when the app is loaded. > > When I get the request I add the item to a synchronized queue (which > > the processing thread blocks on) and return a HTTP 201 Accepted to the > > client. The processing thread picks up tasks from the queue and they > > are completed in the order received. The 201 response also has an > > additional Location header to poll the status of the task. > > > > The question that remained was how to create and manage processing > > thread. I've read a couple of threads on this subject, and hunted > > around google a bit and found a couple of options: > > - > > > http://groups.google.com/group/pylons-discuss/browse_thread/thread/e30fb912ca79b000/7cc1d4a6b1d9919d?lnk=gst&q=background#7cc1d4a6b1d9919d > > - > > > http://groups.google.com/group/pylons-discuss/browse_thread/thread/3e9dfda05af50634/bc914b96e2b96a1b?lnk=gst&q=background#bc914b96e2b96a1b > > > > Now I'm leaning towards creating a process using the python > > multiprocessing module which interfaces like a thread but skips issues > > with the GIL and pylons thread management. However, I didn't find any > > information about how to manage the process lifecycle and allow it to > > shutdown gracefully when the server is stopped. > > > > I'd appreciate feedback on this approach and any pointers to resources > > that will allow me to hook into the app lifecycle and manage my > > subprocess as well. Hopefully I can get a working recipe out of this > > and put it all together in the pylons cook book for future reference. > > > > Thanks. > > G'day, > > I know nothing about how to solve your problem, though it does sound > like something that interests me and which I would like to do myself. I > currently have a controller feeding data into a shared-memory circular > buffer, from which an entirely separate application reads and executes > in it's own time. I return a 400 OK to the client as soon as the > controller is finished validating the request and stores it in the > buffer; the client must then poll the server to get updates on the > status of the request once it is in the buffer. Returning 201 Accepted > sounds like a better solution, could you please let me know how > successful you are in implementing it under pylons and how it works for you? > > Thanks a lot,
Chris, Two questions? What happens to the external application when the pylons application is shut down? Does it empty the buffer and block for new input or do you have a way to shut down the external application from pylons? Secondly, do clients poll the pylons server or the external application for status? I'm guessing you meant you return 200 OK not 400. Whether you choose to return a 200 or 201 status is mostly an aesthetic choice, though 201 is a clearer indication of the result. If you're trying to change the response from 200 to 201 you can manipulate the pylons.response to set its status_code to 201. Hope this helps you out. Kochhar --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "pylons-discuss" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~----------~----~----~----~------~----~------~--~---
