Thanks, Is there a way to put the process in a different thread or background job directly from django view (without going through a queue and cron)? I heard Python is not very good with multi threading because of GIL. I am on Python 2.5
On Aug 15, 6:30 pm, Javier Guerra Giraldez <jav...@guerrag.com> wrote: > On Sat, Aug 14, 2010 at 1:28 PM, ydjango <traderash...@gmail.com> wrote: > > I have a online user initiated synchronous process which runs anywhere > > between 1-5 minutes and gives user status message at the end. It is a > > very DB intensive process that reads and updates lots of mysql rows > > and does many calculations. The process is run as part of a view > > method. > > long-running tasks don't belong to the request-responce cycle, send > them to a background process. > > one of the easiest solutions is to use "Ghetto Queues". it can look like this: > > - create a new 'task' table, with fields to specify how to process a > task and a status field (preparing, ready, processing, done, failed), > if needed a 'result' field and if possible a 'percent done' field too. > > - when the user initiates a task, don't process it in the view > function. just add a new record to the task table, with all the info > needed to process it and set in 'ready' state. then return. don't > forget to note the task record ID somewhere you can relate to the > user's action. > > - another process, separate from the Django server (can be a cron task > in many cases) checks the task table and picks any record in 'ready' > state to do all the needed processing, first changing it to > 'processing' state. it should update the 'percent done' field, and > set to 'done' or 'failed' when finished. > > - make some way for the user to get the status of the task, either > revisiting the same URL (which should show the status if there's a > task in 'processing'), or via some AJAX query. > > there are many variations of this strategy: you can have several > 'workers' if you pay attention to the way it picks a task and sets to > 'processing' (transactions make it easy to atomically pick one), if > you need low latency between queuing a task and picking it by the > worker process, you can replace the task table with a queue manager > (check carrot and/or celery) > > -- > Javier -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-us...@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
