On Oct 21, 9:28 am, Mike Thon <mike.t...@gmail.com> wrote: > I'm new to web programming and I have a basic question about the > design of my Django application. my application will do some number > crunching on data files uploaded by users. The data processing will > take from minutes to hours for each job. I don't expect to ever get a > large number of concurrent users but I'd still like to set it up so > that I can control the maximum number of data processing jobs that are > run in parallel. I was planning to write a simple FIFO queue manager > (in fact I think there is a python package for this) and then run the > data processing in separate threads. I'm also planning to use the > Django data model for storing the data so I would have multiple > threads writing to the data store. What is not clear to me is what > happens when I have more than one visitor to the site. Are multiple > instances of my Django app launched, one per visitor? I need to > ensure that I only have one queue manager running on the server, not > one per visitor. I would be using Apache and either mySQL or sqlite3 > as the database, in case that matters. > > thanks for any help > Mike
Take a look at the Celery project[1]. This is a great distributed task queue for Django that I think will do exactly what you need - each job request is sent to the queue and managed there, so you don't need to worry about multiple instances. [1]:http://ask.github.com/celery/introduction.html -- DR. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~----------~----~----~----~------~----~------~--~---
