Cool, thanks Javier. I will look into the different solutions you suggest. Cheers, Pau
On Thursday, November 28, 2013 9:32:15 AM UTC+1, Javier Guerra wrote: > > On Thu, Nov 28, 2013 at 2:42 AM, Pau Creixell > <paucr...@gmail.com<javascript:>> > wrote: > > I guess if there isn't another simpler way, I'll have to dig into > Celery. > > > there _are_ simpler alternatives to Celery: > > - worker threads: similar to your proposal, but with very well-thought > process control. pros: no extra process. cons: very hard to get it > even half right, very easy to kill your web workers with the lightest > load. probably unsupported by hostings. > > - ghetto queues: a db table + a cron process. pros: simple, easy to > debug. cons: has a hard ceiling, if the queue grows over some (hard > to estimate) limit, it grinds to a halt. > > - uWSGI spool: extra uWSGI containers that work on a queue of tasks. > pros: easy to use, you get the process management by uWSGI, no extra > tools. cons: limited semantics, might not be supported by hosts. > > - simple brokers: for example pub/sub or other structures in Redis. > pros: easier to use and understand than celery, might be lighter on > memory. cons: no standarization, needs a broker process (at least if > you're not already running Redis), might not be supported by hostings. > > as you can see, in most cases the only reasonable answers are Celery > or ghetto queues. the last ones are an easy way out, and if done well > can serve to a surprising load level; but if you hit gold, it will > fall down. this scheme was a big part of the Twitter growing > problems. > > > -- > Javier > -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/2f42a10d-dfe5-48f0-ac16-5f0aa7e28967%40googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
