Re: Simple task dispatching (How heavy is celery + RabbitMQ)

2011-11-28 Thread Brian Schott
There are plenty of tutorials about memory management with RabbitMQ. There are mechanisms for setting the high watermark for memory to throttle producers. You could look at the database backend for django-kombu. It might have a smaller footprint. django-ztask looks interesting, but you don'

Re: Simple task dispatching (How heavy is celery + RabbitMQ)

2011-11-28 Thread Gelonida N
On 11/28/2011 12:14 PM, Gelonida N wrote: > Hi, > > I'd like to use a light weight dispatching system for a web server. > > Basically some django post requests might require processing, which > should be done in the back ground due to its run times. > > The results would be added to the django d

Re: Simple task dispatching (How heavy is celery + RabbitMQ)

2011-11-28 Thread Roberto De Ioris
Il giorno 28/nov/2011, alle ore 12:14, Gelonida N ha scritto: > Hi, > > I'd like to use a light weight dispatching system for a web server. > > Basically some django post requests might require processing, which > should be done in the back ground due to its run times. > > The results would be

Re: Simple task dispatching (How heavy is celery + RabbitMQ)

2011-11-28 Thread Andre Terra
I'm also using redis for both brokering and caching. So far, so good. My use case consists in one HUGE (5 hours) task, and several tiny (30s or less) scheduled tasks. Cheers, AT On Mon, Nov 28, 2011 at 10:54 AM, Carlos Daniel Ruvalcaba Valenzuela < clsdan...@gmail.com> wrote: > Celery has a bun

Re: Simple task dispatching (How heavy is celery + RabbitMQ)

2011-11-28 Thread Carlos Daniel Ruvalcaba Valenzuela
Celery has a bunch of alternative brokers, depending on your constrains you may not need something as "heavy" as RabbitMQ, you could for example use db broker if you don't have many tasks, it is slower but does not add a new server (apart from celery) to your stack, other option is to use Redis as

Re: Simple task dispatching (How heavy is celery + RabbitMQ)

2011-11-28 Thread Markus Gattol
You don't need to roll your own... have a look at https://github.com/dmgctrl/django-ztask It's based on ZeroMQ and really lightweight compared to any alternative with a broker (yes, it's brokerless but that's a good thing I think for most use cases which don't really need a broker). -- You re

Re: Simple task dispatching (How heavy is celery + RabbitMQ)

2011-11-28 Thread Tom Evans
On Mon, Nov 28, 2011 at 11:14 AM, Gelonida N wrote: > Hi, > > I'd like to use a light weight dispatching system for a web server. > > Basically some django post requests might require processing, which > should be done in the back ground due to its run times. > > The results would be added to the