you can do what you want, celery is a task queue "handler" on steroids.
with that you can pass messages, assign more workers if your "message
queue" is lagging behind (i.e. 2 workers if the queue contains < 100
updates, 8 workers if queue contains > 1000) and so on.
For basic task you can roll y
> I think I'm not mistaken by saying that celery is the most used library
> for this kind of operations, it's written in python and founded on rabbitMQ
> (also on others, but primarily rabbit) to handle queues. Seems huge, but
> it's fairly easy to setup (especially if you planned to use rabbi
postgresql definetely scales also with write intensive operations without
blocking.
homemade task queues are real funny to code but gets messy really
soon.blocking operations, tasks that fail and need (or don't) to be
requeued, priorities, timeouts, newtork splits, and so on.
I think I'm not
> I strongly advicee the use of the scheduler because your requirements will
> be fullfilled best from that than a homemade task queue, at least, if not
> trying to use celery.anyway, just my 2 cents:
> SQLITE write operations are locking the entire database. one of 2
> controllers
yes, let me explain better.
Having a task queue that updates heavily a sqlite database while a web
application needs to read it is not a good idea. Sqlite is a wonderful
database and supports some syntax that others "big databases" dream about,
has transactions, is flexible, multiplatform,
>
> 1. with every other database that is not sqlite, it's safe because how
> most relational db work, transactions make you have always a consistent set
> of results (if you use them correctly!!)
It's still "safe" in SQLite -- you just might get a little more blocking
than you would with othe
I strongly advicee the use of the scheduler because your requirements will
be fullfilled best from that than a homemade task queue, at least, if not
trying to use celery.anyway, just my 2 cents:
SQLITE write operations are locking the entire database. one of 2
controllers (or modules
7 matches
Mail list logo
