Thanks. I'll try using SQLite and WAL - that having been said I'd still be vote for getting the scheduler working with NoSQL (couchDB) at some stage.
Ted On Thursday, 26 July 2012, Niphlod wrote: > Couchdb support is still experimental in web2py. If auth is not working > you'd likely to wait (or help Massimo with) a better implementation of the > DAL. > > If you need ultra-fast tasks you'd want to use celery (or pyres, or rq). > The current scheduler with SQLite can process ~5 tasks per second per > single worker (YMMV for complex apps or functions). Multiple workers with > SQLite and WAL archiving disabled is not recommended. Anyway, with WAL > activated, I tested with no problems 8 concurrent workers with a great deal > of tasks (~6000) with a semi-linear scalability. > With Mysql or postgres it's possible to process tasks with even more > concurrent workers (I'd still keep them 50 at most). The only downside of > SQLite instead of a "full relational db" is that you can't share the db > among different machines. > > > > On Thursday, July 26, 2012 7:03:44 AM UTC+2, Ted wrote: > > Hi Niphlod, > > Since the usecase for a Scheduler often needs concurrent fast write/reads > and updates (from multiple clients) with information that is largely > temporary I think a Nosql DB is a great way to handle scheduler data. > > In my case I don't want the overhead of installing a full relational db > and I'm sure that the SQLite adapter will not perform well under load with > multiple scheduler tasks. > > Since I posted I have tried CouchDB as my Auth db and am getting similar > errors (at the same code point as above) about the encoding. I am using > Python 2.6, web2py Version 2.00.0 (2012-07-18 21:46:56) dev and CouchDB > 1.0.1. > > Ted > > > On Thursday, July 26, 2012 2:07:17 AM UTC+7, Niphlod wrote: > > Scheduler was meant to run on relational dbs. Actually, I think noone ever > tested it on Nosql adapters, but if this is a popular requirement I could > see into it. > > BTW, I posted an app to test the scheduler in trunk for this kind of tests. > > https://groups.google.com/d/**topic/web2py/E_mASrZIpvw/**discussion<https://groups.google.com/d/topic/web2py/E_mASrZIpvw/discussion> > > > > On Wednesday, July 25, 2012 1:42:50 PM UTC+2, Ted wrote: > > Hi Web2pyers, > > Has anyone sucessfully used the new scheduler functionality with * > CouchDB? *(I'm assuming this is feasible) > > I'm currently trying to use the scheduler on *Windows 7* with *CouchDB *as > the DAL source. > My code works fine with sqlite but seems to silently fail using *CouchDB*- > although the tables are correctly created. > > e.g. > > scheduler_db = DAL('couchdb://127.0.0.1:5984'**) > scheduler = Scheduler(scheduler_db,dict(**demo1=demo1,demo2=demo2),** > heartbeat=1) > > > (The *CouchDB* scheduler_worker table seems to mysteriously keep growing.) > > When I try to manually add a task through the Admin ( > http://127.0.0.1:8000/init/**appadmin/insert/scheduler_db/**scheduler_task<http://127.0.0.1:8000/init/appadmin/insert/scheduler_db/scheduler_task> > ) > > I get the following error: > > > Traceback (most recent call last): > File "C:\Users\ted\dev\web2py\**gluon\restricted.py", line 205, in > restricted > > > exec ccode in environment > File "C:/Users/ted/dev/Px/**applications/init/controllers/**appadmin.py" > <http://127.0.0.1:8000/admin/default/edit/init/controllers/appadmin.py>, line > 270, in <module> > > > File "C:\Users\ted\dev\web2py\**gluon\globals.py", line 182, in <lambda> > > > self._caller = lambda f: > > --
