simplejson can be extended and it can call for example np.save. The problem is that the json protocol does not allow for cusom de- serialization of arbitrary types. I think it is best to serialize before passing the data to json and have the tasks do de- serialization.
Currently scheduler.py does not have a hook to pass a link to extensions but it is on my todo list. It should not be an issue. On Aug 10, 2:40 pm, G <glenn.calt...@gmail.com> wrote: > Massimo, > The scheduler.py is just what I needed for my application. The one > problem I have is that in most of my work, the data type I pass around > everywhere is a NumPy array, which I have found cannot be serialized > by SimpleJSON. Numpy has it's own serialization which works very well. > It seems like it might be easiest to extend SimpleJSON to first > serialize a numpy array to a binary string using np.save and then pass > the string to the JSON encoder. Will this work for the binary strings? > The other issue is how to decide on the receiving end if the string > should be passed to np.load. > > I appreciate any ideas. > Thanks, > G > > On Aug 8, 7:28 am, Massimo Di Pierro <massimo.dipie...@gmail.com> > wrote: > > > > > > > > > ## preambole > > > I have been working on porting django-celery to > > web2py-celery.http://code.google.com/p/web2py-celery > > There are a few issues to resolve and I am working on it. > > > Yet I found it to be overkill for most users. It has lots of > > dependencies (for example RabbitMQ) and it is not easy to manage. If > > you do not need a huge number of worker nodes there may be a better > > solution. > > > So I added this to trunk: > > > gluon/scheduler.py > > > This email is a request for comments as I think this should replace te > > current cron mechanism. > > > ## What is it? > > It is a lightweight replacement for celery that uses the database > > instead of queues to schedule tasks and uses the default web2py admin > > interface to allow you to schedule tasks. It consists of a single file > > and has no dependencies. > > > ## How does it work? > > > For any existing > > app > > > Create File: app/models/scheduler.py > > ====== > > from gluon.scheduler import > > Scheduler > > > def > > demo1(*args,**vars): > > print 'you passed args=%s and vars=%s' % (args, > > vars) > > return > > 'done!' > > > def > > demo2(): > > > 1/0 > > > scheduler = > > Scheduler(db,dict(demo1=demo1,demo2=demo2)) > > ===================================== > > > Create File: app/modules/scheduler.py > > ====== > > scheduler.worker_loop() > > ===================================== > > > ## run worker nodes > > with: > > python web2py.py -S app -M -N -R applications/app/modules/ > > scheduler.py > > > ## schedule jobs > > usinghttp://127.0.0.1:8000/scheduler/appadmin/insert/db/task_scheduled > > > ## monitor scheduled > > jobshttp://127.0.0.1:8000/scheduler/appadmin/select/db?query=db.task_sche... > > > ## view completed > > jobshttp://127.0.0.1:8000/scheduler/appadmin/select/db?query=db.task_run.... > > > Compared to celery it lacks the ability to bind tasks and workers , > > remotely interrupt tasks and set timeout, yet these features can be > > added easily and I will so eventually. > > > Please let me know what you think. > > > Massimo
