Hi Niphlod, thanks for the great work with the scheduler, I'm using it in a project where it handles lots of big data imports into a database and the migration to your version was without any problems.
On thing catched my eye in the old version and it still seems to be a "problem/missing feature" in the new one. When a long running process gets executed and produces output (print etc.) this output is written to the database only after the task was run (and finished). It would be really great if the output gets written into the task table while the task runs as this would be a possible feedback mechanism (and we would not need another table etc. just for that) just thinking of a progress meter for example. What I really miss though is the output of the task when it produces a timeout - nothing in the task table about the output... Daniel Am Donnerstag, 12. Juli 2012 20:36:38 UTC schrieb Niphlod: > > Hello everybody, in the last month several changes were commited to the > scheduler, in order to improve it. > Table schemas were changed, to add some features that were missed by some > users. > On the verge of releasing web2py v.2.0.0, and seeing that the scheduler > potential is often missed by regular web2py users, I created a test app > with two main objectives: documenting the new scheduler and test the > features. > > App is available on github (https://github.com/niphlod/w2p_scheduler_tests). > All you need is download the trunk version of web2py, download the app and > play with it. > > Current features: > - one-time-only tasks > - recurring tasks > - possibility to schedule functions at a given time > - possibility to schedule recurring tasks with a stop_time > - can operate distributed among machines, given a database reachable for > all workers > - group_names to "divide" tasks among different workers > - group_names can also influence the "percentage" of assigned tasks to > similar workers > - simple integration using modules for "embedded" tasks (i.e. you can use > functions defined in modules directly in your app or have them processed in > background) > - configurable heartbeat to reduce latency: with sane defaults and not > toooo many tasks queued normally a queued task doesn't exceed 5 seconds > execution times > - option to start it, process all available tasks and then die > automatically > - integrated tracebacks > - monitorable as state is saved on the db > - integrated app environment if started as web2py.py -K > - stop processes immediately (set them to "KILL") > - stop processes gracefully (set them to "TERMINATE") > - disable processes (set them to "DISABLED") > - functions that doesn't return results do not generate a scheduler_run > entry > - added a discard_results parameter that doesn't store results "no matter > what" > - added a uuid record to tasks to simplify checkings of "unique" tasks > - task_name is not required anymore > - you can skip passing the function to the scheduler istantiation: > functions can be dinamically retrieved in the app's environment > > So, your mission is: > - test the scheduler with the app and familiarize with it > Secondary mission is: > - report any bug you find here or on github ( > https://github.com/niphlod/w2p_scheduler_tests/issues) > - propose new examples to be embedded in the app, or correct the current > docs (English is not my mother tongue) > > Once approved, docs will be probably embedded in the book ( > http://web2py.com/book) > > Feel free to propose features you'd like to see in the scheduler, I have > some time to spend implementing it. > > > > --
