So I've been struggling with this issue for a while. Here's the scenario:
The user selects their desired options and then clicks submit. The
javascript then sends a POST request to a controller that adds a task to
the task database. The javascript then polls every second to check on the
status of the task and when the task is complete the results are sent back.
All of the tasks take max 1-2 seconds to run. However, it usually takes
around 6 seconds for the scheduler to assign the task to a worker and for
the worker to actually pick up and run the task (heartbeat is set to 2).
Here's a log showing what I mean:
2016-06-02 01:33:00,657 - DEBUG - Adding task to scheduler
2016-06-02 01:33:00,670 - DEBUG - Done adding task to scheduler
2016-06-02 01:33:01,060 - DEBUG - scheduler_task: QUEUED
2016-06-02 01:33:03,253 - DEBUG - scheduler_task: QUEUED
2016-06-02 01:33:05,198 - DEBUG - scheduler_task: QUEUED
2016-06-02 01:33:07,196 - DEBUG - scheduler_task: ASSIGNED
2016-06-02 01:33:09,280 - DEBUG - new task report: COMPLETED
The time it takes from "Done adding task to schedule" to "new task report:
COMPLETED" is about 8 seconds. Given that the tasks only take max 1-2
seconds to run I would really like to cut down on the time it takes for it
to be assigned and picked up by a worker.
Here's some more information (if you need anymore please tell me!):
In models I have *scheduler.py* where I initialize the scheduler:
import resultEngine // importing from the web2py modules directory
scheduler = Scheduler(taskdb, tasks=dict(getResult=resultEngine.getResult),
heartbeat = 2)
In *default.py*:
def queueTask():
session.task = scheduler.queue_task('getResult', [options_list],
timeout=60, immediate=True)
I'm using sqlite as the database for the scheduler, I'm running my webapp
on pythonanywhere and have 5 workers.
Is there a way to cut down on the time for a task to actually be picked up
and run by a worker (to like 1 second)?. I spent a lot of time optimizing
the tasks so that they could be run in 1-2 seconds so I really want to show
snappy behaviour. Any help is appreciated! Loving web2py :)!
--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
