i can confirm that size of the queued records has something to do with
delay to process different queues... once i deleted all outstanding records
from main group, fast_track group started working as expected... sorry for
a long thread, but i think it's a very neat idea to load scheduler with
lots of records which will be processed in a back-end process, while
fast-track works in it's own (faster) pace...
On Sat, Oct 20, 2012 at 12:03 PM, Adnan Smajlovic <adnan.smajlo...@gmail.com
> wrote:
> couple things happened...
>
> The main group worker got created even though I didn't call it... Not sure
> why, but i guess because there are lot of leftover tasks queued (500k) and
> some were assigned when I stopped the process.
>
> Even though "fast_tack" worker started, nothing is getting picked,
> assigned nor completed... I checked the time, conditions, and it should be
> picked...
>
> Only main are getting assigned, but they are not getting completed, even
> though they were completing when I had only main group specified, without
> fast_track, and slow_track
>
> exec python26 /opt/web-apps/web2py/web2py.py -K
> crm:fast_track,crm:slow_track &> /var/log/web2py-scheduler.log
>
>
>
> On Sat, Oct 20, 2012 at 11:13 AM, Adnan Smajlovic <
> adnan.smajlo...@gmail.com> wrote:
>
>> all clear :) in process of implementing.
>>
>> Is new api defined in scheduler.py, since i don't see it in there (2.1.1
>> (2012-10-17 17:00:46) dev), but I'm modifying the existing code to
>> employ fast_track, since order confirmations are getting behind. This will
>> be really good :) Thanks again, and again...
>>
>> On Sat, Oct 20, 2012 at 10:37 AM, Niphlod <niph...@gmail.com> wrote:
>>
>>> no prio available (it's hard to manage.... a task queued 3 hours ago
>>> with prio 7 comes before of after one with prio 8 queued 2 hours ago ?).
>>>
>>> "hackish way": tasks are picked up ordered by next_run_time. So, queue
>>> your tasks with next_runtime = request.now - datetime.timedelta(hours=1)
>>> kinda works.
>>>
>>> Right way: separate queues, "important tasks" and "less important
>>> tasks". You can create different queues assigning different group_name to
>>> tasks and start - at least 2 - separate scheduler processes. By default
>>> tasks are in the group 'main', and the scheduler worker processes those only
>>>
>>>
>>> Then, start one scheduler per queue with
>>> web2py.py -K appname:fast_track,appname:
>>>
>>> def task1(a, b=2):
>>> #need high prio
>>>
>>> def task2(a, b=2):
>>> #needs low prio
>>>
>>> from gluon.scheduler import Scheduler
>>> mysched = Scheduler(db)
>>>
>>> #new api
>>> mysched.queue_task(task1, ['a'], {'b': 1}, group_name='fast_track')
>>> mysched.queue_task(task2, ['a'], {'b' : 1}, group_name='slow_track')
>>>
>>> #old api
>>> from gluon.serializers import json
>>> db.scheduler_task.validate_and_insert(function_name='task1', args=json([
>>> 'a']), vars=json({'b':1}), group_name='fast_track')
>>> db.scheduler_task.validate_and_insert(function_name='task2', args=json([
>>> 'a']), vars=json({'b':1}), group_name='slow_track')
>>>
>>> slow_track
>>>
>>> If you just need some important tasks without assignign "slow_track" to
>>> the zillions you have already, just forget about the
>>> group_name='slow_track' and start schedulers with this command line
>>> web2py.py -K appname,appname:fast_track
>>> Then assign to fast_track only the ones you want to exec first and,
>>> assuming that fast_track has less tasks in queue, they will be executed
>>> before the zillion ones in the main group.
>>>
>>> Clear ?
>>>
>>>
--
