I think we have this scenario happening: https://groups.google.com/forum/#%21searchin/web2py/task_id%7csort:relevance/web2py/AYH5IzCIEMo/hY6aNplbGX8J
our workers seems to be restarting quickly and we're trying to figure out why On Thursday, August 18, 2016 at 3:55:55 AM UTC-4, Niphlod wrote: > > small recap.......a single worker is tasked with assigning tasks (the one > with is_ticker=True) and then that task is picked up only by the assigned > worker (you can see it on the scheduler_task.assigned_worker_name column of > the task). > There's no way the same task (i.e. a scheduler_task "row") is executed > while it is RUNNING (i.e. processed by some worker). > The process running the task is stored also in scheduler_run.worker_name. > > <tl;dr> you shouldn't EVER have scheduler_run records with the same > task_id and 12 different worker_name all in the RUNNING status. > > For a single task to be processed by ALL 12 workers at the same time... is > quite impossible, if everything is running smoothly. And frankly I can't > fathom any scenario in which it is possible. > > > On Wednesday, August 17, 2016 at 6:25:41 PM UTC+2, Jason Solack wrote: >> >> I only see the task_id in the scheduler_run table, it seems to be added >> as many times as it can while the run is going... a short run will add just >> 2 of the workers and stop adding them once the initial run is completed >> >> On Wednesday, August 17, 2016 at 11:15:52 AM UTC-4, Niphlod wrote: >>> >>> task assignment is quite "beefy" (sadly, or fortunately in your case, it >>> favours consistence vs speed) : I don't see any reason why a single task >>> gets picked up by ALL of the 12 workers at the same time if the backend >>> isn't lying (i.e. slaves not replicating master data),.... if your mssql is >>> "single", there shouldn't absolutely be those kind of problems... >>> >>> Are you sure all are crunching the same exact task (i.e. same task id >>> and uuid) ? >>> >>> On Wednesday, August 17, 2016 at 2:47:11 PM UTC+2, Jason Solack wrote: >>>> >>>> I'm using nginx and MSSQL for the db >>>> >>>> On Wednesday, August 17, 2016 at 3:11:11 AM UTC-4, Niphlod wrote: >>>>> >>>>> nothing in particular. what backend are you using ? >>>>> >>>>> On Tuesday, August 16, 2016 at 8:35:17 PM UTC+2, Jason Solack wrote: >>>>>> >>>>>> task = scheduler.queue_task(tab_run, >>>>>> pvars=dict(tab_file_name=tab_file_name, the_form_file=the_form_file), >>>>>> timeout=60 * 60 * 24, sync_output=2, immediate=False, >>>>>> group_name=scheduler_group_name) >>>>>> >>>>>> anything look amiss here? >>>>>> >>>>>> On Tuesday, August 16, 2016 at 2:14:38 PM UTC-4, Dave S wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Tuesday, August 16, 2016 at 9:38:09 AM UTC-7, Jason Solack wrote: >>>>>>>> >>>>>>>> Hello all, i am having a situation where my scheduled jobs are >>>>>>>> being picked up by multiple workers. My last task was picked up by >>>>>>>> all 12 >>>>>>>> workers and is crushing the machines. This is a load balanced machine >>>>>>>> with >>>>>>>> 3 machine and 4 workers on each machine. has anyone experienced >>>>>>>> something >>>>>>>> like this? >>>>>>>> >>>>>>>> Thanks for your help in advance! >>>>>>>> >>>>>>>> jason >>>>>>>> >>>>>>> >>>>>>> What does your queue_task() code look like? >>>>>>> >>>>>>> /dps >>>>>>> >>>>>>> >>>>>> -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
