I think I posted the relevant number of queries issued to the backend for a given number of workers but I do daily use the scheduler on an mssql db and it can easily handle at least 10 workers (with the default heartbeat). Locking kicks in maybe once or twice a day, which means 1 or 2 on 28800 occasions, which is a pretty damn low number :P Of course the backend *should* be able to sustain concurrency, but on a minimal server with very low specs 6 or 7 workers should absolutely pose no threats at all. For 5 workers all that is needed is a backend being able to handle 240 transactions per minute!
On Thursday, January 26, 2017 at 7:47:51 PM UTC+1, Dave S wrote: > > On Thursday, January 26, 2017 at 9:45:20 AM UTC-8, Jason Solack wrote: >> >> using mssql, the code itself is in gluon scheduler.py - this happens with >> no interaction from the app >> >> > How do you instantiate the Scheduler? > > Is the mssql engine on the same machine as any of the web2py nodes? Are > there non-web2py connections to it? > > /dps > > >> On Thursday, January 26, 2017 at 12:03:41 PM UTC-5, Dave S wrote: >>> >>> >>> >>> On Thursday, January 26, 2017 at 8:44:25 AM UTC-8, Jason Solack wrote: >>>> >>>> So the issue is we run 6 workers on a machine and it works. If we do 3 >>>> workers on 2 machines we get deadlocks. That is no exaggeration - 6 >>>> records in our worker table and we're getting dealocks. >>>> >>>> >>> Which DB are you using? Can you show your relevant code? >>> >>> /dps >>> >>> >>>> On Wednesday, January 25, 2017 at 3:05:37 AM UTC-5, Niphlod wrote: >>>>> >>>>> you *should* have one different db for each environment. Each >>>>> scheduler tied to the same db will process incoming tasks, and it doesn't >>>>> matter what app effectively pushes them. >>>>> This is good if you want to have a single scheduler (which can be >>>>> composed by several workers) serving many apps, but *generally* you don't >>>>> want to *merge* prod and beta apps. >>>>> >>>>> The is_ticker bit is fine: only one worker tied to a db is elegible to >>>>> be a ticker, which is the one process than manages asssigning tasks (to >>>>> itself AND to other available workers). >>>>> Locking, once in a while, can happen and is self-healed. Continuous >>>>> locking is not good: either you have too many workers tied to the db OR >>>>> your db isn't processing concurrency at the rate that it needs. >>>>> SQLite can handle at most 2 or 3 workers. All the other "solid" >>>>> backends can manage up to 10, 15 at most. >>>>> If you wanna go higher, you need to turn to the redis-backed scheduler. >>>>> >>>>> On Tuesday, January 24, 2017 at 10:59:31 PM UTC+1, Jason Solack wrote: >>>>>> >>>>>> Hello all, >>>>>> >>>>>> I'm having some re-occurring issue with the scheduler. We are >>>>>> currently running multiple environments (production, beta) and have >>>>>> several >>>>>> nodes in each environment. If we have scheduler services running on all >>>>>> machines on each node we get a lot of deadlock errors. If we drop each >>>>>> environment down to one node we get no deadlock errors. I am noticing >>>>>> the >>>>>> field "is_ticker" in the worker table will only have one ticker across >>>>>> all >>>>>> the workers (spanning environments). Is that the expected behavior? I >>>>>> don't see any documentation about the ticker field so i'm not sure what >>>>>> to >>>>>> expect from that. >>>>>> >>>>>> Also is there any best practices about running the scheduler in an >>>>>> environment that i've described? >>>>>> >>>>>> Thanks in advance >>>>>> >>>>>> Jason >>>>>> >>>>> -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
