I'm totally interested in solutions! It's a big problem I need to solve. The recurring maintenance task does not fix the initialization problem—because now you need to initialize the recurring maintenance task. This results in the same race condition. It does fine with the 40,000 records problem. But it's just a lot of complexity we're introducing to solve a simple problem (looping tasks) with a complex solution (scheduler).
I'd still love to find a clean way to do this. Maybe we should extend the scheduler like this: • Add a daemon_tasks parameter when you call it from models "Scheduler(db, daemon_tasks=[func1, func2])" • When scheduler boots up, it handles locks and everything and ensures there are two tasks that just call these functions • Then it dispatches the workers processes as usual ...ah, shoot, looking in widget.py, it looks like the code that starts schedulers doesn't have access to the parameters passed to Scheduler() because models haven't been run yet. Hmph. On Wednesday, June 27, 2012 12:56:52 AM UTC-7, Niphlod wrote: > > I don't know if continuing to give you fixes and alternative > implementations is to be considered as harassment at this point, stop me if > you're not interested into those. > > There is a very biiig problem in your statements: if your vision is > > Woohoo, this scheduler will *automatically handle locks*—so I don't need > to worry about stray background processes running in parallel > automatically, and it will *automatically start/stop the processes* with > the web2py server with -K, which makes it much easier to deploy the code! > > then the scheduler is the right tool for you. it's your app that doesn't > handle locks, because of your initialization code put into models. > > At least 2 of your problems (initialization and 40,000 scheduler_run > records) could be fixed by a "recurring maintenance" task that will do > check_daemon() without advisory locks and prune the scheduler_run table. > > BTW: I'm pretty sure that when you say "scheduler should be terminated > alongside web2py" you're not perfectly grasping how webdevelopment in > production works. If you're using "standalone" versions, i.e. not mounted > on a webserver, you can start your instances as web2py -a mypassword & > web2py -K myapp and I'm pretty sure when hitting ctrl+c both will shutdown. >
