On 05/08/2014 03:34 PM, Jonathan Gazeley wrote: > We run our puppet agents as daemons with runinterval set to 30m. This > works fine for the mostpart, but recently we took on three MariaDB > Galera cluster nodes. In order for the cluster to survive, at least one > node must be running at all times, i.e. you must not restart the MariaDB > service on all three nodes at the same time, e.g. by pushing a change to > the config file with puppet. > > With the runinterval set to 30m, it is unlikely that all three nodes > will restart their MariaDB service at the same time but not impossible. > It would be a Bad Thing if I pushed a config change, all three nodes did > their puppet run at the same time, and the database cluster disappeared. > > Is there a way of incorporating some sort of locking to ensure that only > one out of the three nodes can run puppet agent simultaneously? I have > half an idea based around using PuppetDB and something that checks a > variable in a run stage which is executed before main, which could cause > the run to abort if another node has locked a row. > > I suppose it would be possible to disable the agent from running as a > daemon and use cron, and the cron job could easily use a MySQL handle as > a locking device. But it still doesn't stop me from simply sshing to > each of the nodes and forcing a puppet run, and breaking the cluster. > > Has anyone done anything like this before? Hope to have some interesting > ideas from you all :) > > Cheers, > Jonathan
Hi, for this and other reasons, I have found cron to be more powerful than the background agent. You can take fine grained control over when or when not to run. For your specific problem, would it make sense to create a schedule for all of your mysql configuration items, and have that schedule be different on each cluster node, so that the possible times of application are disjoint? Note that this may make it Very Hard to deploy a config simultaneously using --test if you really have to. If the general approach does sound appealing to you, I figure this is likely solvable using a special environment on your master that overrides the schedules for that special occasion. HTH, Felix -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/536B894B.3020204%40alumni.tu-berlin.de. For more options, visit https://groups.google.com/d/optout.
