On 05/25/2010 11:01 AM, Phil Pennock wrote: > On 2010-05-25 at 07:31 -0400, Joe McDonagh wrote: > >> I just finished integrating puppet with nagios so that nagios configs >> are automagically built by puppet. It's a little different than what >> you're saying but cool nonetheless. If I include an apache class, an >> apache service checks gets setup on my nagios master, stuff like that. >> > So, if service Foo gets droped from your puppet configs somehow, through > misconfiguration, your monitoring will stop and the service can go down > without you knowing about it? > No- removal from nagios is still manual. During a node's decommissioning, you will still have to remove the node definition, remove it from stored configs and remove its configs from the nagios box. If it's set up once with puppet, theoretically you could stop ever running puppet again and it wouldn't get removed from the nagios configs. > There's an argument for "belt and braces" -- certainly, some of what you > monitor can and should be autogenerated from the canonical definition of > what should exist; eg, machines, etc. But you want some level of > monitoring based on a separate and simpler list of "these are the > services we expect to exist". > Right- which is why in my setup I have made room for easily generating configs such as cisco asa's that can't run puppet. > You can then have a separate audit tool, perhaps invoked at VCS commit > time, which notifies you of inconsistencies. > Yea, I am with you on that. I'd like to write something that goes through various services, like puppet, dns, and nagios, then notifies you of inconsistent IP addresses, services, etc. Like a node diff tool. Something might already exist, but I've been toying with that idea for some time now. > This may just be a difference of opinion with no right or wrong answer, > but your approach scares me a little. > It's fairly common for people to be scared of automation. IMO, as long as the tool(s) is/are reliable and the administrator has a deep understanding of the automation, risk is low. > -Phil >
The automatic configuration of nagios via your stored configs setup was mentioned in passing in my e-mail but the truth is, it takes kind of a lot of effort and a fairly good understanding of both puppet and nagios. I know people build their nagios configs with cron jobs and perl scripts, which scares me a lot more than this approach. However, I probably shouldn't have played it down like "by the way"... it's just one of those features of puppet that I find attractive, because I used to run into inconsistencies like forgetting to update IP addresses etc.. It took me about a week of project time (2-6 hours a day, largely fluxuates) to get it implemented, and I actually still need to fully complete the service escalations. -- Joe McDonagh Operations Engineer AIM: YoosingYoonickz IRC: joe-mac on freenode "When the going gets weird, the weird turn pro." _______________________________________________ Discuss mailing list Discuss@lopsa.org http://lopsa.org/cgi-bin/mailman/listinfo/discuss This list provided by the League of Professional System Administrators http://lopsa.org/
