On Fri, 28 Nov 2014, Johannes Oettl wrote:
Icinga Stats 1.9.1 Copyright (c) 2009 Nagios Core Development Team and Community Contributors
If memory serves, the notification system is single-threaded, which means that if you're shoving 200-odd notifications into it all at once it's going to take some time to process those and will block whilst doing so. You might be able to get some relief by putting the actual process for notification off into the background (using an '&') but must beware that you might run into resource constraints if you're on a tight system. A meta-question to this might be, "What's causing 200-odd objects to go off-normal, and is there a correlation?" That's a heck of a lot of noise to pass to the sysadmins, and if a correlation can be found it'd be vastly better to notify on the correlation and suppress the 200-odd notifications (the root cause implies that all the children will suffer). The mantra should go, "Notify only on what's relevant to fixing the problem and which is actionable." This means not notifying when a hundred things fail because a parent function failed -- notify on the parent and suppress the noise from the children. Failure to do this results in a bad signal-to-noise ratio from the monitoring suite and causes the humans to ignore things. Getting it right remains a bit of a black art, and managers/bosses are typically unwilling to allocate the time required to really define the fault-tree; this just goes hand-in-hand with the task. Homework done up-front really does pay high dividends down the road. Cheers! +------------------------------------------------+---------------------+ | Carl Richard Friend (UNIX Sysadmin) | West Boylston | | Minicomputer Collector / Enthusiast | Massachusetts, USA | | mailto:crfri...@rcn.com +---------------------+ | http://users.rcn.com/crfriend/museum | ICBM: 42:22N 71:47W | +------------------------------------------------+---------------------+ _______________________________________________ icinga-users mailing list icinga-users@lists.icinga.org https://lists.icinga.org/mailman/listinfo/icinga-users
