Hi, I am running a Icinga 1.9.1 installation with IDO, PNP4Nagios and NoMa (Netways Notification Manager).
Stats: Icinga Stats 1.9.1 Copyright (c) 2009 Nagios Core Development Team and Community Contributors Copyright (c) 1999-2009 Ethan Galstad Last Modified: 05-22-2013 License: GPL CURRENT STATUS DATA ------------------------------------------------------ Status File: /var/lib/icinga/status.dat Status File Age: 0d 0h 0m 2s Status File Version: 1.9.1 Program Running Time: 0d 0h 57m 34s Icinga PID: 13488 Used/High/Total Command Buffers: 0 / 714 / 32768 Total Services: 11724 Services Checked: 11724 Services Scheduled: 7585 Services Actively Checked: 7586 Services Passively Checked: 4138 Total Service State Change: 0.000 / 26.180 / 0.009 % Active Service Latency: 0.001 / 0.626 / 0.173 sec Active Service Execution Time: 0.004 / 15.314 / 1.740 sec Active Service State Change: 0.000 / 17.630 / 0.008 % Active Services Last 1/5/15/60 min: 1500 / 7326 / 7532 / 7586 Passive Service Latency: 0.061 / 11.296 / 1.508 sec Passive Service State Change: 0.000 / 26.180 / 0.012 % Passive Services Last 1/5/15/60 min: 478 / 3941 / 4021 / 4043 Services Ok/Warn/Unk/Crit: 11682 / 13 / 3 / 26 Services Flapping: 0 Services In Downtime: 2 Total Hosts: 2067 Hosts Checked: 2066 Hosts Scheduled: 0 Hosts Actively Checked: 2067 Host Passively Checked: 0 Total Host State Change: 0.000 / 11.450 / 0.013 % Active Host Latency: 0.000 / 0.624 / 0.000 sec Active Host Execution Time: 0.000 / 3.038 / 0.023 sec Active Host State Change: 0.000 / 11.450 / 0.013 % Active Hosts Last 1/5/15/60 min: 0 / 1 / 3 / 13 Passive Host Latency: 0.000 / 0.000 / 0.000 sec Passive Host State Change: 0.000 / 0.000 / 0.000 % Passive Hosts Last 1/5/15/60 min: 0 / 0 / 0 / 0 Hosts Up/Down/Unreach: 2064 / 3 / 0 Hosts Flapping: 0 Hosts In Downtime: 0 Active Host Checks Last 1/5/15 min: 8 / 36 / 102 Scheduled: 0 / 0 / 0 On-demand: 8 / 36 / 102 Parallel: 0 / 1 / 3 Serial: 0 / 0 / 0 Cached: 8 / 35 / 99 Passive Host Checks Last 1/5/15 min: 0 / 0 / 0 Active Service Checks Last 1/5/15 min: 1661 / 7552 / 22640 Scheduled: 1661 / 7552 / 22640 On-demand: 0 / 0 / 0 Cached: 0 / 0 / 0 Passive Service Checks Last 1/5/15 min: 370 / 388 / 388 External Commands Last 1/5/15 min: 890 / 4359 / 12693 As you can see, I use a lot of passive checks (Check-mk), I wrote some similar checks for Cisco Router to have only one SNMP check per Host, and submit the results as passive checks. I am getting into high latency, when i submit a lot of check results in the command pipe, and each result creates a notification. When I write 200 Check results in the command pipe, a notification is send out every 3 seconds, so for 200 checks it takes 600 seconds to process them all. During this time I got a very high latency, and the load of the maschine goes up. I also found this in the icinga.log: # grep -i reaper icinga.log [1417156402] Warning: Breaking out of check result reaper: max reaper time (60) exceeded. Reaped 53 results, but more checkresults to process. [1417156468] Warning: Breaking out of check result reaper: max reaper time (60) exceeded. Reaped 56 results, but more checkresults to process. [1417156531] Warning: Breaking out of check result reaper: max reaper time (60) exceeded. Reaped 24 results, but more checkresults to process. [1417156597] Warning: Breaking out of check result reaper: max reaper time (60) exceeded. Reaped 37 results, but more checkresults to process. [1417156761] Warning: Breaking out of check result reaper: max reaper time (60) exceeded. Reaped 89 results, but more checkresults to process. [1417156824] Warning: Breaking out of check result reaper: max reaper time (60) exceeded. Reaped 19 results, but more checkresults to process. [1417163827] Warning: Breaking out of check result reaper: max reaper time (60) exceeded. Reaped 40 results, but more checkresults to process. [1417164030] Warning: Breaking out of check result reaper: max reaper time (60) exceeded. Reaped 64 results, but more checkresults to process. [1417164096] Warning: Breaking out of check result reaper: max reaper time (60) exceeded. Reaped 58 results, but more checkresults to process. reaper setings in icinga.conf: # HOST AND SERVICE CHECK REAPER FREQUENCY #check_result_reaper_frequency=1 #check_result_reaper_frequency=10 check_result_reaper_frequency=5 # MAX CHECK RESULT REAPER TIME # check result reaper event will be allowed to run before max_check_result_reaper_time=60 Has anyone else seen such behaviour? Any suggestions, what I can do to eliminate this problem? regards, Johannes _______________________________________________ icinga-users mailing list icinga-users@lists.icinga.org https://lists.icinga.org/mailman/listinfo/icinga-users
