I've just been able to record my master sending out notifications, with both
master and satellite running with --log-level=notice.
This occured on my satellite:
[2015-04-20 15:57:42 +0200] notice/CheckerComponent: Pending checkables: 0;
Idle checkables: 2; Checks/s: 0
[2015-04-20 15:57:42 +0200] notice/ApiListener: Setting log position for
identity 'master': 2015/04/20 15:57:20
[2015-04-20 15:57:42 +0200] notice/ApiListener: Current zone master: satellite-b
[2015-04-20 15:57:42 +0200] notice/ApiListener: Connected endpoints: master (1)
[2015-04-20 15:57:45 +0200] notice/ApiClient: Received 'log::SetLogPosition'
message from 'master'
[2015-04-20 15:57:46 +0200] notice/Process: PID 12230
('/usr/lib/nagios/plugins/check_ping' '-4' '-H' '1.2.3.4' '-c' '200,15%' '-w'
'100,5%') terminated with exit code 2
[2015-04-20 15:57:46 +0200] notice/ApiListener: Relaying 'event::CheckResult'
message
[2015-04-20 15:57:46 +0200] notice/ApiListener: Sending message to 'master'
[2015-04-20 15:57:47 +0200] notice/ApiListener: Relaying
'event::UpdateRepository' message
[2015-04-20 15:57:47 +0200] notice/ApiListener: Sending message to 'master'
[2015-04-20 15:57:47 +0200] notice/WorkQueue: #2 tasks: 0
[2015-04-20 15:57:47 +0200] notice/CheckerComponent: Pending checkables: 0;
Idle checkables: 2; Checks/s: 0.2
[2015-04-20 15:57:47 +0200] notice/ApiListener: Setting log position for
identity 'master': 2015/04/20 15:57:20
[2015-04-20 15:57:47 +0200] notice/ApiListener: Current zone master: satellite-b
[2015-04-20 15:57:47 +0200] notice/ApiListener: Connected endpoints: master (1)
[2015-04-20 15:57:47 +0200] notice/ThreadPool: Pool #1: Pending tasks: 0;
Average latency: 0ms; Threads: 4; Pool utilization: 0.0178634%
[2015-04-20 15:57:49 +0200] notice/ApiClient: Received 'event::Heartbeat'
message from 'master'
[2015-04-20 15:57:50 +0200] notice/ApiClient: Received 'log::SetLogPosition'
message from 'master'
[2015-04-20 15:57:50 +0200] notice/WorkQueue: #5 tasks: 0
[2015-04-20 15:57:52 +0200] notice/CheckerComponent: Pending checkables: 0;
Idle checkables: 2; Checks/s: 0.2
[2015-04-20 15:57:52 +0200] notice/ApiListener: Setting log position for
identity 'master': 2015/04/20 15:57:20
[2015-04-20 15:57:52 +0200] notice/ApiListener: Current zone master: satellite-b
[2015-04-20 15:57:52 +0200] notice/ApiListener: Connected endpoints: master (1)
[2015-04-20 15:57:55 +0200] notice/ApiClient: Received 'log::SetLogPosition'
message from 'master'
[2015-04-20 15:57:55 +0200] notice/ApiClient: Received
'event::SetNextNotification' message from 'master'
[2015-04-20 15:57:55 +0200] notice/ApiListener: Relaying
'event::SetNextNotification' message
[2015-04-20 15:57:57 +0200] notice/WorkQueue: #2 tasks: 0
[2015-04-20 15:57:57 +0200] notice/CheckerComponent: Pending checkables: 0;
Idle checkables: 2; Checks/s: 0.2
[2015-04-20 15:57:57 +0200] notice/ApiListener: Setting log position for
identity 'master': 2015/04/20 15:57:55
[2015-04-20 15:57:57 +0200] notice/ApiListener: Current zone master: satellite-b
[2015-04-20 15:57:57 +0200] notice/ApiListener: Connected endpoints: master (1)
[2015-04-20 15:57:59 +0200] notice/ApiClient: Received 'event::Heartbeat'
message from 'master'
[2015-04-20 15:58:00 +0200] notice/ApiClient: Received 'log::SetLogPosition'
message from 'master'
The master recorded the following:
[2015-04-20 15:57:50 +0200] notice/ApiListener: Setting log position for
identity 'satellite-b': 2015/04/20 15:57:47
[2015-04-20 15:57:50 +0200] notice/ApiListener: Current zone master: master
[2015-04-20 15:57:50 +0200] notice/ApiListener: Connected endpoints:
satellite-b (1)
[2015-04-20 15:57:50 +0200] notice/ApiListener: Relaying
'event::SetNextNotification' message
[2015-04-20 15:57:50 +0200] notice/ApiListener: Relaying
'event::SetNextNotification' message
[2015-04-20 15:57:50 +0200] notice/ApiListener: Relaying
'event::SetNextNotification' message
[2015-04-20 15:57:50 +0200] notice/ApiListener: Relaying
'event::SetNextNotification' message
[2015-04-20 15:57:50 +0200] notice/CheckerComponent: Pending checkables: 0;
Idle checkables: 29; Checks/s: 0.6
[2015-04-20 15:57:52 +0200] notice/ThreadPool: Thread pool; current: 3;
adjustment: -2
[2015-04-20 15:57:52 +0200] notice/ThreadPool: Thread pool; current: 2;
adjustment: -1
[2015-04-20 15:57:52 +0200] notice/ThreadPool: Thread pool; current: 3;
adjustment: -2
[2015-04-20 15:57:52 +0200] notice/ThreadPool: Thread pool; current: 3;
adjustment: -2
[2015-04-20 15:57:52 +0200] notice/ThreadPool: Thread pool; current: 2;
adjustment: -1
[2015-04-20 15:57:53 +0200] notice/ApiListener: Relaying 'event::SetNextCheck'
message
[2015-04-20 15:57:53 +0200] notice/ApiListener: Relaying 'event::SetNextCheck'
message
[2015-04-20 15:57:53 +0200] notice/ApiListener: Relaying 'event::CheckResult'
message
[2015-04-20 15:57:55 +0200] notice/ThreadPool: Thread pool; current: 4;
adjustment: -2
[2015-04-20 15:57:55 +0200] notice/ThreadPool: Thread pool; current: 4;
adjustment: -2
[2015-04-20 15:57:55 +0200] notice/ThreadPool: Thread pool; current: 3;
adjustment: -2
[2015-04-20 15:57:55 +0200] notice/CheckerComponent: Pending checkables: 0;
Idle checkables: 29; Checks/s: 0.6
[2015-04-20 15:57:55 +0200] notice/ApiListener: Setting log position for
identity 'satellite-b': 2015/04/20 15:57:47
[2015-04-20 15:57:55 +0200] notice/ApiListener: Current zone master: master
[2015-04-20 15:57:55 +0200] notice/ApiListener: Connected endpoints:
satellite-b (1)
[2015-04-20 15:57:55 +0200] notice/ApiListener: Relaying
'event::SetNextNotification' message
[2015-04-20 15:57:55 +0200] notice/ApiListener: Relaying
'event::SetNextNotification' message
[2015-04-20 15:57:55 +0200] notice/ApiListener: Relaying
'event::SetNextNotification' message
[2015-04-20 15:57:55 +0200] notice/ApiListener: Relaying
'event::SetNextNotification' message
[2015-04-20 15:57:55 +0200] notice/ApiListener: Relaying
'event::SetNextNotification' message
[2015-04-20 15:57:55 +0200] notice/ApiListener: Sending message to 'satellite-b'
[2015-04-20 15:57:55 +0200] notice/NotificationComponent: Attempting to send
reminder notification for object 'NotificationTest'
[2015-04-20 15:57:55 +0200] information/Notification: Attempting to send
notifications for notification object
'NotificationTest!dc1.test.host-notify-by-mail'.
[2015-04-20 15:57:55 +0200] information/Notification: Sending notification
'NotificationTest!dc1.test.host-notify-by-mail' for user 'icingaadmin'
[2015-04-20 15:57:55 +0200] notice/Process: Running command
'/etc/icinga2/scripts/mail-host-notification.sh': PID 31858
[2015-04-20 15:57:55 +0200] information/Notification: Completed sending
notification 'NotificationTest!dc1.test.host-notify-by-mail' for checkable
'NotificationTest'
[2015-04-20 15:57:55 +0200] notice/Process: PID 31858
('/etc/icinga2/scripts/mail-host-notification.sh') terminated with exit code 0
[2015-04-20 15:57:55 +0200] notice/ThreadPool: Thread pool; current: 2;
adjustment: -1
[2015-04-20 15:57:55 +0200] notice/ThreadPool: Thread pool; current: 2;
adjustment: -1
[2015-04-20 15:57:55 +0200] notice/ApiListener: Relaying 'event::SetNextCheck'
message
(I've excluded a lot of Dependency related log lines on the master for better
readability)
The notification shown in the log was sent by the master.
All notification objects look exactly the same on master and satellite-b.
What else information could i provide?
Regards,
Markus
-----Original message-----
> From:Michael Friedrich <michael.friedr...@netways.de
> <mailto:michael.friedr...@netways.de> >
> Sent: Friday 17th April 2015 15:59
> To: icinga-users@lists.icinga.org <mailto:icinga-users@lists.icinga.org>
> Subject: Re: [icinga-users] Restrict notifications to one specific
> (satellite) endpoint
>
> Can you also verify the generated notification objects by invoking
> 'icinga2 object list --type Notification --name *...*' on all involved
> nodes? There might be some mismatch too.
>
>
> Kind regards,
> Michael
>
> Am 17.04.2015 um 14:29 schrieb Markus Joosten:
> > Hi Michael,
> >
> > this morning i was able to reproduce the issue easily, unfortunately
> > didn't save the logs.
> > I just tried to reproduce it again, now all notifications are being sent
> > from the satellite endpoint properly. (Without any configuration being
> > changed)
> >
> > Nonetheless, here are my configs:
> >
> > zones.conf on master:
> > object Endpoint "master" {
> > }
> >
> > object Endpoint "satellite-a" {
> > host = "10.0.1.10"
> > }
> >
> > object Endpoint "satellite-b" {
> > host = "10.0.2.10"
> > }
> >
> > object Zone "root" {
> > endpoints = [ "master" ]
> > }
> >
> > object Zone "global-config" {
> > global = true
> > }
> >
> > object Zone "site-b" {
> > parent = "root"
> > endpoints = [ "satellite-b" ]
> > }
> > object Zone "site-c" {
> > parent = "root"
> > endpoints = [ "satellite-c" ]
> > }
> >
> > I'll just focus on satellite-b, since that is the endpoint i am
> > currently testing with.
> >
> > This is my zones.conf on satellite-b:
> > object Endpoint "master" {
> > host = "10.0.0.10"
> > }
> >
> > object Zone "root" {
> > endpoints = [ "master" ]
> > }
> >
> > object Endpoint "satellite-b" {
> > }
> >
> > object Zone "site-b" {
> > endpoints = [ "satellite-b" ]
> > parent = "root"
> > }
> >
> > object Zone "global-config" {
> > global = true
> > }
> >
> > The directory /etc/icinga2/zones.d/site-b contains one single conf file:
> > apply Notification "site-b.host-notify-by-mail" to Host {
> > import "mail-host-notification"
> >
> > command = "mail-host-notification"
> >
> > users = host.vars.notification.mail.users
> > user_groups = host.vars.notification.mail.groups
> >
> > interval = host.vars.notification.interval
> > times.begin = host.vars.notification.delay
> >
> > assign where host.vars.notification.mail && host.zone == "site-b"
> > //ignore where ZoneName != "site-b"
> > }
> >
> > apply Notification "site-b.service-notify-by-mail" to Service {
> > import "mail-service-notification"
> >
> > command = "mail-service-notification"
> >
> > users = host.vars.notification.mail.users
> > user_groups = host.vars.notification.mail.groups
> >
> > interval = service.vars.notification.interval
> > times.begin = service.vars.notification.delay
> >
> > assign where host.vars.notification.mail && host.zone == "site-b"
> > //ignore where ZoneName != "site-b"
> > }
> >
> > object Host "NotificationTest" {
> > import "generic-host"
> > display_name = "NotificationTest"
> > address = "1.2.3.4"
> > check_interval = 30s
> > //command_endpoint = "satellite-b"
> > vars.notification.interval = 30s
> > vars.notification.mail.users = [ "icingaadmin" ]
> > }
> >
> > The templates mail-host-notification, mail-service-notification and
> > generic-host are defined within
> > /etc/icinga2/zones.d/global-config/templates.conf and are the examples
> > provided with Icinga2.
> >
> > I'll keep trying to reproduce the issue and will provide logs as soon as
> > this behaviour occurs again.
> >
> > Regards,
> > Markus
> >
> > -----Original message-----
> > *From:* Michael Friedrich <michael.friedr...@netways.de
> > <mailto:michael.friedr...@netways.de> >
> > *Sent:* Friday 17th April 2015 9:18
> > *To:* icinga-users@lists.icinga.org
> > <mailto:icinga-users@lists.icinga.org>
> > *Subject:* Re: [icinga-users] Restrict notifications to one specific
> > (satellite) endpoint
> >
> > Am 17.04.2015 um 02:03 schrieb Markus Joosten:
> >> My issue isn't easily described in a few config snippets, that's
> >> why i didn't include any so far. But i'll try to describe my
> >> implementation as close as possible.
> >
> > Masses of text are nice, but think about that - nobody sees your
> > shell, your configuration, your logs. I have a hard time imaginating
> > how it looks like, and how the timining points for the notifications
> > happen. Presumingly it's best to get access to your system, but I
> > don't do that, that's something for my employer :)
> >
> > And yes, it's not easy. But it certainly requires _you_ to show _us_
> > every single detail.
> >
> > I'll wait for those parts, maybe you've found a bug and we'll nail
> > it down. Though i think it is a wrong configuration at this time.
> >
> > Kind regards,
> > Michael
> >
> >>
> >> Let's assume we have 3 sites (a, b and c).
> >> Site a is running the parent zone, we call it "root" (and also a
> >> global configuration zone called "global-config"), the endpoint is
> >> called "master".
> >>
> >> Endpoint "satellite-b" is running zone "site-b" whose parent is my
> >> zone "root".
> >>
> >> Endpoint "satellite-c" is running zone "site-c" whose parent is my
> >> zone "root" as well.
> >>
> >> All config is done on my endpoint "master" (in the corresponding
> >> zone directories), which is replicated to my endpoints
> >> "satellite-b" and "satellite-c".
> >>
> >> Each satellite is only monitoring other systems at its local site.
> >> Additionally, i would like all notifications related to systems at
> >> site b to be sent from "satellite b".
> >> Same goes for site c, all notifications related to systems at site
> >> c should be sent from "satellite c",
> >>
> >> I've put all notification apply rules in each zone for each site,
> >> assuming that the notifications will be executed by the endpoint
> >> which is located in that zone.
> >>
> >> This leads me to my main issue, sometimes notifications for a host
> >> at site b or c are being sent out by my _master_ endpoint, but i
> >> would have expected that only "satellite-b" should send
> >> notifications for zone "site-b".
> >>
> >> Strangely enough, it appears very random if a single notification
> >> is being sent from master or the "correct" satellite.
> >>
> >> As of now, i have only found 2 ways of making sure that the master
> >> won't send any unexpected notifications:
> >> - disable notification feature on master (not cool)
> >> - using the following expression in my notification apply rule:
> >> assign where ZoneName == "site-b"
> >>
> >> The second way isn't really nice either, since the master doesn't
> >> know about any notifications happening in the child zones. Also i
> >> regularly see messages like these in my master log:
> >> - checking for notifications for checkable xyz
> >> - checkable xyz does not have any notifications
> >>
> >> So i'm searching for the proper way to make sure that master won't
> >> send notifications for zones "site-b" and "site-c".
> >>
> >> If my descriptions should not suffice, please let me know what
> >> more info i can provide.
> >>
> >> Regards,
> >> Markus
> >>
> >> Sent from my iPhone
> >>
> >> On 16 Apr 2015, at 22:47, Michael Friedrich
> >> <michael.friedr...@netways.de <mailto:michael.friedr...@netways.de>
> >> <mailto:michael.friedr...@netways.de
> >> <mailto:michael.friedr...@netways.de> >> wrote:
> >>
> >>> Reading your mails again and again but I still don't understand
> >>> the issue - it doesn't shed some light into the problem,
> >>> especially not with the assumptions made without any
> >>> configuration snippets (and object list output) and relevant
> >>> (debug) log parts - and I won't do blind guessing.
> >>>
> >>> Von meinem iPad gesendet
> >>>
> >>> Am 16.04.2015 um 15:38 schrieb Markus Joosten
> >>> <markus.joos...@plumbe.de <mailto:markus.joos...@plumbe.de>
> >>> <mailto:markus.joos...@plumbe.de <mailto:markus.joos...@plumbe.de>
> >>> ><mailto:markus.joos...@plumbe.de <mailto:markus.joos...@plumbe.de> >>:
> >>>
> >>> It appears i spoke to soon.
> >>> This behaviour is really weird and sort of erratic.
> >>>
> >>> After a few reloads of Icinga2 on my master and the endpoints, i
> >>> am now receiving notifications from my master again.
> >>> This time, master and satellite seem to take turns in sending the
> >>> notifications (1 from the master, 1 from the satellite, and so on).
> >>>
> >>> I'd really appreciate some feedback regarding this.
> >>>
> >>> Regards,
> >>> Markus
> >>> -----Original message-----
> >>> From: Markus Joosten <markus.joos...@plumbe.de
> >>> <mailto:markus.joos...@plumbe.de>
> >>> <mailto:markus.joos...@plumbe.de <mailto:markus.joos...@plumbe.de>
> >>> ><mailto:markus.joos...@plumbe.de <mailto:markus.joos...@plumbe.de> >>
> >>> Sent: Thursday 16th April 2015 14:29
> >>> To: Icinga User's Corner <icinga-users@lists.icinga.org
> >>> <mailto:icinga-users@lists.icinga.org>
> >>> <mailto:icinga-users@lists.icinga.org
> >>> <mailto:icinga-users@lists.icinga.org>
> >>> ><mailto:icinga-users@lists.icinga.org
> >>> <mailto:icinga-users@lists.icinga.org> >>
> >>> Subject: Re: [icinga-users] Restrict notifications to one
> >>> specific (satellite) endpoint
> >>>
> >>> Hi list,
> >>>
> >>> this is just a follow-up to my previous issue with restricting
> >>> notifications to the proper endpoints since i might have found
> >>> the reason for this behaviour.
> >>>
> >>> A quick recap:
> >>> - i'm running a distributed setup with a global-config zone which
> >>> holds templates, commands and such
> >>> - the cluster master should only monitor all its child endpoints,
> >>> the actual host and service monitoring should be performed by the
> >>> endpoints in their zones
> >>> - when assigning Notification objects to hosts, the cluster
> >>> master would always send *some* of the notifications
> >>>
> >>> The workaround (by using: assign where ZoneName == ...) helped so
> >>> far that the notifications were always sent by the correct
> >>> endpoint, but without the master "knowing" about them.
> >>> (There was no notification object visible on the master using
> >>> "icinga2 object list --type Notification", all GUIs always showed
> >>> 0 notifications for the hosts and services being monitored from
> >>> my satellites.)
> >>>
> >>> Today i disabled the import of the mail-host-notification
> >>> template (which was defined in my global-config zone) and voila,
> >>> notifications were only sent from the correct endpoint.
> >>>
> >>> My theory is that somehow the master thinks he is responsible for
> >>> a notification because its template is defined in the
> >>> global-config zone.
> >>> Could that be the case? Would that be a desired behaviour or a bug?
> >>>
> >>> Looking forward to a reply :)
> >>>
> >>> Regards,
> >>> Markus
> >>>
> >>> -----Original message-----
> >>> From: Markus Joosten <markus.joos...@plumbe.de
> >>> <mailto:markus.joos...@plumbe.de>
> >>> <mailto:markus.joos...@plumbe.de <mailto:markus.joos...@plumbe.de>
> >>> ><mailto:markus.joos...@plumbe.de <mailto:markus.joos...@plumbe.de> >>
> >>> Sent: Monday 2nd February 2015 10:26
> >>> To: Icinga User's Corner <icinga-users@lists.icinga.org
> >>> <mailto:icinga-users@lists.icinga.org>
> >>> <mailto:icinga-users@lists.icinga.org
> >>> <mailto:icinga-users@lists.icinga.org>
> >>> ><mailto:icinga-users@lists.icinga.org
> >>> <mailto:icinga-users@lists.icinga.org> >>
> >>> Subject: Re: [icinga-users] Restrict notifications to one
> >>> specific (satellite) endpoint
> >>>
> >>> Markus,
> >>>
> >>> thanks alot for pointing me in the right direction!
> >>>
> >>> Whilst using host.zone did not change the behaviour, i've been
> >>> able to restrict the notifications to specific endpoint(s) by
> >>> applying the notifications using constants as such:
> >>>
> >>> apply Notification "host-notify-by-mail" to Host {
> >>> [...]
> >>> assign where host.vars.notification.mail && ZoneName ==
> >>> "satellite-zone-2"
> >>> }
> >>>
> >>> (using NodeName instead of ZoneName works as well)
> >>>
> >>> Best regards,
> >>> Markus
> >>>
> >>>
> >>> -----Original message-----
> >>> > From:Markus Frosch <mar...@lazyfrosch.de
> >>> <mailto:mar...@lazyfrosch.de>
> >>> <mailto:mar...@lazyfrosch.de <mailto:mar...@lazyfrosch.de>
> >>> ><mailto:mar...@lazyfrosch.de <mailto:mar...@lazyfrosch.de> >>
> >>> > Sent: Saturday 31st January 2015 17:26
> >>> > To: Icinga User's Corner <icinga-users@lists.icinga.org
> >>> <mailto:icinga-users@lists.icinga.org>
> >>> <mailto:icinga-users@lists.icinga.org
> >>> <mailto:icinga-users@lists.icinga.org>
> >>> ><mailto:icinga-users@lists.icinga.org
> >>> <mailto:icinga-users@lists.icinga.org> >>
> >>> > Subject: Re: [icinga-users] Restrict notifications to one
> >>> specific (satellite) endpoint
> >>> >
> >>> > On 31.01.2015 14:47, Markus Joosten wrote:
> >>> > > thanks alot for your response.
> >>> > > Actually i would like the master _not_ to notify for hosts in
> >>> the satellite zones.
> >>> > > Only the satellite endpoints should generate alerts for the
> >>> hosts in their zone.
> >>> > >
> >>> > > The master should only generate alerts for hosts located in
> >>> the parent zones
> >>> > >
> >>> > > As i've written before, it seems to be impossible to
> >>> accomplish this - the master always sends notifications for the
> >>> child zones as well.
> >>> > >
> >>> > > Unless i disable the notification on the master, which is not
> >>> what i want of course.
> >>> >
> >>> > That should be doable with different notification rules for
> >>> master and
> >>> > satellite, or a const helper.
> >>> >
> >>> > The attribute host.zone is internally set due to the zones loading.
> >>> >
> >>> > apply Notification ... {
> >>> > ...
> >>> > assign where host.zone == "master"
> >>> > //assign where host.zone == ZoneName
> >>> > }
> >>> >
> >>> > Have not tested this!
> >>> >
> >>> > But please try and give us feedback.
> >>> >
> >>> > Cheers from FOSDEM :)
> >>> > Markus
> >>> >
> >>> > --
> >>> > mar...@lazyfrosch.de <mailto:mar...@lazyfrosch.de>
> >>> <mailto:mar...@lazyfrosch.de <mailto:mar...@lazyfrosch.de>
> >>> ><mailto:mar...@lazyfrosch.de <mailto:mar...@lazyfrosch.de> >
> >>> > http://www.lazyfrosch.de <http://www.lazyfrosch.de>
> >>> >
> >>> > _______________________________________________
> >>> > icinga-users mailing list
> >>> > icinga-users@lists.icinga.org
> >>> <mailto:icinga-users@lists.icinga.org>
> >>> <mailto:icinga-users@lists.icinga.org
> >>> <mailto:icinga-users@lists.icinga.org>
> >>> ><mailto:icinga-users@lists.icinga.org
> >>> <mailto:icinga-users@lists.icinga.org> >
> >>> > https://lists.icinga.org/mailman/listinfo/icinga-users
> >>> >
> >>>
> >>> _______________________________________________
> >>>
> >>> icinga-users mailing list
> >>>
> >>> icinga-users@lists.icinga.org <mailto:icinga-users@lists.icinga.org>
> >>> <mailto:icinga-users@lists.icinga.org
> >>> <mailto:icinga-users@lists.icinga.org>
> >>> ><mailto:icinga-users@lists.icinga.org
> >>> <mailto:icinga-users@lists.icinga.org> >
> >>>
> >>> https://lists.icinga.org/mailman/listinfo/icinga-users
> >>>
> >>>
> >>>
> >>> _______________________________________________
> >>>
> >>> icinga-users mailing list
> >>>
> >>> icinga-users@lists.icinga.org <mailto:icinga-users@lists.icinga.org>
> >>> <mailto:icinga-users@lists.icinga.org
> >>> <mailto:icinga-users@lists.icinga.org>
> >>> ><mailto:icinga-users@lists.icinga.org
> >>> <mailto:icinga-users@lists.icinga.org> >
> >>>
> >>> https://lists.icinga.org/mailman/listinfo/icinga-users
> >>>
> >>>
> >>>
> >>> _______________________________________________
> >>> icinga-users mailing list
> >>> icinga-users@lists.icinga.org <mailto:icinga-users@lists.icinga.org>
> >>> <mailto:icinga-users@lists.icinga.org
> >>> <mailto:icinga-users@lists.icinga.org>
> >>> ><mailto:icinga-users@lists.icinga.org
> >>> <mailto:icinga-users@lists.icinga.org> >
> >>> https://lists.icinga.org/mailman/listinfo/icinga-users
> >>>
> >>> --
> >>> Michael Friedrich, DI (FH)
> >>> Application Developer
> >>>
> >>> NETWAYS GmbH | Deutschherrnstr. 15-19 | D-90429 Nuernberg
> >>> Tel: +49 911 92885-0 | Fax: +49 911 92885-77
> >>> GF: Julian Hein, Bernd Erk | AG Nuernberg HRB18461
> >>> http://www.netways.de <http://www.netways.de> |
> >>> michael.friedr...@netways.de <mailto:michael.friedr...@netways.de>
> >>> <mailto:michael.friedr...@netways.de
> >>> <mailto:michael.friedr...@netways.de> >
> >>>
> >>> ** OSDC 2015 - April - osdc.de <http://osdc.de <http://osdc.de> > **
> >>> ** Puppet Camp Berlin 2015 - April - netways.de/puppetcamp
> >>> <" target="_blank">http://netways.de/puppetcamp>
> >>> <http://netways.de/puppetcamp> ; **
> >>> ** OSBConf 2015 - September - osbconf.org <http://osbconf.org
> >>> <http://osbconf.org> > **
> >>> _______________________________________________
> >>> icinga-users mailing list
> >>> icinga-users@lists.icinga.org <mailto:icinga-users@lists.icinga.org>
> >>> <mailto:icinga-users@lists.icinga.org
> >>> <mailto:icinga-users@lists.icinga.org> >
> >>> https://lists.icinga.org/mailman/listinfo/icinga-users
> >>>
> >>
> >>
> >> _______________________________________________
> >> icinga-users mailing list
> >> icinga-users@lists.icinga.org <mailto:icinga-users@lists.icinga.org>
> >> https://lists.icinga.org/mailman/listinfo/icinga-users
> >
> >
> > --
> > Michael Friedrich, DI (FH)
> > Application Developer
> >
> > NETWAYS GmbH | Deutschherrnstr. 15-19 | D-90429 Nuernberg
> > Tel: +49 911 92885-0 | Fax: +49 911 92885-77
> > GF: Julian Hein, Bernd Erk | AG Nuernberg HRB18461
> > http://www.netways.de <http://www.netways.de> |
> > michael.friedr...@netways.de <mailto:michael.friedr...@netways.de>
> >
> > ** OSDC 2015 - April - osdc.de **
> > ** Puppet Camp Berlin 2015 - April - netways.de/puppetcamp **
> > ** OSBConf 2015 - September - osbconf.org **
> >
> > _______________________________________________
> >
> > icinga-users mailing list
> >
> > icinga-users@lists.icinga.org <mailto:icinga-users@lists.icinga.org>
> >
> > https://lists.icinga.org/mailman/listinfo/icinga-users
> >
> >
> >
> > _______________________________________________
> > icinga-users mailing list
> > icinga-users@lists.icinga.org <mailto:icinga-users@lists.icinga.org>
> > https://lists.icinga.org/mailman/listinfo/icinga-users
> >
>
>
> --
> Michael Friedrich, DI (FH)
> Application Developer
>
> NETWAYS GmbH | Deutschherrnstr. 15-19 | D-90429 Nuernberg
> Tel: +49 911 92885-0 | Fax: +49 911 92885-77
> GF: Julian Hein, Bernd Erk | AG Nuernberg HRB18461
> http://www.netways.de <http://www.netways.de> | michael.friedr...@netways.de
> <mailto:michael.friedr...@netways.de>
>
> ** OSDC 2015 - April - osdc.de **
> ** Puppet Camp Berlin 2015 - April - netways.de/puppetcamp **
> ** OSBConf 2015 - September - osbconf.org **
> _______________________________________________
> icinga-users mailing list
> icinga-users@lists.icinga.org <mailto:icinga-users@lists.icinga.org>
> https://lists.icinga.org/mailman/listinfo/icinga-users
>
_______________________________________________
icinga-users mailing list
icinga-users@lists.icinga.org
https://lists.icinga.org/mailman/listinfo/icinga-users
