Hi, IOT device my be is a wrong word. Sorry.
Well, the machines are Hypriot Linux based Raspberry Pies. RPI2 & RPI3. We want to measure all the basic things like disk, CPU, Mem, Network, NTP, docker containers, etc plus other things. There are a total of 10 checks currently after all the reduction. On Sat, Jan 14, 2017 at 4:30 PM, <icinga-users-requ...@lists.icinga.org> wrote: > Send icinga-users mailing list submissions to > icinga-users@lists.icinga.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.icinga.org/mailman/listinfo/icinga-users > or, via email, send a message with subject or body 'help' to > icinga-users-requ...@lists.icinga.org > > You can reach the person managing the list at > icinga-users-ow...@lists.icinga.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of icinga-users digest..." > > > Today's Topics: > > 1. Re: icinga-users Digest, Vol 37, Issue 9 (Vikas Tiwari) > 2. Re: icinga-users Digest, Vol 37, Issue 9 (Michael Friedrich) > 3. Re: icinga-users Digest, Vol 37, Issue 9 (Antony Stone) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 13 Jan 2017 19:56:46 +0530 > From: Vikas Tiwari <vikask...@gmail.com> > To: icinga-users@lists.icinga.org > Subject: Re: [icinga-users] icinga-users Digest, Vol 37, Issue 9 > Message-ID: > <CAEVQhqpTJ2AgMT7YHb=Po2O+C8+Y1HGLy0kq5TAnSrWhVOph2g@mail. > gmail.com> > Content-Type: text/plain; charset="utf-8" > > Hi Antony, > Thank you for taking time to reply. > > We are monitoring IOT devices. These devices are in remote locations and > behind one-way NAT. We do not have the luxury of unlimited network and > network at times can go bad. > > *Why passive?* > *Because I want the replay logs. Many times the connection b/w master and > client breaks and in such scenarios we want the client to keep running the > checks and send the check output when the connection re-establishes.* > > > *Is the Master able to connect to the Clients through the NAT connection? > The fact that you said above that "the clients (behind NAT) connect to the > the master directly" suggests that it cannot (ie: it sounds like one-way > NAT), so* > > *have you made sure that the Master Endpoint definitions > in /etc/icinga2/zones.conf on the Clients contains the hostname (or IP > address) of the Master, but that the Client Endpoint definitions on the > Master does notcontain the hostname or IP address of the Clients?* > *No, I dont want the master to connect to client. Yes its a one-way NAT. > Yes i have made sure that the master endpoint definitions in zones.conf of > the client contain resolvable hostname and client endpoint definitions on > the master do not contain the hostname.* > > *What bandwidth do you have available per client connection?* > *I do not want 1 MB per hour. This will cost us 720 MB just for monitoring > the IOT device. I want the data to be as much less as possible. * > * Please note that i have already optimised all the checks to output as > much data as possible.* > > *That equals around 256 bits per second (if KB means kilobits) or 2kbits > per second (if KB means kilobytes).* > *That is Kilobytes.* > > *Is that *really* a significant overhead on your network?* > *I am very happy to deploy Icinga2 client on an AWS machine and there we > don't care about the data usage at all. But for an IOT device, is it the > right tool ? Given i have already given you all the information* > > *Changing the heartbeat interval from 10s to 100s and log::SetLogPosition > interval from 5s to 60s will reduce a significant overhead. But that is > hard coded. Can we have it as a configurable entity ?* > > Good day ! > Vikas > > On Fri, Jan 13, 2017 at 4:30 PM, <icinga-users-requ...@lists.icinga.org> > wrote: > > > Send icinga-users mailing list submissions to > > icinga-users@lists.icinga.org > > > > To subscribe or unsubscribe via the World Wide Web, visit > > https://lists.icinga.org/mailman/listinfo/icinga-users > > or, via email, send a message with subject or body 'help' to > > icinga-users-requ...@lists.icinga.org > > > > You can reach the person managing the list at > > icinga-users-ow...@lists.icinga.org > > > > When replying, please edit your Subject line so it is more specific > > than "Re: Contents of icinga-users digest..." > > > > > > Today's Topics: > > > > 1. Icinga2 reduce network data usage (Vikas Tiwari) > > 2. Re: Icinga2 reduce network data usage (Antony Stone) > > > > > > ---------------------------------------------------------------------- > > > > Message: 1 > > Date: Thu, 12 Jan 2017 18:43:48 +0530 > > From: Vikas Tiwari <vikask...@gmail.com> > > To: icinga-users@lists.icinga.org > > Subject: [icinga-users] Icinga2 reduce network data usage > > Message-ID: > > <CAEVQhqrWHmYCxsXGpTVP+fRiOucKsrHFJD3P2pOd4RSsE40weg@ > > mail.gmail.com> > > Content-Type: text/plain; charset="utf-8" > > > > Hi, > > > > I am testing Icinga2 as a monitoring tool for our infrastructure which > will > > monitor close to 3000 nodes. > > > > I have a top down config sync. There is a single master and and there are > > no fail overs(for now). The clients(behind NAT) connect to the master > > directly. > > > > I have configured around 20 passive checks on each client. What i have > > observed is that the heartbeats from both the ends i.e the master and the > > clients are very aggressive which ping every 10 seconds. > > > > Further the JSONRPC pings are as frequent as 5 seconds, there is also a > lot > > of other data exchanged between the client and the master. This all leads > > to high consumption of network data, Given we have limited data on the > > client machines we cannot afford to spend this much data just for > > monitoring. > > > > Currently, per hour 800-900 KB is consumed.Out of this only around 300KB > > seems to be the check output. Check intervals are 6 minutes. > > > > The hearbeat and other intervals are hard coded in the source code itself > > which will be my last option. > > > > Is there any other way i can reduce the network data consumption. ? > > > > Vikas Tiwari > > -------------- next part -------------- > > An HTML attachment was scrubbed... > > URL: <http://lists.icinga.org/pipermail/icinga-users/ > > attachments/20170112/14ecfd34/attachment-0001.html> > > > > ------------------------------ > > > > Message: 2 > > Date: Thu, 12 Jan 2017 14:58:41 +0100 > > From: Antony Stone <antony.st...@icinga.open.source.it> > > To: "Icinga User's Corner" <icinga-users@lists.icinga.org> > > Subject: Re: [icinga-users] Icinga2 reduce network data usage > > Message-ID: <201701121458.42028.antony.st...@icinga.open.source.it> > > Content-Type: Text/Plain; charset="iso-8859-15" > > > > On Thursday 12 January 2017 at 14:13:48, Vikas Tiwari wrote: > > > > > Hi, > > > > > > I am testing Icinga2 as a monitoring tool for our infrastructure which > > will > > > monitor close to 3000 nodes. > > > > > > I have a top down config sync. There is a single master and and there > are > > > no fail overs(for now). The clients(behind NAT) connect to the master > > > directly. > > > > > > I have configured around 20 passive checks on each client. > > > > Why passive? > > > > > What i have observed is that the heartbeats from both the ends i.e the > > > master and the clients are very aggressive which ping every 10 seconds. > > > > Is the Master able to connect to the Clients through the NAT connection? > > The > > fact that you said above that "the clients (behind NAT) connect to the > the > > master directly" suggests that it cannot (ie: it sounds like one-way > NAT), > > so > > have you made sure that the Master Endpoint definitions in > > /etc/icinga2/zones.conf on the Clients contains the hostname (or IP > > address) > > of the Master, but that the Client Endpoint definitions on the Master > does > > not > > contain the hostname or IP address of the Clients? > > > > > Further the JSONRPC pings are as frequent as 5 seconds, there is also a > > lot > > > of other data exchanged between the client and the master. This all > leads > > > to high consumption of network data, Given we have limited data on the > > > client machines we cannot afford to spend this much data just for > > > monitoring. > > > > What bandwidth do you have available per client connection? > > > > > Currently, per hour 800-900 KB is consumed. > > > > That equals around 256 bits per second (if KB means kilobits) or 2kbits > per > > second (if KB means kilobytes). > > > > Is that *really* a significant overhead on your network? > > > > > > Antony. > > > > -- > > I want to build a machine that will be proud of me. > > > > - Danny Hillis, creator of The Connection Machine > > > > Please reply to the > > list; > > please *don't* > CC > > me. > > > > > > ------------------------------ > > > > Subject: Digest Footer > > > > _______________________________________________ > > icinga-users mailing list > > icinga-users@lists.icinga.org > > https://lists.icinga.org/mailman/listinfo/icinga-users > > > > > > ------------------------------ > > > > End of icinga-users Digest, Vol 37, Issue 9 > > ******************************************* > > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: <http://lists.icinga.org/pipermail/icinga-users/ > attachments/20170113/2b22f60a/attachment-0001.html> > > ------------------------------ > > Message: 2 > Date: Fri, 13 Jan 2017 15:36:51 +0100 > From: Michael Friedrich <michael.friedr...@icinga.com> > To: Icinga User's Corner <icinga-users@lists.icinga.org> > Subject: Re: [icinga-users] icinga-users Digest, Vol 37, Issue 9 > Message-ID: <e3e120be-6bcf-4ae0-a2d1-42e0eeaa7...@icinga.com> > Content-Type: text/plain; charset=us-ascii > > Hi, > > > On 13 Jan 2017, at 15:26, Vikas Tiwari <vikask...@gmail.com> wrote: > > > > I am very happy to deploy Icinga2 client on an AWS machine and there we > don't care about the data usage at all. But for an IOT device, is it the > right tool ? > > What kind of IoT device? > > Kind regards, > Michael > > > > > ------------------------------ > > Message: 3 > Date: Fri, 13 Jan 2017 17:25:21 +0100 > From: Antony Stone <antony.st...@icinga.open.source.it> > To: "Icinga User's Corner" <icinga-users@lists.icinga.org> > Subject: Re: [icinga-users] icinga-users Digest, Vol 37, Issue 9 > Message-ID: <201701131725.21584.antony.st...@icinga.open.source.it> > Content-Type: Text/Plain; charset="iso-8859-15" > > On Friday 13 January 2017 at 15:26:46, Vikas Tiwari wrote: > > > Hi Antony, > > Thank you for taking time to reply. > > > > We are monitoring IOT devices. These devices are in remote locations and > > behind one-way NAT. We do not have the luxury of unlimited network and > > network at times can go bad. > > You call them "IoT devices" - how much are they doing, and how complex is > the > monitoring you need to perform on them? > > I assume you're not getting a whole set of data such as disk usage, number > of > processes, memory usage, network bandwidth... which would be common with > full > network client/server type devices. > > So, what are you trying to measure about these things? > > > > Why passive? > > > > Because I want the replay logs. Many times the connection b/w master and > > client breaks and in such scenarios we want the client to keep running > the > > checks and send the check output when the connection re-establishes. > > Okay, that sounds reasonable. > > > > What bandwidth do you have available per client connection? > > > > I do not want 1 MB per hour. This will cost us 720 MB just for monitoring > > the IOT device. I want the data to be as much less as possible. > > Please note that i have already optimised all the checks to output as > > much data as possible. > > Er, if you really mean what you just said, that could be your problem? > > Surely to reduce bandwidth you want to minimise the data output from the > checks? > > How many checks are you performing on each device? > > > > That equals around 256 bits per second (if KB means kilobits) or 2kbits > > > per second (if KB means kilobytes). > > > > That is Kilobytes. > > So, 2kbits / second per device is excessive... hm, that's quite a tough > challenge... > > > for an IOT device, is it the right tool ? > > Difficult to say - what do *you* mean by "IoT", and what are you trying to > measure about these things? > > > Antony. > > -- > I lay awake all night wondering where the sun went, and then it dawned on > me. > > Please reply to the > list; > please *don't* CC > me. > > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > icinga-users mailing list > icinga-users@lists.icinga.org > https://lists.icinga.org/mailman/listinfo/icinga-users > > > ------------------------------ > > End of icinga-users Digest, Vol 37, Issue 10 > ******************************************** >
_______________________________________________ icinga-users mailing list icinga-users@lists.icinga.org https://lists.icinga.org/mailman/listinfo/icinga-users
