Re: SNMP - monitoring large number of devices

2015-09-30 Thread Anton Berezin
Hello, On Tue, Sep 29, 2015 at 10:20:19PM +0200, Pavel Dimow wrote: > recently I have been tasked with a NMS project. The idea is to pool about > 20 OID's from 50k cable modems in less then 5 minutes (yes, I know it's a > one million OID's). Before you say check out some very professional and > ex

Re: SNMP - monitoring large number of devices

2015-09-30 Thread Ray Ludendorff
Please try the following/ https://statseeker.com/ Sent from my virtual office On Sep 30, 2015, at 09:44, Pavel Dimow mailto:paveldi...@gmail.com>> wrote: Thank you all for you suggestions, I knew it that NANOG is a perfect place for those kind of questions. I will discuss your comments with

Re: SNMP - monitoring large number of devices

2015-09-30 Thread Colton Conor
Check out Science Logic. Their platform is made to scale to these levels. On Wed, Sep 30, 2015 at 2:43 AM, Pavel Dimow wrote: > Thank you all for you suggestions, I knew it that NANOG is a perfect place > for those kind of questions. > I will discuss your comments with my colleagues to see what

Re: SNMP - monitoring large number of devices

2015-09-30 Thread Pavel Dimow
Thank you all for you suggestions, I knew it that NANOG is a perfect place for those kind of questions. I will discuss your comments with my colleagues to see what would be the best solution. Once again thank you all for your valuable suggestions, I hope I will update you soon with some results/tes

Re: SNMP - monitoring large number of devices

2015-09-29 Thread Tom Sands
We have used ZenOss for a number of years at this scale (40k+ devices, at intervals of 1-5 minutes). It is possible to do if you have the hardware and storage performance to throw at it. We used OpenNMS before that and had to change due to scale. During that time we evaluated a number of the big

RE: SNMP - monitoring large number of devices

2015-09-29 Thread Joel Whitcomb
So we have used www.zenoss.org for many years. Individual collectors are easily handling snmp poll rates of 1.5k oids per second(450k per 5m). As zenoss core is open source Its probably worth a look for you. -Joel -Original Message- From: NANOG [mailto:nanog-bounces+joel.whitcomb=citrix

Re: SNMP - monitoring large number of devices

2015-09-29 Thread Robert Drake
OpenNMS has a poller that will do what you want. The problem is figuring out what you wish to collect and how to use it. Most of the time it's not as simple as pointing at the modem and saying go. I've added a few oids for some of the modems we support, just so I can get SNR on them. I don'

RE: SNMP - monitoring large number of devices

2015-09-29 Thread Jameson, Daniel
- monitoring large number of devices I'm able to poll a few thousand CMs in a few seconds using perl's Net::SNMP and async calls. 50k seems pretty doable. --Blake Pavel Dimow wrote on 9/29/2015 3:20 PM: > Hi all, > > recently I have been tasked with a NMS project. The idea is to

Re: SNMP - monitoring large number of devices

2015-09-29 Thread Blake Hudson
I'm able to poll a few thousand CMs in a few seconds using perl's Net::SNMP and async calls. 50k seems pretty doable. --Blake Pavel Dimow wrote on 9/29/2015 3:20 PM: Hi all, recently I have been tasked with a NMS project. The idea is to pool about 20 OID's from 50k cable modems in less then 5

RE: SNMP - monitoring large number of devices

2015-09-29 Thread Nick Ellermann
Pavel, It's all going to be how you deploy a selected polling system. Most server operating systems are going to struggle with that many transactions in a short period of time no matter the awesomeness of the polling engine. Look for a distributed polling solution. If you can spread the connect

Re: SNMP - monitoring large number of devices

2015-09-29 Thread Dale W. Carder
Thus spake Dan White (dwh...@olp.net) on Tue, Sep 29, 2015 at 03:37:51PM -0500: > On 09/29/15 22:20 +0200, Pavel Dimow wrote: > >recently I have been tasked with a NMS project. The idea is to pool about > >20 OID's from 50k cable modems in less then 5 minutes (yes, I know it's a > >one million OID'

Re: SNMP - monitoring large number of devices

2015-09-29 Thread William Herrin
On Tue, Sep 29, 2015 at 4:20 PM, Pavel Dimow wrote: > recently I have been tasked with a NMS project. The idea is to pool about > 20 OID's from 50k cable modems in less then 5 minutes (yes, I know it's a > one million OID's). [...] You always first perform snmp > walk in order to discover interfac

Re: SNMP - monitoring large number of devices

2015-09-29 Thread Dan White
On 09/29/15 22:20 +0200, Pavel Dimow wrote: recently I have been tasked with a NMS project. The idea is to pool about 20 OID's from 50k cable modems in less then 5 minutes (yes, I know it's a one million OID's). Before you say check out some very professional and expensive solutions I would like

RE: SNMP - monitoring large number of devices

2015-09-29 Thread Jameson, Daniel
Zabbix is probably one of the more robust snmp platform, it's database backed by either postgres, mysql or oracle and scales pretty big. If this is more than a one-time event, you'll need some real horsepower and HDD space to keep all that data. It might be worth writing a custom ruby/pytho

Re: SNMP - monitoring large number of devices

2015-09-29 Thread Jared Mauch
We built our own system for this purpose and just spawn one process per device being polled. This seems to work out OK and many cores can make this work out. You can also just split the workload horizontally across multiple servers. The challenges are as usual how to report from a dataset like

Re: SNMP - monitoring large number of devices

2015-09-29 Thread Scott Helms
Pavel, AFAIK there are no frameworks that solve, or even come close to solving, this problem. The first thing to learn is how to do asynchronous SNMP calls that will let you send out requests without having to wait for the response. How you do those will vary by the language that you're using. N