Thank you for the reply. Haven't revisited this until now.
Do you have steps or info on configuring nprobe/ntop in such a manner?
Also, we are an educational organization and received a single license
for free. Can this same license be used for multiple instances or would
we have to obtain licensing individually for this tiered setup? Does it
matter if I use multiple vm's and the single license etc?
I am also considering breaking-out the netflow if possible by
site/vlan-group with multiple collectors and a central ntopng instance.
Is that something that would allow me to only need a single license?
Thanks!
On 2016-03-11 19:00, Simone Mainardi wrote:
The traffic charts enclosed show peaks at around .5 Gbps. I don't know
how many flows you are generating but I think they are too much for a
single ntopng and a single MySQL instance. MySQL tuning helps but not
to the necessary extent. Also, we (as ntopng devs) may improve
performances for example by batching insertions and thus avoiding a
single INSERT INTO for each flow. This is something we will address in
the future. However, I am not sure it be enough to handle the volume
of flows you have using a single ntopng and a single database.
Also, I am not sure your data is inaccurate. You get only active hosts
from the host pane so, if an host has transmitted TBs of data but is
now inactive, it won't show up in the list. Use the report if you want
to see historical host stats
Simone
On Wed, Mar 9, 2016 at 2:54 AM, <[email protected]> wrote:
ntopng 2.3.160204 & nprobe 7.3.160204 on Debian 8.3
8CPU / 16GB VM
Using netflow from Cisco 6509 w/nprobe and ntop as collector on
network w/around 150K hosts. nprobe is started as such:
nprobe -G --zmq "tcp://*:5556" -i none -n none --collector-port 2055
-V 9
Ntop using ntopng.conf w/following parameters:
-G=/var/tmp/ntopng.pid
-i=tcp://127.0.0.1:5556 [1]
-S=all
-m=[NETWORKS]
-X=550000
-x=550000
-F="mysql;localhost;ntopng;flows;user;pass"
When using MySQL the webui is extremely slow and I see constant
writes to disk, mostly inserts to flowsv4 table in ntopng db. I also
am not seeing accurate info in dashboard graphs & reports however I
do see accurate host information and historical data for ntop
interface. I have taken steps to expand various innodb
configurations like buffer pool size, log buffer size, innodb
read/write io threads and there is no difference with performance.
Using various tools to view perf data for i/o I am seeing contant
2+MB/sec - 10MB/sec disk writes and very high CPU wait percentages.
My VM infrastructure consists of 4 IBM M3 ESXi hosts and a gen 2 XIV
SAN so I'm pretty confident it's not the hardware.
When I configure ntopng.conf to not use MySQL backend (everything
written to /var/tmp/ntopng) the UI is much more responsive and the
dashboard is accurate however, historical data for both hosts
(traffic) and the ntop interface is inaccurate. I can have ntop
running for a week and see ~10TB of data total for a given network
but will not have info for hosts when I sort on traffic totals. When
I select hosts the default view may show a host that has gigs and
gigs more total traffic than any other host but when I sort on
traffic (descending) that host is not represented in the list. My
goal was to use SQL backend to retain that historical data which it
does but at a huge performance cost.
I've attached a report screencap that shows on the left a typical
day without using MySQL highlighted in yellow. Today I started the
day configured the same (yellow highlight) but switched to MySQL
backend (red) and then back and forth after various tweaks.
Historical info is there when I view ntop interface and traffic for
given time frame. I also see all hosts traffic totals represented
seemingly accurately.
Not sure where to look next so any suggestions are appreciated.
_______________________________________________
Ntop mailing list
[email protected]
http://listgateway.unipi.it/mailman/listinfo/ntop
Links:
------
[1] http://127.0.0.1:5556
_______________________________________________
Ntop mailing list
[email protected]
http://listgateway.unipi.it/mailman/listinfo/ntop
_______________________________________________
Ntop mailing list
[email protected]
http://listgateway.unipi.it/mailman/listinfo/ntop
