ntopng 2.3.160204 & nprobe 7.3.160204 on Debian 8.3
8CPU / 16GB VM
Using netflow from Cisco 6509 w/nprobe and ntop as collector on network
w/around 150K hosts. nprobe is started as such:
nprobe -G --zmq "tcp://*:5556" -i none -n none --collector-port 2055 -V
9
Ntop using ntopng.conf w/following parameters:
-G=/var/tmp/ntopng.pid
-i=tcp://127.0.0.1:5556
-S=all
-m=[NETWORKS]
-X=550000
-x=550000
-F="mysql;localhost;ntopng;flows;user;pass"
When using MySQL the webui is extremely slow and I see constant writes
to disk, mostly inserts to flowsv4 table in ntopng db. I also am not
seeing accurate info in dashboard graphs & reports however I do see
accurate host information and historical data for ntop interface. I have
taken steps to expand various innodb configurations like buffer pool
size, log buffer size, innodb read/write io threads and there is no
difference with performance. Using various tools to view perf data for
i/o I am seeing contant 2+MB/sec - 10MB/sec disk writes and very high
CPU wait percentages. My VM infrastructure consists of 4 IBM M3 ESXi
hosts and a gen 2 XIV SAN so I'm pretty confident it's not the hardware.
When I configure ntopng.conf to not use MySQL backend (everything
written to /var/tmp/ntopng) the UI is much more responsive and the
dashboard is accurate however, historical data for both hosts (traffic)
and the ntop interface is inaccurate. I can have ntop running for a week
and see ~10TB of data total for a given network but will not have info
for hosts when I sort on traffic totals. When I select hosts the default
view may show a host that has gigs and gigs more total traffic than any
other host but when I sort on traffic (descending) that host is not
represented in the list. My goal was to use SQL backend to retain that
historical data which it does but at a huge performance cost.
I've attached a report screencap that shows on the left a typical day
without using MySQL highlighted in yellow. Today I started the day
configured the same (yellow highlight) but switched to MySQL backend
(red) and then back and forth after various tweaks. Historical info is
there when I view ntop interface and traffic for given time frame. I
also see all hosts traffic totals represented seemingly accurately.
Not sure where to look next so any suggestions are appreciated.
_______________________________________________
Ntop mailing list
[email protected]
http://listgateway.unipi.it/mailman/listinfo/ntop
