I meant to include this as well: [image: image.png]
On Fri, Jul 17, 2020 at 7:59 AM Munroe Sollog <m...@lehigh.edu> wrote: > While they may not me NVMe, I currently have 6 ssds in a raid 10 > configuration. > > On Fri, Jul 17, 2020 at 3:06 AM Marsh-Mossadeghi, Dariush < > dariush.marsh-mossade...@lv.com> wrote: > >> Hi, >> >> >> >> I’d guess your disks can’t keep up. >> >> As a rule of thumb, if you’re using any timeseries database and you’re >> not using solid state storage in 2020 you’re gonna have a bad time ;-) >> >> >> >> I’ve had ntopng monitoring a busy 10G pipe with upwards of 8k active >> hosts on the ‘local’ network and running comfortably at a load average of >> no more than 3-4 on a 16 core Xeon, using up less than 100G of memory. But >> it does need some serious disk IO. >> >> >> >> Use iostat to check what %iowait looks like, it’s the percentage of the >> time the CPU’s are waiting around for disk IO to complete. If that’s >> anything more than 1-2% your bottleneck is most likely disk IO. >> >> >> >> HTH >> >> Dariush >> >> >> >> *From:* ntop-boun...@listgateway.unipi.it [mailto: >> ntop-boun...@listgateway.unipi.it] *On Behalf Of *Munroe Sollog >> *Sent:* 16 July 2020 19:21 >> *To:* ntop@listgateway.unipi.it >> *Subject:* [Ntop] Performance pains >> >> >> >> ATTENTION: External email - This message has been sent from outside LV=. >> >> looking in the ntopng log I am getting a lot of: >> >> >> >> [ts_5min_dump_utils.lua:461] ERROR: [Inside_VPN]Cannot complete local >> hosts timeseries dump. Disk slow or too many local hosts? >> >> ERROR: [Outside]Cannot complete local hosts timeseries dump. Disk slow or >> too many local hosts? >> >> >> >> >> I'm assuming the host can't keep up. Our local network is defined as a >> /16 and it looks like we received about 50-60k flows per minute. For >> reference my cpu: >> >> >> >> CPU(s): 56 >> >> On-line CPU(s) list: 0-55 >> >> Thread(s) per core: 2 >> >> Core(s) per socket: 14 >> >> Socket(s): 2 >> >> NUMA node(s): 2 >> >> Vendor ID: GenuineIntel >> >> CPU family: 6 >> >> Model: 63 >> >> Model name: Intel(R) Xeon(R) CPU E5-2697 v3 @ 2.60GHz >> >> >> >> # free -h >> >> total used free shared buff/cache >> available >> >> Mem: 251Gi 186Gi 45Gi 209Mi 20Gi >> 161Gi >> >> Swap: 29Gi 347Mi 29Gi >> >> >> >> >> >> >> >> -- >> >> Munroe Sollog >> >> Senior Network Engineer >> >> mun...@lehigh.edu >> >> This email (including any attachment) may contain confidential and/ or >> legally privileged information. If you are not the intended recipient, >> please notify us at d...@lv.com and destroy it and any copies. >> Unauthorised access, use, disclosure, storage or copying of this email is >> not permitted and, unless you are the intended recipient, you are not >> entitled to rely on it in any way. Any opinions expressed in this email are >> those of the individual sending it and not necessarily those of LV=. >> >> This email is believed to be free of any virus or other defect. However, >> communication by email cannot be guaranteed to be free from defect, error >> free or secure. If you choose to communicate with us by email you must >> realise that there can be no guarantee of privacy and you should carry out >> your own security checks before opening any email or attachment. >> >> LV= accepts no liability for any loss or damage which may be caused by >> any lack of privacy, software viruses or other defect. >> >> LV= reserves the right to monitor and inspect any email (including any >> attachment) sent to and/or from LV= for reasons of security and for >> monitoring internal compliance with our office policies. LV= may use email >> monitoring or blocking software at its discretion. You are responsible for >> ensuring that any email you send is appropriate and within the bounds of >> the law. >> >> LV= and Liverpool Victoria are trademarks of Liverpool Victoria Financial >> Services Limited and LV= and Liverpool Victoria are trading styles of the >> Liverpool Victoria group of companies. Liverpool Victoria Financial >> Services Limited , registered in England with registration number 12383237 >> is authorised by the Prudential Regulation Authority and regulated by the >> Financial Conduct Authority and the Prudential Regulation Authority, >> register number 110035.The registered office address for all LV= companies >> is County Gates, Bournemouth, BH1 2NF. Information about the LV= group of >> companies can be found via this link www.lv.com/legal/lvcompanies >> _______________________________________________ >> Ntop mailing list >> Ntop@listgateway.unipi.it >> http://listgateway.unipi.it/mailman/listinfo/ntop > > > > -- > Munroe Sollog > Senior Network Engineer > mun...@lehigh.edu > -- Munroe Sollog Senior Network Engineer mun...@lehigh.edu
_______________________________________________ Ntop mailing list Ntop@listgateway.unipi.it http://listgateway.unipi.it/mailman/listinfo/ntop
