Hello, I hope this is a good list to post this on, I have a feeling the solution is somewhere obscure in the networking layer.
I've set up an IPS system, using: * FreeBSD 10.1 (guest OS, plenty of RAM/CPU) * ESXi 5.5 (host OS, using Intel X520 10Gb cards. Not overloaded, all graphs show it's got plenty of RAM/CPU spare at all times) * vmxnet3 drivers * ipfw (very small ruleset, basically just a divert rule) * suricata, in ipfw divert mode I'm having a couple of major issues. The first is that every so often, even with relatively little traffic, the load on the box suddenly spikes and pings to a neighbouring router (via the divert rule) go from <1ms to >300ms. Generally this resolves itself after a few minutes, although last night it went on for an hour until I restarted ipfw and suricata. The second is that if I do a large download, eg a FreeBSD ISO, the download usually hangs somewhere between 5MB and 100MB through. I can see traffic trying to get through on neighbouring routers, it's just the interface with the divert to suricata where they disappear into a black hole. The connection speed is around 50Mb, btw. Now it's possible it's suricata being weird, but there's nothing untoward in its events and stats logs, and if I replay the traffic from a pcap file then suricata processes everything fine (a pcap taken over a 90s period during a slowdown is processed in under a second). So my guess is that if suricata takes slightly longer than normal to process a packet, something in the networking or ipfw divert system is tripping itself up. Maybe a queue is filling up? I've set net.inet.ip.fw.dyn_buckets=16384, and done an ipfw flush, but net.inet.ip.fw.curr_dyn_buckets is stubbornly sticking at 256: have I done something wrong? Other tunables I've set are: kern.random.sys.harvest.ethernet=0 kern.random.sys.harvest.point_to_point=0 kern.random.sys.harvest.interrupt=0 kern.ipc.soacceptqueue=1024 Can anyone suggest either tests to see what might be going wrong, or tunables to help things run smoother? Both myself and a colleague have used FreeBSD for over 15 years, and never quite seen anything like it. Many thanks, Oliver. _______________________________________________ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
