Hello Ryan, 2016-04-27 17:28 GMT-03:00 Ryan Stone <ryst...@gmail.com>:
> From a quick look at the vlan code, I can identify a few cases that might > cause that counter to increment: > > 1) Error from the underlying ixgbe device. Does "netstat -dI ix0" show > that the driver has been dropping packets? > No, it does not increase drop counters on ix port, only on the vlan device. > > 2) Link down events on the underlying NIC. I believe that link flaps will > be logged to /var/log/messages and dmesg; do you see anything there that > might correspond to the time of the packet drops? > No, dmesg is clean, only a couple down/up link when I actually did disconnect the port, and no other message on /var/log/messages that grabs my attention. > > 3) If VLAN_HWTAGGING is disabled through ifconfig on the port, then in > theory a low memory event could cause the packet to be dropped. Does > "netstat -m" show that "requests for mbufs denied" increasing? > Here is the ifconfig -v output for the vlan6 on the 10.1-STABLE system vlan6: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=303<RXCSUM,TXCSUM,TSO4,TSO6> ether a0:36:9f:2a:6d:ae inet6 fe80::a236:9fff:fe2a:6dae%vlan6 prefixlen 64 scopeid 0x19 inet6 2804:1054:bad:b1fe::1 prefixlen 64 nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> media: Ethernet autoselect (10Gbase-SR <full-duplex>) status: active vlan: 3005 parent interface: ix3 groups: vlan And here it is on the 10.3-STABLE system, I dont know why the only difference is no options were printed on the newer system, everything else is the same. vlan6: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 ether a0:36:9f:2a:6d:ae inet6 fe80::a236:9fff:fe2a:6dae%vlan6 prefixlen 64 scopeid 0x19 inet6 2804:1054:bad:b1fe::1 prefixlen 64 nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> media: Ethernet autoselect (10Gbase-SR <full-duplex>) status: active vlan: 3005 parent interface: ix3 groups: vlan This is the netstat -m output when system has packet loss. Denied and delayed counters are zeroed. % netstat -m 12365/21040/33405 mbufs in use (current/cache/total) 12310/14530/26840/505076 mbuf clusters in use (current/cache/total/max) 12310/14508 mbuf+clusters out of packet secondary zone in use (current/cache) 0/225/225/252538 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/74826 9k jumbo clusters in use (current/cache/total/max) 0/0/0/42089 16k jumbo clusters in use (current/cache/total/max) 27711K/35220K/62931K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile > > On Wed, Apr 27, 2016 at 2:41 PM, Zé Claudio Pastore <zclau...@bsd.com.br> > wrote: > >> Hello, >> >> On a BGP border router I help manage, we run FreeBSD 10.1-STABLE, >> version r281235 and it works fine for several years now. >> >> We have around 4Gbit/s and 1.8Mpps routed on peak while per port interface >> we peak at 300Kpps. >> >> Our quality metrics are measured with: >> >> ping -s 1472 -i 0.1 <our-other-ibgp-router> >> >> As well as iperf bidirecional. >> >> This metric is similar to what Speedy Test and SIMET tests are done and >> our >> customers reference. >> >> Systems working w/o problem: >> - 10.1-STABLE / r281235 >> >> Systems tested with drops: >> - 10.2-STABLE / r292035M >> - 10.3-STABLE / r298705 >> - 11.0-CURRENT / r295683 (downloaded snapshot from ftp.freebsd.org) >> - 11.0-CURRENT Melifaro Routing Branch / r297731M >> >> While testing, when errors happen I can see output errs on the vlan port >> on >> the output from "netstat -w1 -I vlan6" >> >> input vlan6 output >> packets errs idrops bytes packets errs bytes colls >> 1 0 0 66 30557 2 33310968 0 >> 1 0 0 105 31458 3 33912219 0 >> 2 0 0 2954 32001 8 34983986 0 >> 1 0 0 1512 33150 6 35942558 0 >> 1 0 0 1512 33654 4 37311862 0 >> 1 0 0 1512 34825 3 38213793 0 >> 3 0 0 1683 35376 4 39488912 0 >> 5 0 0 7280 32423 3 35551869 0 >> >> Problems may happen under high load (~200Kpps) or low load (~30Kpps) on a >> vlan port. The observed frame loss never happens on untagged ports, only >> vlan related. The observed loss happens with packets sized 900 bytes and >> above but noticeably loss rate is higher with packets close to 1400 (1472 >> is my reference size). >> >> Loss rate on all listed systems different from r281235 is 9-19% with >> ping(1) and iperf, while it's 0% on r281235. >> >> First I believed it to be a Intel driver error on systems newer than 10.1. >> My reference card are dual port 82599EB 10-Gigabit SFI/SFP+ Network >> Connection (2x2 on x8 PCIe bus, total 4x10G). But yesterday I replaced >> Intel by Chelsio T5 and the problem is still exactly the same, so it's not >> related to card vendor. >> >> I always test the very same hardware, I have two SSD drives in this >> router, >> one for the 10.1 which just runs fine and the other disk to test the >> various versions of FreeBSD. >> >> Only minor loader and sysctl confs are tweaked: >> >> kern.hz=2000 >> net.inet.ip.redirect=1 # do not send IP redirects >> net.inet.ip.accept_sourceroute=0 # drop source routed packets since >> they ca >> net.inet.ip.sourceroute=0 # if source routed packets are >> accepted th >> net.inet.tcp.drop_synfin=1 # SYN/FIN packets get dropped on >> initial c >> net.inet.udp.blackhole=1 # drop udp packets destined for >> closed soc >> net.inet.tcp.blackhole=2 # drop tcp packets destined for >> closed por >> security.bsd.see_other_uids=0 >> >> Can anyone suggest what might be a fix/tuning for this behavior? Was there >> any relevant change on vlan code from particular revisions close to the >> one >> I run on 10.1 and later which would lead to such a big difference? >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org" >> > > _______________________________________________ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
