Hi Billy, The output of “show l2fib” is showing how many MAC entries exist in the L2FIB and is not relevant to the size of L2FIB table. The L2FIB table size is not configurable. It is a bi-hash table with size set by the following #def’s in l2_fib.h and has not changed for quite a while, definitely not between 1704, 1707 and current master: /* * The size of the hash table */ #define L2FIB_NUM_BUCKETS (64 * 1024) #define L2FIB_MEMORY_SIZE (256<<20)
It is interesting to note that at the end of the test run, there is different number of MAC entries in the L2FIB. I think this may have to do with a change in 1707 where an interface up/down would cause MACs learned on that interface to be flushed. So when the interface come back up, the MACs needs to be learned again. With 1704, the stale learned MACs from an interface will remain in L2FIB even if the interface is down or deleted unless aging is enabled to removed them at the BD aging interval. Another improvement added in 1707 was a check in the l2-fwd node so when a MAC entry is found in L2FIB, its sequence number is checked to make sure it is not stale and subject to flushing (such as MAC learned when this interface sw_if_index was up but went down, or if this sw_if_index was used, deleted and reused). If the MAC is stale, the packet will be flooded instead of making use of the stale MAC entry to forward it. I wonder if the test script for the performance does create/delete interfaces or set interface to admin up/down states causing stale MACs be flushed in 1707? With 1704, it may be using stale MAC entries to forward packets rather than flooding to learn the MACs again. This can explain the l2-flood and l2-input count ratio difference between 1704 and 1707. When measuring l2-bridg forwarding performance, are you setup to measure the forwarding rate in the steady forwarding state? If all the 10K or 1M flows are started at the same time for a particular test, there will be an initial low PPS throughput period when all packets needs to be flooded and MACs earned before it settle down to a higher steady state PPS forwarding rate. If there is interface flap or other events that causes MAC flush, the MAC will needs to be learned again. I wonder if the forwarding performance for 10K or 1M flows is measure at the steady forwarding state or not. Above are a few generic comments I can think of, without knowing much details about how the tests are setup and measured. Hope it can help to explain the different behavior observed between 1704 and 1707. Regards, John From: vpp-dev-boun...@lists.fd.io [mailto:vpp-dev-boun...@lists.fd.io] On Behalf Of Billy McFall Sent: Monday, August 14, 2017 6:40 PM To: vpp-dev@lists.fd.io Subject: [vpp-dev] VPP Performance drop from 17.04 to 17.07 In the last VPP call, I reported some internal Red Hat performance testing was showing a significant drop in performance between releases 17.04 to 17.07. This with l2-bridge testing - PVP - 0.002% Drop Rate: VPP-17.04: 256 Flow 7.8 MP/s 10k Flow 7.3 MP/s 1m Flow 5.2 MP/s VPP-17.07: 256 Flow 7.7 MP/s 10k Flow 2.7 MP/s 1m Flow 1.8 MP/s The performance team re-ran some of the tests for me with some additional data collected. Looks like the size of the L2 FIB table was reduced in 17.07. Below are the number of entries in the MAC Table after the tests are run: 17.04: show l2fib 4000008 l2fib entries 17.07: show l2fib 1067053 l2fib entries with 1048576 learned (or non-static) entries This caused more packets to be flooded (see out of 'show node counters' below). I looked but couldn't find anything. Is the size of the L2 FIB Table table configurable? Thanks, Billy McFall 17.04: show node counters Count Node Reason : 313035313 l2-input L2 input packets 555726 l2-flood L2 flood packets : 310115490 l2-input L2 input packets 824859 l2-flood L2 flood packets : 313508376 l2-input L2 input packets 1041961 l2-flood L2 flood packets : 313691024 l2-input L2 input packets 698968 l2-flood L2 flood packets 17.07: show node counters Count Node Reason : 97810569 l2-input L2 input packets 72557612 l2-flood L2 flood packets : 97830674 l2-input L2 input packets 72478802 l2-flood L2 flood packets : 97714888 l2-input L2 input packets 71655987 l2-flood L2 flood packets : 97710374 l2-input L2 input packets 70058006 l2-flood L2 flood packets -- Billy McFall SDN Group Office of Technology Red Hat
_______________________________________________ vpp-dev mailing list vpp-dev@lists.fd.io https://lists.fd.io/mailman/listinfo/vpp-dev
