Thanks Olivier, I will try it with the settings that you proposed.
Regards, Lyubo On Wed, 5 Feb 2020 at 17:15, Olivier Cochard-Labbé <oliv...@cochard.me> wrote: > On Wed, Feb 5, 2020 at 3:49 PM Lyubomir Yotov <l.yo...@gmail.com> wrote: > >> Dear Olivier, >> >> Hi, > > >> The switch is configured with: >> >> configure fdb agingtime 0 >> disable lldp ports 1,3,5,7 >> >> disable flow-control tx ports 1,3,5,7 >> disable flow-control rx ports 1,3,5,7 >> >> create vlan vlan2 tag 2 >> create vlan vlan3 tag 3 >> >> configure vlan "Default" del port 1,3,5,7 >> configure vlan2 add ports 1,3 >> configure vlan3 add ports 5,7 >> > > No static MAC address configured on the switch ? > So are you pinging the device between them before starting the benches to > be sure the switch have fully populated MAC addresses table ? > => If not the switch will "broadcast" packets to all ports belonging to > the vlan (with potential huge performance impact) because it doesn't know > where the destination is. > > cf line 73 of this example, where I'm using ping to check the lab status > (and populating switch MAC table) on one of my script here: > > https://github.com/ocochard/netbenches/blob/master/Atom_C2758_8Cores-Chelsio_T540-CR/bench-lab-2nodes.config#L73 > >> >> #####dut (DL360 G6) configuration: >> #cxl0 MAC 00:07:43:2f:29:b0 >> #cxl1 MAC 00:07:43:2f:29:b8 >> ifconfig cxl0 198.18.0.203/24 >> ifconfig cxl1 198.19.0.203/24 >> #to create static arp records on restart use >> sysrc static_arp_pairs="generator receiver" >> sysrc static_arp_generator="198.18.0.201 00:07:43:32:b1:61" >> sysrc static_arp_receiver="198.19.0.201 00:07:43:32:b1:69" >> #to create static arp records on the fly (lost after restart) >> arp -S 198.18.0.201 00:07:43:32:b1:61 >> arp -S 198.19.0.201 00:07:43:32:b1:69 >> #add a static route for 198.19.0.0/16 >> route add -net 198.19.0.0/16 198.19.0.201 >> >> > Did you disable ICMP redirect too on the router ? And the Chelsio advanced > hardware feature ? > Here is an example of my configuration file: > > https://github.com/ocochard/netbenches/tree/master/Atom_C2758_8Cores-Chelsio_T540-CR/forwarding-pf-ipfw/configs/forwarding > > >> #####start the test >> #start the receiver part >> pkt-gen -N -f rx -i vcxl1 -w 4 >> #start the generator part, use '-p 2' on order to achieve higher >> throughput from the processor. >> pkt-gen -N -f tx -i vcxl0 -n 1000000000 -4 -d 198.19.10.1:2000-198.19.10.100 >> -D 00:07:43:2f:29:b0 -s 198.18.10.1:2000-198.18.10.20 -S >> 00:07:43:32:b1:61 -w 4 -l 60 -U -p 2 >> > > pkt-gen option -U (software checksum calculation) has a big performance > impact (because my quick&dirty patch didn't use a pre-calculate table but > calculate checksum for each packet generated) and Chelsio NIC (vcxl0 here) > support doing hardware checksum calculation in netmap mode. > >> >> >> #####Results: >> The generator can generate about 5.5 Mpps. >> > > There is a problem here: With your hardware you should be able to generate > line-rate (14.48Mpps). > >> The router drops about 1.5 Mpps. >> The reciever gets about 4 Mpps. >> > > Wow, this is not good because lower than a small Atom (8core) CPU: And > this small Atom has a PDT of 20W comparing to this Xeon with a PDT of 80W. > > I never tested a Xeon E5410 with only 4 cores, but I would expected it be > able to forward a lot more than 4Mpps. > > cf Atom results here: > > https://github.com/ocochard/netbenches/tree/master/Atom_C2758_8Cores-Chelsio_T540-CR/forwarding-pf-ipfw/results/fbsd12-stable.r354440.BSDRP.1.96 > > There is a 100% saturation on one of the CPU cores and about 75% evenly >> distributed among the other cores on the router: >> > > If the load is evenly distributed, this confirm you are correctly > generating multi-flows. > >> >> The generator can generate more than 10 Mpps but the number of flows >> should be one or two for this hardware configuration: >> pkt-gen -N -f tx -i vcxl0 -n 1000000000 -4 -d 198.19.10.1:2000-198.19.10.2 >> -D 00:07:43:2f:29:b0 -s 198.18.10.1:2000-198.18.10.10 -S >> 00:07:43:32:b1:61 -w 4 -l 60 -U -p 2 >> >> > Try without -U, and it will be able to reach line-rate. > Disable all unneeded Chelsio advanced feature too (check the > loader.conf.local examples). > > Regards, > > Olivier > _______________________________________________ > Bsdrp-users mailing list > Bsdrp-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bsdrp-users >
_______________________________________________ Bsdrp-users mailing list Bsdrp-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bsdrp-users
