Some more discovery. That flag PKT_TX_VLAN_PKT eventually send the following command to the chips (ixgbe_xmit_pkts(), tx_desc_vlan_flags_to_cmdtype()) . Is this actually adding a VLAN tag during tx ?
#define IXGBE_ADVTXD_DCMD_VLE IXGBE_TXD_CMD_VLE /* VLAN pkt enable */ #define IXGBE_TXD_CMD_VLE 0x40000000 /* Add VLAN tag */ After I intentionally remove the tag before sending out the traffic, the Spirent can receive tagged packets. I am not sure the DPDK codes is adding a tag. I think the ixgbevf on the host is adding VLAN tag based on the configured tag ID. Is this expected ? How can I configure the host to not add tag since traffic from the VM is already tagged. Otherwise it becomes double tagged. Thanks James On Fri, Jan 24, 2014 at 6:07 AM, James Yu <ypyu2011 at gmail.com> wrote: > Stephen, > > I added that to the l2fwd main.c and it is still not received by the > Spirent. I obtained the stat using rte_eth_stats_get() and printout the > following when I send only one VLAN pkt > ipackets: increased by 1 > opackets: no change (pkt not sent out by the device) > ierrors, oerrors, otxerror, otxdiscard: all 0 > > The same thing with 10 packets which increase 10 ipackets counts. > > I found out that the rte_eth_stats_get() obtains the stats from > ixgbevf_dev_stats_get(). Is this expected for SR-IOV that uses > ixgbe_ethdev.c ? > If that's correct, I think this tells me that the tagged packet is > received by the VF PMD codes but not sent out and has no error or packets > dropped. > > So it was not sent out of the VF PMD. But why there is not error or packet > dropped. Where did it go ? I am trying to find out where it got dropped and > how to fix it. > > I tried DPDK1.5.2r1. It exhibits the same issue even with the patch. > > Thanks > > James > > > > On Thu, Jan 23, 2014 at 1:43 PM, Stephen Hemminger < > stephen at networkplumber.org> wrote: > >> On Thu, 23 Jan 2014 11:27:14 -0800 >> James Yu <ypyu2011 at gmail.com> wrote: >> >> > Hi Daniel, >> > >> > Untagged traffic could be looped back. I am doing a one-directional loop >> > back. In the Frame tabe of the StreamBlock editor, I did the following >> > stream1: >> > destination address: MAC address of the receiving port of the guest >> > VM. (obtained from the debug print out when l2fwd is started, or >> ifconfig >> > before l2fwd is started) >> > stream2: >> > destination address: MAC address of the sending port of the guest >> VM >> > (obtained from the debug print out when l2fwd is started, or ifconfig >> > before l2fwd is started) >> > >> > >> > The only thing I add is to tag both the SR-IOV VF ports on the host with >> > the same tag using ip command and add that tag number as the VID to the >> > Spirent streams. After that that traffic can reach l2fwd which reported >> > forwarded the packets on the periodic printout. However the packets did >> not >> > reach the Spirent. >> > >> > Also it is exercising rte_ixgbevf_pmd codes as indicated during starting >> > l2fwd. >> > >> > EAL: Master core 0 is ready (tid=b77ab7d0) >> > >> > EAL: Core 1 is ready (tid=adffeb70) >> > >> > EAL: probe driver: 8086:100e rte_em_pmd >> > >> > EAL: probe driver: 8086:100e rte_em_pmd >> > >> > *EAL: probe driver: 8086:10ed rte_ixgbevf_pmd* >> > I wonder do I have to change any codes in the DPDK library or call >> certain >> > routines to setup the VLAN in the DPDK stack during starting the l2fwd ? >> > >> > >> > Were you able to send VLAN traffic from your iXia and looped back to >> your >> > iXia ? >> > >> > >> > Thanks >> > >> > James >> > >> > >> > >> > >> > On Thu, Jan 23, 2014 at 12:45 AM, Daniel Kaminsky < >> > daniel.kaminsky at infinitelocality.com> wrote: >> > >> > > Hi James, >> > > >> > > Did you try a non-tagged traffic? I suspect this is the limitation of >> the >> > > l2fwd or the requirements of Spirent. From my experience with Ixia, >> the L2 >> > > forward device must implement the ARP protocol (at least a subset of >> it). >> > > >> > > Also, notice that the l2fwd example sets the destination MAC address >> to >> > > "02:00:00:00:00:xx" which the Spirent might doesn't like. >> > > >> > > Daniel >> > > >> > > >> > > On Thu, Jan 23, 2014 at 7:08 AM, James Yu <ypyu2011 at gmail.com> wrote: >> > > >> > >> I could not pass tagged traffic through the l2fwd program running >> inside a >> > >> CentOS VM. The l2fwd program reported sending out all the packets >> received >> > >> from one port to the other port. But the outside Spirent tester >> could not >> > >> receive a packet at all. I am wondering maybe the tagged packets are >> > >> dropped somewhere in the rte_em_pmd or in the DPDK library. Anyone >> knows >> > >> how to pass/send VLAN traffic through the DPDK ? >> > >> >> > >> With tagged SR-IOV ports, Spirent and guest VM CentOS could ping each >> > >> other >> > >> without using DPDK. Once I used DPDK 1.3.1r2, I encountered issues >> with >> > >> sending out tagged packets out of DPDK to the host. >> > >> >> > >> I have done the following: >> > >> >> > >> On RHEL6.1 host >> > >> 1. setup the VLAN using "ip link set eth2 vf 0 vlan 3" for port 1 >> and "ip >> > >> link set eth3 vf 0 vlan 3" for port 2 >> > >> [root at cent64x64-137189 ~]# ip link show eth2 >> > >> 96: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state >> UP >> > >> qlen >> > >> 1000 >> > >> link/ether 90:e2:ba:2e:af:f0 brd ff:ff:ff:ff:ff:ff >> > >> vf 0 MAC 52:54:00:f9:9a:90, vlan 3 >> > >> [root at cent64x64-137189 ~]# ip link show eth3 >> > >> 98: eth3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state >> UP >> > >> qlen >> > >> 1000 >> > >> link/ether 90:e2:ba:2e:af:f1 brd ff:ff:ff:ff:ff:ff >> > >> vf 0 MAC 52:54:00:30:37:98, vlan 3 >> > >> >> > >> 2. virsh configure SR-IOV hostdev >> > >> <interface type='hostdev' managed='yes'> >> > >> <mac address='52:54:00:f9:9a:90'/> >> > >> <source> >> > >> <address type='pci' domain='0x0000' bus='0x1a' slot='0x10' >> > >> function='0x0'/> >> > >> </source> >> > >> <address type='pci' domain='0x0000' bus='0x00' slot='0x08' >> > >> function='0x0'/> >> > >> </interface> >> > >> <interface type='hostdev' managed='yes'> >> > >> <mac address='52:54:00:30:37:98'/> >> > >> <source> >> > >> <address type='pci' domain='0x0000' bus='0x1a' slot='0x10' >> > >> function='0x1'/> >> > >> </source> >> > >> <address type='pci' domain='0x0000' bus='0x00' slot='0x09' >> > >> function='0x0'/> >> > >> </interface> >> > >> >> > >> [root at cent64x64-137189 ~]# lspci |grep Eth >> > >> 1a:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit >> SFI/SFP+ >> > >> Network Connection (rev 01) >> > >> 1a:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit >> SFI/SFP+ >> > >> Network Connection (rev 01) >> > >> 1a:10.0 Ethernet controller: Intel Corporation 82599 Ethernet >> Controller >> > >> Virtual Function (rev 01) >> > >> 1a:10.1 Ethernet controller: Intel Corporation 82599 Ethernet >> Controller >> > >> Virtual Function (rev 01) >> > >> >> > >> >> > >> On guest VM >> > >> 1. check all the ports are started up and check their corresponding >> port >> > >> type. Use the two SR-IOV ports with device type as ixgbevf for l2fwd >> > >> [root at VirtualADX ~]# lspci |grep Eth >> > >> 00:03.0 Ethernet controller: Intel Corporation 82540EM Gigabit >> Ethernet >> > >> Controller (rev 03) >> > >> 00:07.0 Ethernet controller: Intel Corporation 82540EM Gigabit >> Ethernet >> > >> Controller (rev 03) >> > >> 00:08.0 Ethernet controller: Intel Corporation 82599 Ethernet >> Controller >> > >> Virtual Function (rev 01) >> > >> 00:09.0 Ethernet controller: Intel Corporation 82599 Ethernet >> Controller >> > >> Virtual Function (rev 01) >> > >> 00:0a.0 Ethernet controller: Intel Corporation 82540EM Gigabit >> Ethernet >> > >> Controller (rev 03) >> > >> >> > >> 2. run this command: >> > >> /root/l2fwd -c 3 -n 1 -b 000:00:03.0 -b 000:00:07.0 -b >> > >> 000:00:0a.0 -- -q 1 -p 3 >> > >> >> > >> Although the l2fwd forwards all received tagged packets, the >> Spirent >> > >> tester did not receive any packet at all. >> > >> >> > > >> > > >> >> The VLAN offload bit in the mbuf is different for TX and RX. >> Probably would work with something like this: >> >> diff --git a/examples/l2fwd/main.c b/examples/l2fwd/main.c >> index 05ef330..70dbf85 100644 >> --- a/examples/l2fwd/main.c >> +++ b/examples/l2fwd/main.c >> @@ -289,6 +289,9 @@ l2fwd_simple_forward(struct rte_mbuf *m, unsigned >> portid) >> /* src addr */ >> ether_addr_copy(&l2fwd_ports_eth_addr[dst_port], ð->s_addr); >> >> + if (m->ol_flags & PKT_RX_VLAN_PKT) >> + m->ol_flags |= PKT_TX_VLAN_PKT; >> + >> l2fwd_send_packet(m, (uint8_t) dst_port); >> } >> >> >> >> >
