[dpdk-dev] Very low performance with l2fwd in a VM with PCI Passthrough
Hi, I am seeing very low throughput when I run l2fwd in a VM. When I send 1g data, I could see only about 10mbps of traffic being received by the VM. I dont see this problem when running l2fwd on the host. I could get 10gbps traffic in and out of both the ports. My setup details are - Traffic coming in on a 10G interface(eth0) and going out on another 10G interface(eth1) - Both 10G NICs are 82599 - Created virtual functions eth5 and eth8 on eth0 and eth1 respectively - eth5 and eth8 are mapped to eth0 and eth1 in a VM, with device type as e1000 in Passthru mode. - Host OS : Centos 6.2, Guest OS : Ubuntu 12.04 Network devices using IGB_UIO driver :00:07.0 '82540EM Gigabit Ethernet Controller' drv=igb_uio unused=e1000 :00:08.0 '82540EM Gigabit Ethernet Controller' drv=igb_uio unused=e1000 The counters on the virtual interface (eth5/eth8) show all 10g traffic received, but I see very 10mbps worth traffic at the l2fwd app counters on the VM. I also see the following logs in the VM dmesg output. [75399.491215] irq 0xb not handled [75400.142025] irq 0xb not handled [75400.142150] irq 0xb not handled [75400.142913] irq 0xb not handled [75400.142920] irq 0xb not handled Has anyone seen this issue? Am I missing anything? Thanks, Surya
[dpdk-dev] Which vmxnet3 pmd is to be used in dpdk 1.6.x?
Prashant, Are you still hitting a crash? Do you have hugetlbfs enabled? if not try adding enabling hugetlbfs. I was able to get vxnet3 up using the vmxnet3 pmd alone in DPDK 1.60r1. I did not use the vmxnet3-usermap kernel driver. I am using Esxi 5.5 Eg: echo 1024 > /proc/sys/vm/nr_hugepages; mkdir /huge; mount -t hugetlbfs nodev /huge; On Thu, Mar 6, 2014 at 12:48 PM, Prashant Upadhyaya wrote: > And if it is any help, here is the startup log -- > > EAL: Detected lcore 0 as core 0 on socket 0 > EAL: Detected lcore 1 as core 1 on socket 0 > EAL: Detected lcore 2 as core 2 on socket 0 > EAL: Detected lcore 3 as core 3 on socket 0 > EAL: Detected lcore 4 as core 4 on socket 0 > EAL: Detected lcore 5 as core 5 on socket 0 > EAL: Detected lcore 6 as core 6 on socket 0 > EAL: Detected lcore 7 as core 7 on socket 0 > EAL: Skip lcore 8 (not detected) > EAL: Skip lcore 9 (not detected) > EAL: Skip lcore 10 (not detected) > EAL: Skip lcore 11 (not detected) > EAL: Skip lcore 12 (not detected) > EAL: Skip lcore 13 (not detected) > EAL: Skip lcore 14 (not detected) > EAL: Skip lcore 15 (not detected) > EAL: Skip lcore 16 (not detected) > EAL: Skip lcore 17 (not detected) > EAL: Skip lcore 18 (not detected) > EAL: Skip lcore 19 (not detected) > EAL: Skip lcore 20 (not detected) > EAL: Skip lcore 21 (not detected) > EAL: Skip lcore 22 (not detected) > EAL: Skip lcore 23 (not detected) > EAL: Skip lcore 24 (not detected) > EAL: Skip lcore 25 (not detected) > EAL: Skip lcore 26 (not detected) > EAL: Skip lcore 27 (not detected) > EAL: Skip lcore 28 (not detected) > EAL: Skip lcore 29 (not detected) > EAL: Skip lcore 30 (not detected) > EAL: Skip lcore 31 (not detected) > EAL: Skip lcore 32 (not detected) > EAL: Skip lcore 33 (not detected) > EAL: Skip lcore 34 (not detected) > EAL: Skip lcore 35 (not detected) > EAL: Skip lcore 36 (not detected) > EAL: Skip lcore 37 (not detected) > EAL: Skip lcore 38 (not detected) > EAL: Skip lcore 39 (not detected) > EAL: Skip lcore 40 (not detected) > EAL: Skip lcore 41 (not detected) > EAL: Skip lcore 42 (not detected) > EAL: Skip lcore 43 (not detected) > EAL: Skip lcore 44 (not detected) > EAL: Skip lcore 45 (not detected) > EAL: Skip lcore 46 (not detected) > EAL: Skip lcore 47 (not detected) > EAL: Skip lcore 48 (not detected) > EAL: Skip lcore 49 (not detected) > EAL: Skip lcore 50 (not detected) > EAL: Skip lcore 51 (not detected) > EAL: Skip lcore 52 (not detected) > EAL: Skip lcore 53 (not detected) > EAL: Skip lcore 54 (not detected) > EAL: Skip lcore 55 (not detected) > EAL: Skip lcore 56 (not detected) > EAL: Skip lcore 57 (not detected) > EAL: Skip lcore 58 (not detected) > EAL: Skip lcore 59 (not detected) > EAL: Skip lcore 60 (not detected) > EAL: Skip lcore 61 (not detected) > EAL: Skip lcore 62 (not detected) > EAL: Skip lcore 63 (not detected) > EAL: Setting up memory... > EAL: Ask a virtual area of 0x8000 bytes > EAL: Virtual area found at 0x7f848ae0 (size = 0x8000) > EAL: Requesting 1024 pages of size 2MB from socket 0 > EAL: TSC frequency is ~200 KHz > EAL: Master core 0 is ready (tid=b3f3f00) > EAL: Core 1 is ready (tid=8a1f2700) > EAL: Core 2 is ready (tid=899f1700) > EAL: Core 3 is ready (tid=891f0700) > EAL: Core 4 is ready (tid=889ef700) > EAL: Core 5 is ready (tid=7bfff700) > EAL: Core 6 is ready (tid=7b7fe700) > EAL: Core 7 is ready (tid=7affd700) > Pool initialized > Global Variables initialized > PMD: rte_vmxnet3_pmd_init(): >> > EAL: PCI device :0b:00.0 on NUMA socket -1 > EAL: probe driver: 15ad:7b0 rte_vmxnet3_pmd > PMD: eth_vmxnet3_dev_init(): >> > Segmentation fault (core dumped) > > -Original Message- > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Prashant Upadhyaya > Sent: Thursday, March 06, 2014 12:20 PM > To: David Marchand > Cc: dev at dpdk.org > Subject: Re: [dpdk-dev] Which vmxnet3 pmd is to be used in dpdk 1.6.x? > > Hi, > > Some further update on the crash I am facing. > I am using DPDK1.6.0r1 and take over the vmxnet3 with igb_uio and then start > the application. (so no external ko or vmxnet usermap etc.) During the port > initializations, the crash is happening in the following function - > > eth_vmxnet3_dev_init > > and the crash is happening at the following line - > /* Check h/w version compatibility with driver. */ > ver = VMXNET3_READ_BAR1_REG(hw, VMXNET3_REG_VRRS); > > Any hints regarding what could be wrong ? > > Regards > -Prashant > > > From: Prashant Upadhyaya > Sent: Wednesday, March 05, 2014 9:01 PM > To: 'David Marchand' > Cc: Srinivasan J; dev at dpdk.org > Subject: RE: [dpdk-dev] Which vmxnet3 pmd is to be used in dpdk 1.6.x? > > Hi David, > > The compilation error with debug flags on was that the functions -- > vmxnet3_txq_dump, vmxnet3_rxq_dump - are defined but not used. > Not a serious error, I will try to get rid of the compiler flag which > generates this. > > However, I must reiterate, I _did_ bind my vmxnet3 device with igb_uio (and I > did not use
[dpdk-dev] Which vmxnet3 pmd is to be used in dpdk 1.6.x?
Hi Srini, Yes, I am still stuck, I am using ESXi5.1 though and Fedora18 in Guest. Please let me know what is the guest OS distro you are using in the VM. I'll try with that once. Regards -Prashant -Original Message- From: Srinivasan J [mailto:srinid...@gmail.com] Sent: Friday, March 07, 2014 9:08 AM To: Prashant Upadhyaya Cc: David Marchand; dev at dpdk.org Subject: Re: [dpdk-dev] Which vmxnet3 pmd is to be used in dpdk 1.6.x? Prashant, Are you still hitting a crash? Do you have hugetlbfs enabled? if not try adding enabling hugetlbfs. I was able to get vxnet3 up using the vmxnet3 pmd alone in DPDK 1.60r1. I did not use the vmxnet3-usermap kernel driver. I am using Esxi 5.5 Eg: echo 1024 > /proc/sys/vm/nr_hugepages; mkdir /huge; mount -t hugetlbfs nodev /huge; On Thu, Mar 6, 2014 at 12:48 PM, Prashant Upadhyaya wrote: > And if it is any help, here is the startup log -- > > EAL: Detected lcore 0 as core 0 on socket 0 > EAL: Detected lcore 1 as core 1 on socket 0 > EAL: Detected lcore 2 as core 2 on socket 0 > EAL: Detected lcore 3 as core 3 on socket 0 > EAL: Detected lcore 4 as core 4 on socket 0 > EAL: Detected lcore 5 as core 5 on socket 0 > EAL: Detected lcore 6 as core 6 on socket 0 > EAL: Detected lcore 7 as core 7 on socket 0 > EAL: Skip lcore 8 (not detected) > EAL: Skip lcore 9 (not detected) > EAL: Skip lcore 10 (not detected) > EAL: Skip lcore 11 (not detected) > EAL: Skip lcore 12 (not detected) > EAL: Skip lcore 13 (not detected) > EAL: Skip lcore 14 (not detected) > EAL: Skip lcore 15 (not detected) > EAL: Skip lcore 16 (not detected) > EAL: Skip lcore 17 (not detected) > EAL: Skip lcore 18 (not detected) > EAL: Skip lcore 19 (not detected) > EAL: Skip lcore 20 (not detected) > EAL: Skip lcore 21 (not detected) > EAL: Skip lcore 22 (not detected) > EAL: Skip lcore 23 (not detected) > EAL: Skip lcore 24 (not detected) > EAL: Skip lcore 25 (not detected) > EAL: Skip lcore 26 (not detected) > EAL: Skip lcore 27 (not detected) > EAL: Skip lcore 28 (not detected) > EAL: Skip lcore 29 (not detected) > EAL: Skip lcore 30 (not detected) > EAL: Skip lcore 31 (not detected) > EAL: Skip lcore 32 (not detected) > EAL: Skip lcore 33 (not detected) > EAL: Skip lcore 34 (not detected) > EAL: Skip lcore 35 (not detected) > EAL: Skip lcore 36 (not detected) > EAL: Skip lcore 37 (not detected) > EAL: Skip lcore 38 (not detected) > EAL: Skip lcore 39 (not detected) > EAL: Skip lcore 40 (not detected) > EAL: Skip lcore 41 (not detected) > EAL: Skip lcore 42 (not detected) > EAL: Skip lcore 43 (not detected) > EAL: Skip lcore 44 (not detected) > EAL: Skip lcore 45 (not detected) > EAL: Skip lcore 46 (not detected) > EAL: Skip lcore 47 (not detected) > EAL: Skip lcore 48 (not detected) > EAL: Skip lcore 49 (not detected) > EAL: Skip lcore 50 (not detected) > EAL: Skip lcore 51 (not detected) > EAL: Skip lcore 52 (not detected) > EAL: Skip lcore 53 (not detected) > EAL: Skip lcore 54 (not detected) > EAL: Skip lcore 55 (not detected) > EAL: Skip lcore 56 (not detected) > EAL: Skip lcore 57 (not detected) > EAL: Skip lcore 58 (not detected) > EAL: Skip lcore 59 (not detected) > EAL: Skip lcore 60 (not detected) > EAL: Skip lcore 61 (not detected) > EAL: Skip lcore 62 (not detected) > EAL: Skip lcore 63 (not detected) > EAL: Setting up memory... > EAL: Ask a virtual area of 0x8000 bytes > EAL: Virtual area found at 0x7f848ae0 (size = 0x8000) > EAL: Requesting 1024 pages of size 2MB from socket 0 > EAL: TSC frequency is ~200 KHz > EAL: Master core 0 is ready (tid=b3f3f00) > EAL: Core 1 is ready (tid=8a1f2700) > EAL: Core 2 is ready (tid=899f1700) > EAL: Core 3 is ready (tid=891f0700) > EAL: Core 4 is ready (tid=889ef700) > EAL: Core 5 is ready (tid=7bfff700) > EAL: Core 6 is ready (tid=7b7fe700) > EAL: Core 7 is ready (tid=7affd700) > Pool initialized > Global Variables initialized > PMD: rte_vmxnet3_pmd_init(): >> > EAL: PCI device :0b:00.0 on NUMA socket -1 > EAL: probe driver: 15ad:7b0 rte_vmxnet3_pmd > PMD: eth_vmxnet3_dev_init(): >> > Segmentation fault (core dumped) > > -Original Message- > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Prashant > Upadhyaya > Sent: Thursday, March 06, 2014 12:20 PM > To: David Marchand > Cc: dev at dpdk.org > Subject: Re: [dpdk-dev] Which vmxnet3 pmd is to be used in dpdk 1.6.x? > > Hi, > > Some further update on the crash I am facing. > I am using DPDK1.6.0r1 and take over the vmxnet3 with igb_uio and then > start the application. (so no external ko or vmxnet usermap etc.) > During the port initializations, the crash is happening in the > following function - > > eth_vmxnet3_dev_init > > and the crash is happening at the following line - > /* Check h/w version compatibility with driver. */ > ver = VMXNET3_READ_BAR1_REG(hw, VMXNET3_REG_VRRS); > > Any hints regarding what could be wrong ? > > Regards > -Prashant > > > From: Prashant Upadhyaya > Sent: Wednesday, March 05, 2014 9:01 PM > To: 'David Marchand
[dpdk-dev] Very low performance with l2fwd in a VM with PCI Passthrough
When change the device emulation in the NIC to virtio, I could get 900mbps when sending 1g of input. Good so far. But when I increased the traffic rate to 10g, output became very turbulent - ranging from 0 to 150mbps. I tried rate limiting on the vfs using the following commands, but didnt help. ip link set eth0 vf 0 rate 1000 ip link set eth1 vf 0 rate 1000 Are there any recommended settings for using DPDK over virtual function in a VM? Thanks, Surya On Mar 6, 2014, at 7:08 PM, Surya Nimmagadda wrote: > Hi, > > I am seeing very low throughput when I run l2fwd in a VM. > > When I send 1g data, I could see only about 10mbps of traffic being received > by the VM. > > I dont see this problem when running l2fwd on the host. I could get 10gbps > traffic in and out of both the ports. > > My setup details are > > - Traffic coming in on a 10G interface(eth0) and going out on another 10G > interface(eth1) > - Both 10G NICs are 82599 > - Created virtual functions eth5 and eth8 on eth0 and eth1 respectively > - eth5 and eth8 are mapped to eth0 and eth1 in a VM, with device type as > e1000 in Passthru mode. > - Host OS : Centos 6.2, Guest OS : Ubuntu 12.04 > > > Network devices using IGB_UIO driver > > :00:07.0 '82540EM Gigabit Ethernet Controller' drv=igb_uio unused=e1000 > :00:08.0 '82540EM Gigabit Ethernet Controller' drv=igb_uio unused=e1000 > > > The counters on the virtual interface (eth5/eth8) show all 10g traffic > received, but I see very 10mbps worth traffic at the l2fwd app counters on > the VM. > > I also see the following logs in the VM dmesg output. > > [75399.491215] irq 0xb not handled > [75400.142025] irq 0xb not handled > [75400.142150] irq 0xb not handled > [75400.142913] irq 0xb not handled > [75400.142920] irq 0xb not handled > > Has anyone seen this issue? Am I missing anything? > > Thanks, > Surya > >
[dpdk-dev] Very low performance with l2fwd in a VM with PCI Passthrough
> My setup details are > > - Traffic coming in on a 10G interface(eth0) and going out on another 10G > interface(eth1) > - Both 10G NICs are 82599 > - Created virtual functions eth5 and eth8 on eth0 and eth1 respectively > - eth5 and eth8 are mapped to eth0 and eth1 in a VM, with device type as > e1000 in Passthru mode. If the NIC in the VM is showing up as an e1000 then you have not passed the VF through to the VM at all, instead you are using an emulated NIC, which will be very, very slow in comparison - if nothing else, it's an emulated 1G NIC, not a 10G one. To set up a VM and use the virtual functions from the 10G NIC, I suggest referring to Chapter 9 of the Programmers Guide available at http://www.intel.com/content/www/us/en/intelligent-systems/intel-technology/intel-dpdk-programmers-guide.html . Instructions for setting up the VM to use VFs are covered by section 9.2
[dpdk-dev] Intel 82571EB PMD support
Hi, Do we have PMD support for intel 82571EB. We are having e1000e driver supported for 82571EB in linux. DPDK Version : 1.6.0-r1 Linux Kernel : CentOS 6.4 [ kernel version 2.6.32-358.el6.x86_64 ] Issue: When we run DPDK example [ l2fwd ], we are able to see successful init of rte_pmd_em and link is also detected successfully. But the RX, TX count are zero. If we try to fetch nic stats using rte_eth_stats_get(), we can see around 256 ipacket count but after that error count [rx error ] starts increasing. We are not able to see any packet on user app [ l2fwd ]. Does anyone else face the same issue? Regards Chetan Bali === Please refer to http://www.aricent.com/legal/email_disclaimer.html for important disclosures regarding this electronic communication. ===
[dpdk-dev] Topology question
Hi guys, I am running DPDK on native Linux environment. I also running Windows 7 under KVM virtualized environment for GUI purposes. What is the best way of creating Layer 2 communication channel between Windows application (which uses winpcap) and DPDK application running under Linux. Performance is not important here, as it used for control purposes only. I somehow need to exchange packets with KVM virtual Linux driver. According to my understanding, KNI device must be bounded to physical NIC only. Thanks
[dpdk-dev] [PATCH] mk: pass CROSS_COMPILE when compiling kernel modules
Hi, On Tue, Feb 25, 2014 at 10:55:36PM +0100, Thomas Monjalon wrote: > 07/02/2014 18:44, Aaro Koskinen : > > Pass CROSS_COMPILE to the kernel build system when compiling kernel > > modules. Although we export CC etc. the top level kernel Makefile will > > override the environment. As a result it will end up using wrong tools > > if cross-compilation is desired but CROSS_COMPILE is not set. > > > > Signed-off-by: Aaro Koskinen > > Could you explain why it's needed ? > In a basic test, CC=$(CROSS)gcc in rte.vars.mk seems sufficient. If you compile on 32-bit host to 64-bit target, the build fails with errors such as: [...]include/linux/kernfs.h:331:1: note: expected 'const char *' but argument is of type 'const unsigned char *' kernfs_find_and_get(struct kernfs_node *kn, const char *name) ^ cc1: all warnings being treated as errors Build with V=1 shows that following appears in the command line when compiling the kernel modules: -isystem /usr/lib/gcc/i486-linux-gnu/4.8/include With the patch this will be corrected to point to my target's sysroot: -isystem /foo/bar/lib/gcc/x86_64-linux-gnu/4.8.2/include And the build succeeds without any warnings. A.
[dpdk-dev] [PATCH 0/6] DPDK 1.6.1 fixes
These are patches for issues found during integration of 1.6.1 and also during product development on earlier release.
[dpdk-dev] [PATCH 1/6] rte_mbuf: copy offload flags when doing attach/clone of mbuf
An embedded and charset-unspecified text was scrubbed... Name: mbuf-ol-flags.patch URL: <http://dpdk.org/ml/archives/dev/attachments/20140307/f57fa0db/attachment.ksh>
[dpdk-dev] [PATCH 2/6] qos: use rte_zmalloc instead of memzone for allocation
An embedded and charset-unspecified text was scrubbed... Name: sched-rte-malloc.patch URL: <http://dpdk.org/ml/archives/dev/attachments/20140307/e7675261/attachment.ksh>
[dpdk-dev] [PATCH 3/6] rte_jhash: make arg to rte_jhash2 const
An embedded and charset-unspecified text was scrubbed... Name: jhash2-const.patch URL: <http://dpdk.org/ml/archives/dev/attachments/20140307/315eea71/attachment.ksh>
[dpdk-dev] [PATCH 4/6] mempool: use GCC push/pop
An embedded and charset-unspecified text was scrubbed... Name: mempool-opts.patch URL: <http://dpdk.org/ml/archives/dev/attachments/20140307/954a6606/attachment.ksh>
[dpdk-dev] [PATCH 5/6] xen: dont create dependency on cmdline library
An embedded and charset-unspecified text was scrubbed... Name: xen-eth-aton.patch URL: <http://dpdk.org/ml/archives/dev/attachments/20140307/d60206e7/attachment.ksh>
[dpdk-dev] [PATCH 6/6] ivshmem: fix errors identified by hardening
An embedded and charset-unspecified text was scrubbed... Name: ivshmem-fortify.patch URL: <http://dpdk.org/ml/archives/dev/attachments/20140307/7a8e04e3/attachment.ksh>
