Hello dpdk experts,
I am following the documentation available on dpdk.org & the documentation
published by Intel to learn how to use DPDK libraries.
I am using dpdk v1.7.1 downloaded from dpdk.org. I wish to run testpmd app
provided with this release with 2 VMXNET3 vNICs on my setup.
My test setup is a 64-bit Ubunutu 14.04 (3.13.0) Linux VM running on a VMWare
ESXi-5.5.0 hypervisor on Dell server with Intel Xeon E5-2440.
I followed the following stpes to capture the vmxnet3 vNICs in DPDK (all
commands run as a root user):
cd /home/sandesh/dpdk/dpdk-1.7.1/
export RTE_SDK=/home/sandesh/dpdk/dpdk-1.7.1/
export RTE_TARGET=build
modprobe uio
insmod build/kmod/igb_uio.ko
modprobe vfio-pci
mkdir -p /mnt/huge
mount -t hugetlbfs nodev /mnt/huge
echo 64 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages
tools/dpdk_nic_bind.py --status
Network devices using DPDK-compatible driver
============================================
<none>
Network devices using kernel driver
===================================
0000:02:01.0 '82545EM Gigabit Ethernet Controller (Copper)' if=eth2 drv=e1000
unused=igb_uio,vfio-pci *Active*
0000:03:00.0 'VMXNET3 Ethernet Controller' if=eth0 drv=vmxnet3
unused=igb_uio,vfio-pci *Active*
0000:0b:00.0 'VMXNET3 Ethernet Controller' if=eth1 drv=vmxnet3
unused=igb_uio,vfio-pci *Active*
Other network devices
=====================
<none>
Then I bind VMXNET3 vNICs to igb_uio driver:
../dpdk-1.7.1/tools/dpdk_nic_bind.py --bind=igb_uio eth0 eth1
../dpdk-1.7.1/tools/dpdk_nic_bind.py --status
Network devices using DPDK-compatible driver
============================================
<none>
Network devices using kernel driver
===================================
0000:02:01.0 '82545EM Gigabit Ethernet Controller (Copper)' if=eth2 drv=e1000
unused=igb_uio,vfio-pci *Active*
0000:03:00.0 'VMXNET3 Ethernet Controller' if=eth0 drv=vmxnet3
unused=igb_uio,vfio-pci *Active*
0000:0b:00.0 'VMXNET3 Ethernet Controller' if=eth1 drv=vmxnet3
unused=igb_uio,vfio-pci *Active*
Other network devices
=====================
<none>
Note: I downloaded vmxnet3-usermap extension 1.2 source from dpdk.org to work
with VMXNET3.
I got a few compilation errors for the same which I resolved and got the driver
& .so built.
(I belive these errors in driver code were due to its code not being uptodate
wrt linux kernel 3.13).
Next I proceeded to replace the orignial vmxnet3 driver with vmxnet3-usermap.ko
as follows:
cd /home/sandesh/dpdk/vmxnet3-usermap-1.2/
rmmod vmxnet3
insmod vmxnet3-usermap.ko enable_shm=2,2 num_rqs=1,1 num_rxds=512 num_txds=512
and then executed:
../dpdk-1.7.1/build/app/testpmd -c 0xff -n 1 -d librte_pmd_vmxnet3.so -- -i
--rxq=1 --rxd=512 --txd=512
EAL: Detected lcore 0 as core 0 on socket 0
EAL: Detected lcore 1 as core 1 on socket 0
EAL: Detected lcore 2 as core 2 on socket 0
EAL: Detected lcore 3 as core 3 on socket 0
EAL: Detected lcore 4 as core 0 on socket 0
EAL: Detected lcore 5 as core 1 on socket 0
EAL: Detected lcore 6 as core 2 on socket 0
EAL: Detected lcore 7 as core 3 on socket 0
EAL: Support maximum 64 logical core(s) by configuration.
EAL: Detected 8 lcore(s)
EAL: unsupported IOMMU type!
EAL: VFIO support could not be initialized
EAL: Setting up memory...
EAL: Ask a virtual area of 0x5800000 bytes
EAL: Virtual area found at 0x7fb8f7000000 (size = 0x5800000)
EAL: Ask a virtual area of 0x400000 bytes
EAL: Virtual area found at 0x7fb8f6a00000 (size = 0x400000)
EAL: Ask a virtual area of 0x2400000 bytes
EAL: Virtual area found at 0x7fb8f4400000 (size = 0x2400000)
EAL: Requesting 64 pages of size 2MB from socket 0
EAL: TSC frequency is ~2400001 KHz
EAL: open shared lib
/home/sandesh/dpdk/vmxnet3-usermap-1.2/librte_pmd_vmxnet3.so
librte_pmd_vmxnet3 version 1.2
Copyright 2012-2014 6WIND S.A. provided without warranty.
EAL: Master core 0 is ready (tid=fd716880)
EAL: Core 7 is ready (tid=f09ed700)
EAL: Core 6 is ready (tid=f11ee700)
EAL: Core 5 is ready (tid=f19ef700)
EAL: Core 4 is ready (tid=f21f0700)
EAL: Core 3 is ready (tid=f29f1700)
EAL: Core 2 is ready (tid=f31f2700)
EAL: Core 1 is ready (tid=f39f3700)
EAL: PCI device 0000:02:01.0 on NUMA socket -1
EAL: probe driver: 8086:100f rte_em_pmd
EAL: 0000:02:01.0 not managed by UIO driver, skipping
EAL: PCI device 0000:03:00.0 on NUMA socket -1
EAL: probe driver: 15ad:7b0 rte_vmxnet3_pmd
PMD: eth_vmxnet3_dev_init(): No matching device found
EAL: Error - exiting with code: 1
Cause: Requested device 0000:03:00.0 cannot be used
root at pnq1-vm-eng-dhcp13:/home/sandesh/dpdk/vmxnet3-usermap-1.2#
Can someone please point out what I am missing here?
Am I supposed to bind VMXNET3 vNICs to the vmxnet3-usermap driver instead of
igb_uio driver here?
Note: Earlier when I was not running testpmd without vmxnet3-usermap extension,
when I was hitting the following error:
dpdk-1.7.1/build/app/testpmd -c 0xff -n 1 -- -i
.
.
.
EAL: PCI device 0000:0b:00.0 on NUMA socket -1
EAL: probe driver: 15ad:7b0 rte_vmxnet3_pmd
EAL: PCI memory mapped at 0x7f945efe7000
EAL: PCI memory mapped at 0x7f945efe6000
EAL: PCI memory mapped at 0x7f945efe4000
Interactive-mode selected
Configuring Port 0 (socket 0)
Fail to configure port 0 tx queues
EAL: Error - exiting with code: 1
Cause: Start ports failed
Please let me know the correct method to run this application.
Thanks in advance.
Best,
Sandesh
