I also got the segfault. Setting CONFIG_RTE_EAL_UNBIND_PORTS=y is not ideal
because it would also unbind my VMNET3 management interface.
Does anyone know why the crash is happening? According to the stacktrace,
hw_addrX is 0x0 during eth_vmxnet3_dev_init.
I?m running on ubuntu 12.04LTS on ESXi 5.5 with Intel I350 NIC as the physical
adapter for VMNET3.
Program received signal SIGSEGV, Segmentation fault.
0x00000000004fe0eb in vmxnet3_read_addr (addr=0x0) at
/home/dkan/nyansa/3rd-party/dpdk-1.6.0r1/lib/librte_pmd_vmxnet3/vmxnet3_ethdev.h:139
(gdb) up
#1 0x00000000004fe331 in eth_vmxnet3_dev_init (eth_drv=0x799440
<rte_vmxnet3_pmd>, eth_dev=0x7d5280 <rte_eth_devices>) at
/home/dkan/nyansa/3rd-party/dpdk-1.6.0r1/lib/librte_pmd_v\
mxnet3/vmxnet3_ethdev.c:218
(gdb) print *hw
$12 = {hw_addr0 = 0x0, hw_addr1 = 0x0, back = 0x0, device_id = 1968, vendor_id
= 5549, subsystem_device_id = 0, subsystem_vendor_id = 0, adapter_stopped = 0,
perm_addr = "\000\000\\
000\000\000", num_tx_queues = 1 '\001', num_rx_queues = 1 '\001', bufs_per_pkt
= 1 '\001', cur_mtu = 1500, tqd_start = 0x0, rqd_start = 0x0, shared = 0x0,
sharedPA = 0, queueDescPA\
= 0, queue_desc_len = 0, rss_conf = 0x0, rss_confPA = 0, mf_table = 0x0}
Thanks in advance.
Dan
On Mar 10, 2014, at 1:00 AM, Prashant Upadhyaya <prashant.upadhyaya at
aricent.com> wrote:
> Hi Srini,
>
> Thanks, I could also make it work, thanks to your cue !
>
> Now then, this multi-segment not being supported in vmxnet3 driver is a big
> party-pooper for me. Unfortunately in my usecase, I do indeed make heavy use
> of multisegment buffers for sending out the data, so my usecase has failed
> and I will have to fix that.
>
> Also, can you please adivse how much is the max data rates you have been able
> to achieve with one vmxnet3 10G port.
>
> Thanks a lot for the advice once again.
>
> Regards
> -Prashant
>
>
> -----Original Message-----
> From: Srinivasan J [mailto:srinidpdk at gmail.com]
> Sent: Sunday, March 09, 2014 12:38 AM
> To: Prashant Upadhyaya
> Cc: David Marchand; dev at dpdk.org
> Subject: Re: [dpdk-dev] Which vmxnet3 pmd is to be used in dpdk 1.6.x?
>
> Prashant,
> I was also able to hit the issue your hitting using Esxi
> 5.1.0 evaluation and Fedora 20 X86_64 guest. I was able to fix the issue by
> setting CONFIG_RTE_EAL_UNBIND_PORTS=y option in
> defconfig_x86_64-default-linuxapp-gcc configuration file.
>
> Issue seen
> ========
>
> EAL: PCI device 0000:03:00.0 on NUMA socket -1
> EAL: probe driver: 15ad:7b0 rte_vmxnet3_pmd
> EAL: Device is blacklisted, not initializing
> EAL: PCI device 0000:0b:00.0 on NUMA socket -1
> EAL: probe driver: 15ad:7b0 rte_vmxnet3_pmd
>
> Program received signal SIGSEGV, Segmentation fault.
> eth_vmxnet3_dev_init (eth_drv=<optimized out>, eth_dev=0x754480
> <rte_eth_devices>) at
> /root/source/dpdk-1.6.0r1/lib/librte_pmd_vmxnet3/vmxnet3_ethdev.c:218
> 218 ver = VMXNET3_READ_BAR1_REG(hw, VMXNET3_REG_VRRS);
> Missing separate debuginfos, use: debuginfo-install glibc-2.18-11.fc20.x86_64
> (gdb) p hw
> $1 = (struct vmxnet3_hw *) 0x7fffd8fc1040
> (gdb) p *hw
> $2 = {hw_addr0 = 0x0, hw_addr1 = 0x0, back = 0x0, device_id = 1968, vendor_id
> = 5549, subsystem_device_id = 0, subsystem_vendor_id = 0,
> adapter_stopped = 0, perm_addr = "\000\000\000\000\000", num_tx_queues = 1
> '\001', num_rx_queues = 1 '\001', bufs_per_pkt = 1 '\001', cur_mtu = 0,
> tqd_start = 0x0, rqd_start = 0x0, shared = 0x0, sharedPA = 0, queueDescPA =
> 0, queue_desc_len = 0, rss_conf = 0x0, rss_confPA = 0, mf_table = 0x0}
> (gdb)
>
> #define VMXNET3_PCI_BAR1_REG_ADDR(hw, reg) \
> ((volatile uint32_t *)((char *)(hw)->hw_addr1 + (reg))) #define
> VMXNET3_READ_BAR1_REG(hw, reg) \
> vmxnet3_read_addr(VMXNET3_PCI_BAR1_REG_ADDR((hw), (reg)))
> lib/librte_pmd_vmxnet3/vmxnet3_ethdev.h
>
> Issue not seen after enabling CONFIG_RTE_EAL_UNBIND_PORTS=y
> ==================================================
>
> [root at localhost build]# ./l2fwd -c 0xf -b 0000:03:00.0 -n 1 -- -p 0x6
> EAL: Detected lcore 0 as core 0 on socket 0
> EAL: Detected lcore 1 as core 1 on socket 0
> EAL: Detected lcore 2 as core 2 on socket 0
> EAL: Detected lcore 3 as core 3 on socket 0
> EAL: Skip lcore 4 (not detected)
> EAL: Skip lcore 5 (not detected)
> EAL: Skip lcore 6 (not detected)
> EAL: Skip lcore 7 (not detected)
> EAL: Skip lcore 8 (not detected)
> EAL: Skip lcore 9 (not detected)
> EAL: Skip lcore 10 (not detected)
> EAL: Skip lcore 11 (not detected)
> EAL: Skip lcore 12 (not detected)
> EAL: Skip lcore 13 (not detected)
> EAL: Skip lcore 14 (not detected)
> EAL: Skip lcore 15 (not detected)
> EAL: Skip lcore 16 (not detected)
> EAL: Skip lcore 17 (not detected)
> EAL: Skip lcore 18 (not detected)
> EAL: Skip lcore 19 (not detected)
> EAL: Skip lcore 20 (not detected)
> EAL: Skip lcore 21 (not detected)
> EAL: Skip lcore 22 (not detected)
> EAL: Skip lcore 23 (not detected)
> EAL: Skip lcore 24 (not detected)
> EAL: Skip lcore 25 (not detected)
> EAL: Skip lcore 26 (not detected)
> EAL: Skip lcore 27 (not detected)
> EAL: Skip lcore 28 (not detected)
> EAL: Skip lcore 29 (not detected)
> EAL: Skip lcore 30 (not detected)
> EAL: Skip lcore 31 (not detected)
> EAL: Skip lcore 32 (not detected)
> EAL: Skip lcore 33 (not detected)
> EAL: Skip lcore 34 (not detected)
> EAL: Skip lcore 35 (not detected)
> EAL: Skip lcore 36 (not detected)
> EAL: Skip lcore 37 (not detected)
> EAL: Skip lcore 38 (not detected)
> EAL: Skip lcore 39 (not detected)
> EAL: Skip lcore 40 (not detected)
> EAL: Skip lcore 41 (not detected)
> EAL: Skip lcore 42 (not detected)
> EAL: Skip lcore 43 (not detected)
> EAL: Skip lcore 44 (not detected)
> EAL: Skip lcore 45 (not detected)
> EAL: Skip lcore 46 (not detected)
> EAL: Skip lcore 47 (not detected)
> EAL: Skip lcore 48 (not detected)
> EAL: Skip lcore 49 (not detected)
> EAL: Skip lcore 50 (not detected)
> EAL: Skip lcore 51 (not detected)
> EAL: Skip lcore 52 (not detected)
> EAL: Skip lcore 53 (not detected)
> EAL: Skip lcore 54 (not detected)
> EAL: Skip lcore 55 (not detected)
> EAL: Skip lcore 56 (not detected)
> EAL: Skip lcore 57 (not detected)
> EAL: Skip lcore 58 (not detected)
> EAL: Skip lcore 59 (not detected)
> EAL: Skip lcore 60 (not detected)
> EAL: Skip lcore 61 (not detected)
> EAL: Skip lcore 62 (not detected)
> EAL: Skip lcore 63 (not detected)
> EAL: Setting up memory...
> EAL: Ask a virtual area of 0x200000 bytes
> EAL: Virtual area found at 0x7f3a76a00000 (size = 0x200000)
> EAL: Ask a virtual area of 0x7c00000 bytes
> EAL: Virtual area found at 0x7f3a6ec00000 (size = 0x7c00000)
> EAL: Ask a virtual area of 0x200000 bytes
> EAL: Virtual area found at 0x7f3a6e800000 (size = 0x200000)
> EAL: Requesting 64 pages of size 2MB from socket 0
> EAL: TSC frequency is ~2200000 KHz
> EAL: Master core 0 is ready (tid=76cfe880)
> EAL: Core 3 is ready (tid=6cffc700)
> EAL: Core 1 is ready (tid=6dffe700)
> EAL: Core 2 is ready (tid=6d7fd700)
> EAL: PCI device 0000:03:00.0 on NUMA socket -1
> EAL: probe driver: 15ad:7b0 rte_vmxnet3_pmd
> EAL: Device is blacklisted, not initializing
> EAL: PCI device 0000:0b:00.0 on NUMA socket -1
> EAL: probe driver: 15ad:7b0 rte_vmxnet3_pmd
> EAL: PCI memory mapped at 0x7f3a76d10000
> EAL: PCI memory mapped at 0x7f3a76d0f000
> EAL: PCI memory mapped at 0x7f3a76d0d000
> EAL: PCI device 0000:13:00.0 on NUMA socket -1
> EAL: probe driver: 15ad:7b0 rte_vmxnet3_pmd
> EAL: PCI memory mapped at 0x7f3a76d0c000
> EAL: PCI memory mapped at 0x7f3a76d0b000
> EAL: PCI memory mapped at 0x7f3a76d09000
> Notice: odd number of ports in portmask.
> Lcore 0: RX port 1
> Skipping disabled port 0
> Initializing port 1... done:
> Port 1, MAC address: 00:0C:29:F1:9D:A4
>
>
> Checking link statusdone
> Port 1 Link Up - speed 10000 Mbps - full-duplex
> L2FWD: lcore 1 has nothing to do
> L2FWD: lcore 2 has nothing to do
> L2FWD: lcore 3 has nothing to do
> L2FWD: entering main loop on lcore 0
> L2FWD: -- lcoreid=0 portid=1
>
> Port statistics ====================================
> Statistics for port 1 ------------------------------
> Packets sent: 0
> Packets received: 0
> Packets dropped: 0
> Statistics for port 2 ------------------------------
> Packets sent: 0
> Packets received: 0
> Packets dropped: 0
> Aggregate statistics ===============================
> Total packets sent: 0
> Total packets received: 0
> Total packets dropped: 0
> ====================================================
>
>
>
>
> On Fri, Mar 7, 2014 at 10:43 AM, Prashant Upadhyaya <prashant.upadhyaya at
> aricent.com> wrote:
>> Hi Srini,
>>
>> Yes, I am still stuck, I am using ESXi5.1 though and Fedora18 in Guest.
>> Please let me know what is the guest OS distro you are using in the VM. I'll
>> try with that once.
>>
>> Regards
>> -Prashant
>>
>>
>> -----Original Message-----
>> From: Srinivasan J [mailto:srinidpdk at gmail.com]
>> Sent: Friday, March 07, 2014 9:08 AM
>> To: Prashant Upadhyaya
>> Cc: David Marchand; dev at dpdk.org
>> Subject: Re: [dpdk-dev] Which vmxnet3 pmd is to be used in dpdk 1.6.x?
>>
>> Prashant,
>> Are you still hitting a crash? Do you have hugetlbfs
>> enabled? if not try adding enabling hugetlbfs. I was able to get
>> vxnet3 up using the vmxnet3 pmd alone in DPDK 1.60r1. I did not use
>> the vmxnet3-usermap kernel driver. I am using Esxi 5.5
>>
>> Eg: echo 1024 > /proc/sys/vm/nr_hugepages; mkdir /huge; mount -t
>> hugetlbfs nodev /huge;
>>
>> On Thu, Mar 6, 2014 at 12:48 PM, Prashant Upadhyaya <prashant.upadhyaya at
>> aricent.com> wrote:
>>> And if it is any help, here is the startup log --
>>>
>>> EAL: Detected lcore 0 as core 0 on socket 0
>>> EAL: Detected lcore 1 as core 1 on socket 0
>>> EAL: Detected lcore 2 as core 2 on socket 0
>>> EAL: Detected lcore 3 as core 3 on socket 0
>>> EAL: Detected lcore 4 as core 4 on socket 0
>>> EAL: Detected lcore 5 as core 5 on socket 0
>>> EAL: Detected lcore 6 as core 6 on socket 0
>>> EAL: Detected lcore 7 as core 7 on socket 0
>>> EAL: Skip lcore 8 (not detected)
>>> EAL: Skip lcore 9 (not detected)
>>> EAL: Skip lcore 10 (not detected)
>>> EAL: Skip lcore 11 (not detected)
>>> EAL: Skip lcore 12 (not detected)
>>> EAL: Skip lcore 13 (not detected)
>>> EAL: Skip lcore 14 (not detected)
>>> EAL: Skip lcore 15 (not detected)
>>> EAL: Skip lcore 16 (not detected)
>>> EAL: Skip lcore 17 (not detected)
>>> EAL: Skip lcore 18 (not detected)
>>> EAL: Skip lcore 19 (not detected)
>>> EAL: Skip lcore 20 (not detected)
>>> EAL: Skip lcore 21 (not detected)
>>> EAL: Skip lcore 22 (not detected)
>>> EAL: Skip lcore 23 (not detected)
>>> EAL: Skip lcore 24 (not detected)
>>> EAL: Skip lcore 25 (not detected)
>>> EAL: Skip lcore 26 (not detected)
>>> EAL: Skip lcore 27 (not detected)
>>> EAL: Skip lcore 28 (not detected)
>>> EAL: Skip lcore 29 (not detected)
>>> EAL: Skip lcore 30 (not detected)
>>> EAL: Skip lcore 31 (not detected)
>>> EAL: Skip lcore 32 (not detected)
>>> EAL: Skip lcore 33 (not detected)
>>> EAL: Skip lcore 34 (not detected)
>>> EAL: Skip lcore 35 (not detected)
>>> EAL: Skip lcore 36 (not detected)
>>> EAL: Skip lcore 37 (not detected)
>>> EAL: Skip lcore 38 (not detected)
>>> EAL: Skip lcore 39 (not detected)
>>> EAL: Skip lcore 40 (not detected)
>>> EAL: Skip lcore 41 (not detected)
>>> EAL: Skip lcore 42 (not detected)
>>> EAL: Skip lcore 43 (not detected)
>>> EAL: Skip lcore 44 (not detected)
>>> EAL: Skip lcore 45 (not detected)
>>> EAL: Skip lcore 46 (not detected)
>>> EAL: Skip lcore 47 (not detected)
>>> EAL: Skip lcore 48 (not detected)
>>> EAL: Skip lcore 49 (not detected)
>>> EAL: Skip lcore 50 (not detected)
>>> EAL: Skip lcore 51 (not detected)
>>> EAL: Skip lcore 52 (not detected)
>>> EAL: Skip lcore 53 (not detected)
>>> EAL: Skip lcore 54 (not detected)
>>> EAL: Skip lcore 55 (not detected)
>>> EAL: Skip lcore 56 (not detected)
>>> EAL: Skip lcore 57 (not detected)
>>> EAL: Skip lcore 58 (not detected)
>>> EAL: Skip lcore 59 (not detected)
>>> EAL: Skip lcore 60 (not detected)
>>> EAL: Skip lcore 61 (not detected)
>>> EAL: Skip lcore 62 (not detected)
>>> EAL: Skip lcore 63 (not detected)
>>> EAL: Setting up memory...
>>> EAL: Ask a virtual area of 0x80000000 bytes
>>> EAL: Virtual area found at 0x7f848ae00000 (size = 0x80000000)
>>> EAL: Requesting 1024 pages of size 2MB from socket 0
>>> EAL: TSC frequency is ~2000000 KHz
>>> EAL: Master core 0 is ready (tid=b3f3f00)
>>> EAL: Core 1 is ready (tid=8a1f2700)
>>> EAL: Core 2 is ready (tid=899f1700)
>>> EAL: Core 3 is ready (tid=891f0700)
>>> EAL: Core 4 is ready (tid=889ef700)
>>> EAL: Core 5 is ready (tid=7bfff700)
>>> EAL: Core 6 is ready (tid=7b7fe700)
>>> EAL: Core 7 is ready (tid=7affd700)
>>> Pool initialized
>>> Global Variables initialized
>>> PMD: rte_vmxnet3_pmd_init(): >>
>>> EAL: PCI device 0000:0b:00.0 on NUMA socket -1
>>> EAL: probe driver: 15ad:7b0 rte_vmxnet3_pmd
>>> PMD: eth_vmxnet3_dev_init(): >>
>>> Segmentation fault (core dumped)
>>>
>>> -----Original Message-----
>>> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Prashant
>>> Upadhyaya
>>> Sent: Thursday, March 06, 2014 12:20 PM
>>> To: David Marchand
>>> Cc: dev at dpdk.org
>>> Subject: Re: [dpdk-dev] Which vmxnet3 pmd is to be used in dpdk 1.6.x?
>>>
>>> Hi,
>>>
>>> Some further update on the crash I am facing.
>>> I am using DPDK1.6.0r1 and take over the vmxnet3 with igb_uio and
>>> then start the application. (so no external ko or vmxnet usermap
>>> etc.) During the port initializations, the crash is happening in the
>>> following function -
>>>
>>> eth_vmxnet3_dev_init
>>>
>>> and the crash is happening at the following line -
>>> /* Check h/w version compatibility with driver. */
>>> ver = VMXNET3_READ_BAR1_REG(hw, VMXNET3_REG_VRRS);
>>>
>>> Any hints regarding what could be wrong ?
>>>
>>> Regards
>>> -Prashant
>>>
>>>
>>> From: Prashant Upadhyaya
>>> Sent: Wednesday, March 05, 2014 9:01 PM
>>> To: 'David Marchand'
>>> Cc: Srinivasan J; dev at dpdk.org
>>> Subject: RE: [dpdk-dev] Which vmxnet3 pmd is to be used in dpdk 1.6.x?
>>>
>>> Hi David,
>>>
>>> The compilation error with debug flags on was that the functions --
>>> vmxnet3_txq_dump, vmxnet3_rxq_dump - are defined but not used.
>>> Not a serious error, I will try to get rid of the compiler flag which
>>> generates this.
>>>
>>> However, I must reiterate, I _did_ bind my vmxnet3 device with igb_uio (and
>>> I did not use any .so, because I was intending to use the builtin vmxnet3
>>> driver of dpdk 1.6.0r1), the bind succeeded, but then when I started the
>>> application, the dev init for vmxnet3 gave a core dump.
>>> Your patch and solution seems to be suggesting the reverse, i.e. when I
>>> don't bind with igb_uio but try to use the native driver. So please do try
>>> the above combination as well.
>>>
>>> Regards
>>> -Prashant
>>>
>>>
>>> From: David Marchand [mailto:david.marchand at 6wind.com]
>>> Sent: Wednesday, March 05, 2014 8:41 PM
>>> To: Prashant Upadhyaya
>>> Cc: Srinivasan J; dev at dpdk.org<mailto:dev at dpdk.org>
>>> Subject: Re: [dpdk-dev] Which vmxnet3 pmd is to be used in dpdk 1.6.x?
>>>
>>> Hello Prashant,
>>>
>>> On Wed, Mar 5, 2014 at 3:28 PM, Prashant Upadhyaya <prashant.upadhyaya at
>>> aricent.com<mailto:prashant.upadhyaya at aricent.com>> wrote:
>>> Hi,
>>>
>>> I am also keen to know the answer to the question posted by Srini.
>>> The real question is -- is dpdk1.6.0r1 self-sufficient so that I don't need
>>> any extensions etc., or do I still need something from outside like the
>>> usermap kernel driver etc.
>>>
>>> Secondly, if I turn on all the debug options for the vmxnet3 pmd in the
>>> config file, 1.6.0r1 compilation runs into a problem and reports a function
>>> which is defined but not used.
>>>
>>> Can you send your build error ? (maybe in a separate thread ?)
>>>
>>>
>>> I am trying to bring up DPDK inside Fedora18 Guest on ESXi -- when I used
>>> DPDK1.6.0r1 (without debug options turned on for vmxnet3 pmd) the igb_uio
>>> could take over the vmxnet3 NIC but I encountered a core dump in the dev
>>> init function for the vmxnet3 driver -- anybody encountered a similar issue
>>> ?
>>>
>>> I encountered these problems as well.
>>>
>>> - igb_uio module does not check if you disable vmxnet3-uio pmds, it will
>>> always try to take over vmxnet3 devices.
>>> I have a patch waiting in my working dir to cleanly disable vmxnet3-uio pmd.
>>>
>>> - If you don't bind vmxnet3 devices to uio, but forget to enable
>>> vmxnet3-usermap pmd (by specifiying -d librte_pmd_vmxnet3.so), then
>>> internal vmxnet3-uio pmd will try to initialise and crash.
>>> I did not look any deeper into this, the easiest way is to disable
>>> vmxnet3-uio pmd + apply the patch I will send in a few minutes, as a first
>>> workaround.
>>>
>>>
>>> Regards,
>>> --
>>> David Marchand
>>>
>>>
>>>
>>>
>>> =====================================================================
>>> =
>>> ========= Please refer to
>>> http://www.aricent.com/legal/email_disclaimer.html
>>> for important disclosures regarding this electronic communication.
>>> =====================================================================
>>> =
>>> =========
>>>
>>>
>>>
>>>
>>> =====================================================================
>>> =
>>> ========= Please refer to
>>> http://www.aricent.com/legal/email_disclaimer.html
>>> for important disclosures regarding this electronic communication.
>>> =====================================================================
>>> =
>>> =========
>>
>>
>>
>>
>> ======================================================================
>> ========= Please refer to
>> http://www.aricent.com/legal/email_disclaimer.html
>> for important disclosures regarding this electronic communication.
>> ======================================================================
>> =========
>
>
>
>
> ===============================================================================
> Please refer to http://www.aricent.com/legal/email_disclaimer.html
> for important disclosures regarding this electronic communication.
> ===============================================================================
