[dpdk-dev] [PATCH v3 19/25] ether: Fix vlan strip/insert issue

[Ouyang, Changchun]

Hi huawei,

> -Original Message-
> From: Xie, Huawei
> Sent: Wednesday, February 4, 2015 6:54 PM
> To: Ouyang, Changchun; dev at dpdk.org
> Cc: stephen at networkplumber.org; Cao, Waterman; Xu, Qian Q
> Subject: Re: [PATCH v3 19/25] ether: Fix vlan strip/insert issue
> 
> On 1/29/2015 3:24 PM, Ouyang Changchun wrote:
> > Need swap the data from cpu to BE(big endian) for vlan-type.
> >
> > Signed-off-by: Changchun Ouyang 
> > ---
> > lib/librte_ether/rte_ether.h | 4 ++--
> > 1 file changed, 2 insertions(+), 2 deletions(-)
> >
> > diff --git a/lib/librte_ether/rte_ether.h
> > b/lib/librte_ether/rte_ether.h index 74f71c2..0797908 100644
> > --- a/lib/librte_ether/rte_ether.h
> > +++ b/lib/librte_ether/rte_ether.h
> > @@ -351,7 +351,7 @@ static inline int rte_vlan_strip(struct rte_mbuf *m)
> > struct ether_hdr *eh
> >  = rte_pktmbuf_mtod(m, struct ether_hdr *);
> >
> > -if (eh->ether_type != ETHER_TYPE_VLAN)
> > +if (eh->ether_type != rte_cpu_to_be_16(ETHER_TYPE_VLAN))
> rte_be_to_cpu_16?

Eh->ether_type is in network byte order(big endian),
While  ETHER_TYPE_VLAN is host byte order(little endian on x86), so it need 
change into big endian.

> > return -1;
> >
> > struct vlan_hdr *vh = (struct vlan_hdr *)(eh + 1); @@ -401,7
> > +401,7 @@ static inline int rte_vlan_insert(struct rte_mbuf
> > **m)
> > return -ENOSPC;
> >
> > memmove(nh, oh, 2 * ETHER_ADDR_LEN);
> > -nh->ether_type = ETHER_TYPE_VLAN;
> > +nh->ether_type = rte_cpu_to_be_16(ETHER_TYPE_VLAN);
> rte_be_to_cpu_16?

Similar reason as above.

Thanks
Changchun

[dpdk-dev] [PATCH v6 00/13] Port Hotplug Framework

[Tetsuya Mukawa]

On 2015/02/03 22:03, Iremonger, Bernard wrote:
>> -Original Message-
>> From: Tetsuya Mukawa [mailto:mukawa at igel.co.jp]
>> Sent: Sunday, February 1, 2015 4:02 AM
>> To: dev at dpdk.org
>> Cc: Qiu, Michael; Iremonger, Bernard; Tetsuya Mukawa
>> Subject: [PATCH v6 00/13] Port Hotplug Framework
>>
>> This patch series adds a dynamic port hotplug framework to DPDK.
>> With the patches, DPDK apps can attach or detach ports at runtime.
>>
>> The basic concept of the port hotplug is like followings.
>> - DPDK apps must have responsibility to manage ports.
>>   DPDK apps only know which ports are attached or detached at the moment.
>>   The port hotplug framework is implemented to allow DPDK apps to manage 
>> ports.
>>   For example, when DPDK apps call port attach function, attached port number
>>   will be returned. Also DPDK apps can detach port by port number.
>> - Kernel support is needed for attaching or detaching physical device ports.
>>   To attach a new physical device port, the device will be recognized by
>>   userspace directly I/O framework in kernel at first. Then DPDK apps can
>>   call the port hotplug functions to attach ports.
>>   For detaching, steps are vice versa.
>> - Before detach ports, ports must be stopped and closed.
>>   DPDK application must call rte_eth_dev_stop() and rte_eth_dev_close() 
>> before
>>   detaching ports. These function will call finalization codes of PMDs.
>>   But so far, no PMD frees all resources allocated by initialization.
>>   It means PMDs are needed to be fixed to support the port hotplug.
>>   'RTE_PCI_DRV_DETACHABLE' is a new flag indicating a PMD supports detaching.
>>   Without this flag, detaching will be failed.
>> - Mustn't affect legacy DPDK apps.
>>   No DPDK EAL behavior is changed, if the port hotplug functions are't 
>> called.
>>   So all legacy DPDK apps can still work without modifications.
>>
>> And a few limitations.
>> - The port hotplug functions are not thread safe.
>>   DPDK apps should handle it.
>> - Only support Linux and igb_uio so far.
>>   BSD and VFIO is not supported. I will send VFIO patches at least, but I 
>> don't
>>   have a plan to submit BSD patch so far.
>>
>>
>> Here is port hotplug APIs.
>> ---
>> /**
>>  * Attach a new device.
>>  *
>>  * @param devargs
>>  *   A pointer to a strings array describing the new device
>>  *   to be attached. The strings should be a pci address like
>>  *   ':01:00.0' or virtual device name like 'eth_pcap0'.
>>  * @param port_id
>>  *  A pointer to a port identifier actually attached.
>>  * @return
>>  *  0 on success and port_id is filled, negative on error  */ int 
>> rte_eal_dev_attach(const char *devargs,
>> uint8_t *port_id);
>>
>> /**
>>  * Detach a device.
>>  *
>>  * @param port_id
>>  *   The port identifier of the device to detach.
>>  * @param addr
>>  *  A pointer to a device name actually detached.
>>  * @return
>>  *  0 on success and devname is filled, negative on error  */ int 
>> rte_eal_dev_detach(uint8_t port_id,
>> char *devname);
>> ---
>>
>> This patch series are for DPDK EAL. To use port hotplug function by DPDK 
>> apps, each PMD should be
>> fixed to support 'RTE_PCI_DRV_DETACHABLE' flag. Please check a patch for 
>> pcap PMD.
>>
>> Also please check testpmd patch. It will show you how to fix your legacy 
>> applications to support port
>> hotplug feature.
>>
>> PATCH v6 changes
>>  - Fix rte_eth_dev_uninit() to handle a return value of uninit
>>function of PMD. To do this, below changes also be applied.
>>- Fix a paramter of rte_eth_dev_free().
>>- Use rte_eth_dev structure as the paramter of rte_eth_dev_free().
>>
>> PATCH v5 changes
>>  - Add runtime check passthrough driver type, like vfio-pci, igb_uio
>>and uio_pci_generic.
>>This was done by Qiu, Michael. Thanks a lot.
>>  - Change function names like below.
>>- rte_eal_dev_find_and_invoke() to rte_eal_vdev_find_and_invoke().
>>- rte_eal_dev_invoke() to rte_eal_vdev_invoke().
>>  - Add code to handle a return value of rte_eal_devargs_remove().
>>  - Fix pci address format in rte_eal_dev_detach().
>>  - Remove RTE_EAL_INVOKE_TYPE_UNKNOWN, because it's unused.
>>  - Change function definition of rte_eal_devargs_remove().
>>  - Fix pci_unmap_device() to check pt_driver.
>>  - Fix return value of below functions.
>>- rte_eth_dev_get_changed_port().
>>- rte_eth_dev_get_port_by_addr().
>>  - Change paramters of rte_eth_dev_validate_port() to cleanup code.
>>  - Fix pci_scan_one to handle pt_driver correctly.
>>(Thanks to Qiu, Michael for above sugesstions)
>>
>> PATCH v4 changes
>>  - Merge patches to review easier.
>>  - Fix indent of 'if' statement.
>>  - Fix calculation method of eal_compare_pci_addr().
>>  - Fix header file declaration.
>>  - Add header file to determine if hotplug can be enabled.
>>(Tha

[dpdk-dev] [PATCH v6 00/13] Port Hotplug Framework

[Tetsuya Mukawa]

On 2015/02/04 3:35, Iremonger, Bernard wrote:
>
>> -Original Message-
>> From: Tetsuya Mukawa [mailto:mukawa at igel.co.jp]
>> Sent: Sunday, February 1, 2015 4:02 AM
>> To: dev at dpdk.org
>> Cc: Qiu, Michael; Iremonger, Bernard; Tetsuya Mukawa
>> Subject: [PATCH v6 00/13] Port Hotplug Framework
>>
>> This patch series adds a dynamic port hotplug framework to DPDK.
>> With the patches, DPDK apps can attach or detach ports at runtime.
>>
>> The basic concept of the port hotplug is like followings.
>> - DPDK apps must have responsibility to manage ports.
>>   DPDK apps only know which ports are attached or detached at the moment.
>>   The port hotplug framework is implemented to allow DPDK apps to manage 
>> ports.
>>   For example, when DPDK apps call port attach function, attached port number
>>   will be returned. Also DPDK apps can detach port by port number.
>> - Kernel support is needed for attaching or detaching physical device ports.
>>   To attach a new physical device port, the device will be recognized by
>>   userspace directly I/O framework in kernel at first. Then DPDK apps can
>>   call the port hotplug functions to attach ports.
>>   For detaching, steps are vice versa.
>> - Before detach ports, ports must be stopped and closed.
>>   DPDK application must call rte_eth_dev_stop() and rte_eth_dev_close() 
>> before
>>   detaching ports. These function will call finalization codes of PMDs.
>>   But so far, no PMD frees all resources allocated by initialization.
>>   It means PMDs are needed to be fixed to support the port hotplug.
>>   'RTE_PCI_DRV_DETACHABLE' is a new flag indicating a PMD supports detaching.
>>   Without this flag, detaching will be failed.
>> - Mustn't affect legacy DPDK apps.
>>   No DPDK EAL behavior is changed, if the port hotplug functions are't 
>> called.
>>   So all legacy DPDK apps can still work without modifications.
>>
>> And a few limitations.
>> - The port hotplug functions are not thread safe.
>>   DPDK apps should handle it.
>> - Only support Linux and igb_uio so far.
>>   BSD and VFIO is not supported. I will send VFIO patches at least, but I 
>> don't
>>   have a plan to submit BSD patch so far.
>>
>>
>> Here is port hotplug APIs.
>> ---
>> /**
>>  * Attach a new device.
>>  *
>>  * @param devargs
>>  *   A pointer to a strings array describing the new device
>>  *   to be attached. The strings should be a pci address like
>>  *   ':01:00.0' or virtual device name like 'eth_pcap0'.
>>  * @param port_id
>>  *  A pointer to a port identifier actually attached.
>>  * @return
>>  *  0 on success and port_id is filled, negative on error  */ int 
>> rte_eal_dev_attach(const char *devargs,
>> uint8_t *port_id);
>>
>> /**
>>  * Detach a device.
>>  *
>>  * @param port_id
>>  *   The port identifier of the device to detach.
>>  * @param addr
>>  *  A pointer to a device name actually detached.
>>  * @return
>>  *  0 on success and devname is filled, negative on error  */ int 
>> rte_eal_dev_detach(uint8_t port_id,
>> char *devname);
>> ---
>>
>> This patch series are for DPDK EAL. To use port hotplug function by DPDK 
>> apps, each PMD should be
>> fixed to support 'RTE_PCI_DRV_DETACHABLE' flag. Please check a patch for 
>> pcap PMD.
>>
>> Also please check testpmd patch. It will show you how to fix your legacy 
>> applications to support port
>> hotplug feature.
>>
>> PATCH v6 changes
>>  - Fix rte_eth_dev_uninit() to handle a return value of uninit
>>function of PMD. To do this, below changes also be applied.
>>- Fix a paramter of rte_eth_dev_free().
>>- Use rte_eth_dev structure as the paramter of rte_eth_dev_free().
>>
>> PATCH v5 changes
>>  - Add runtime check passthrough driver type, like vfio-pci, igb_uio
>>and uio_pci_generic.
>>This was done by Qiu, Michael. Thanks a lot.
>>  - Change function names like below.
>>- rte_eal_dev_find_and_invoke() to rte_eal_vdev_find_and_invoke().
>>- rte_eal_dev_invoke() to rte_eal_vdev_invoke().
>>  - Add code to handle a return value of rte_eal_devargs_remove().
>>  - Fix pci address format in rte_eal_dev_detach().
>>  - Remove RTE_EAL_INVOKE_TYPE_UNKNOWN, because it's unused.
>>  - Change function definition of rte_eal_devargs_remove().
>>  - Fix pci_unmap_device() to check pt_driver.
>>  - Fix return value of below functions.
>>- rte_eth_dev_get_changed_port().
>>- rte_eth_dev_get_port_by_addr().
>>  - Change paramters of rte_eth_dev_validate_port() to cleanup code.
>>  - Fix pci_scan_one to handle pt_driver correctly.
>>(Thanks to Qiu, Michael for above sugesstions)
>>
>> PATCH v4 changes
>>  - Merge patches to review easier.
>>  - Fix indent of 'if' statement.
>>  - Fix calculation method of eal_compare_pci_addr().
>>  - Fix header file declaration.
>>  - Add header file to determine if hotplug can be enabled.
>>(Th

[dpdk-dev] [PATCH v6] testpmd: Add port hotplug support

[Tetsuya Mukawa]

On 2015/02/04 10:44, Qiu, Michael wrote:
> On 2/3/2015 6:30 PM, Tetsuya Mukawa wrote:
>> On 2015/02/03 18:14, Qiu, Michael wrote:
>>> On 2/3/2015 2:16 PM, Qiu, Michael wrote:
 On 2/1/2015 12:02 PM, Tetsuya Mukawa wrote:
> The patch introduces following commands.
> - port attach [ident]
> - port detach [port_id]
>  - attach: attaching a port
>  - detach: detaching a port
>  - ident: pci address of physical device.
>   Or device name and paramerters of virtual device.
>  (ex. :02:00.0, eth_pcap0,iface=eth0)
>  - port_id: port identifier
>
> v5:
> - Add testpmd documentation.
>   (Thanks to Iremonger, Bernard)
> v4:
>  - Fix strings of command help.
>
> Signed-off-by: Tetsuya Mukawa 
>>> [...]
>>>
> +static int
> +port_is_closed(portid_t port_id)
> +{
> + if (port_id_is_invalid(port_id, ENABLED_WARN))
> + return 0;
> +
> + if (ports[port_id].port_status != RTE_PORT_CLOSED)
> + return 0;
> +
> + return 1;
> +}
> +
> +int
>  start_port(portid_t pid)
>  {
>   int diag, need_check_link_status = 0;
> @@ -1296,8 +1347,8 @@ start_port(portid_t pid)
>  
>   if(dcb_config)
>   dcb_test = 1;
> - for (pi = 0; pi < nb_ports; pi++) {
> - if (pid < nb_ports && pid != pi)
> + FOREACH_PORT(pi, ports) {
> + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi)
 Here may it be:

 if (!port_id_is_invalid(pid, DISABLED_WARN) && (pid != pi || pid == 
 RET_PORT_ALL))
>>> Sorry, should be:
>>>
>>> if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi && pid != 
>>> (portid_t)RET_PORT_ALL)
>>>
>>> Otherwise, should check for "RET_PORT_ALL" in function port_id_is_invalid()
>> Thanks for comment. I've found 2 issues.
>> (I guess the original code has same issue.)
> Original code may not have this issue, cause RET_PORT_ALL is always
> greater than nb_ports, so "continue" will not exec. The logic may be 
> right, but it is a little hard to understand.
>
>> One is that "port_id_is_invalid" should receives "pi" instead of "pid".
>> The other is if statement is wrong as you said.
>>
>> I guess following statement will be good.
>>
>> if (port_id_is_invalid(pi, DISABLED_WARN) || (pid != pi && pid !=
>> (portid_t)RTE_PORT_ALL))
> Actually, "port_id_is_invalid(pi, DISABLED_WARN)" could be removed as
> "FOREACH_PORT" will find a valid port.

Good point!

> So it could be:
>
> if (pid != pi && pid != (portid_t)RTE_PORT_ALL)
>
> What do you think?

I agree with you.
I will change like above.

Thanks,
Tetsuya

> Thanks,
> Michael
>> How about it?
>>
>> Thanks,
>> Tetsuya
>>
>>
>>> Thanks,
>>> Michael
>>>
 Otherwise no port will be start by default.


 Thanks,
 Michael

>   continue;
>  
>   port = &ports[pi];
> @@ -1421,7 +1472,7 @@ start_port(portid_t pid)
>   }
>  
>   if (need_check_link_status && !no_link_check)
> - check_all_ports_link_status(nb_ports, RTE_PORT_ALL);
> + check_all_ports_link_status(RTE_PORT_ALL);
>   else
>   printf("Please stop the ports first\n");
>  
> @@ -1446,8 +1497,8 @@ stop_port(portid_t pid)
>   }
>   printf("Stopping ports...\n");
>  
> - for (pi = 0; pi < nb_ports; pi++) {
> - if (pid < nb_ports && pid != pi)
> + FOREACH_PORT(pi, ports) {
> + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi)
>   continue;
>  
>   port = &ports[pi];
> @@ -1463,7 +1514,7 @@ stop_port(portid_t pid)
>   need_check_link_status = 1;
>   }
>   if (need_check_link_status && !no_link_check)
> - check_all_ports_link_status(nb_ports, RTE_PORT_ALL);
> + check_all_ports_link_status(RTE_PORT_ALL);
>  
>   printf("Done\n");
>  }
> @@ -1481,8 +1532,8 @@ close_port(portid_t pid)
>  
>   printf("Closing ports...\n");
>  
> - for (pi = 0; pi < nb_ports; pi++) {
> - if (pid < nb_ports && pid != pi)
> + FOREACH_PORT(pi, ports) {
> + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi)
>   continue;
>  
>   port = &ports[pi];
> @@ -1502,31 +1553,83 @@ close_port(portid_t pid)
>   printf("Done\n");
>  }
>  
> -int
> -all_ports_stopped(void)
> +void
> +attach_port(char *identifier)
>  {
> - portid_t pi;
> - struct rte_port *port;
> + portid_t i, j, pi = 0;
>  
> - for (pi = 0; pi < nb_ports; pi++) {
> - port = &ports[pi];
> - if (port->port_status != RTE_PORT_STOPPED)
> - return 0;
> + printf("Attaching a new port...\n");
> +
> + if (identifier == NULL) {
> + printf("Invalid parameters are speficied\n");
> + return;
>

[dpdk-dev] [PATCH] maintainer: claim review for virtio/vhost

[Ouyang Changchun]

I will be a volunteer of reviewing the following files:
   lib/librte_pmd_virtio/
   doc/guides/prog_guide/poll_mode_drv_emulated_virtio_nic.rst
   lib/librte_vhost/
   doc/guides/prog_guide/vhost_lib.rst
   examples/vhost/
   doc/guides/sample_app_ug/vhost.rst

Signed-off-by: Changchun Ouyang 
---
 MAINTAINERS | 1 +
 1 file changed, 1 insertion(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index 9a63714..b4f1c1c 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -198,6 +198,7 @@ Intel i40e
 F: lib/librte_pmd_i40e/

 RedHat virtio
+M: Changchun Ouyang 
 F: lib/librte_pmd_virtio/
 F: doc/guides/prog_guide/poll_mode_drv_emulated_virtio_nic.rst
 F: lib/librte_vhost/
-- 
1.8.4.2

[dpdk-dev] Bond feature can't running well on 40G i40e NICs

[lhffjzh]

Hi DPDK maintainers,



We encounter confusion recently which is 40G i40e NICs bond issue, our
bonding logic as below, the ixgbe bonding is running well both 1.7.1 and
1.8.0, but i40e are all failed.



Create bond port -> config slave ports(tx/rx) -> start slave ports -> add
slave port into bond port -> start bond port -> tx/rx on bond port



On 1.7.1, the 40G bonding port always can't be started, them status are down
and speed are 0.

On 1.8.0, the 40G bonding port start and send network package correctly, but
can't receive any package with BONDING_MODE_BALANCE
  mode and all policies like BALANCE_XMIT_POLICY_LAYER2. In
BONDING_MODE_ROUND_ROBIN
  mode, bonding port can send/receive a few network packages(less
than 100) per seconds.



Could you please help us for this? any clue is welcome.





Thanks and Regards,

Haifeng

[dpdk-dev] [PATCH] maintainer: claim review for Xen Dom0 support

[Jijiang Liu]

Signed-off-by: Jijiang Liu 
---
 MAINTAINERS |1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/MAINTAINERS b/MAINTAINERS
index 9a63714..206783d 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -110,6 +110,7 @@ Linux VFIO
 F: lib/librte_eal/linuxapp/eal/*vfio*

 Linux Xen
+M: Jijiang Liu 
 F: lib/librte_eal/linuxapp/xen_dom0/
 F: lib/librte_eal/linuxapp/eal/*xen*
 F: lib/librte_eal/linuxapp/eal/include/exec-env/rte_dom0_common.h
-- 
1.7.7.6

[dpdk-dev] [PATCH] maintainer: claim review for Xen Dom0 support

[Jijiang Liu]

I will be a volunteer of reviewing the following files:
   lib/librte_eal/linuxapp/xen_dom0/*
   lib/librte_eal/linuxapp/eal/eal_xen_memory.c
   lib/librte_eal/linuxapp/eal/include/exec-env/rte_dom0_common.h

Signed-off-by: Jijiang Liu 
---
 MAINTAINERS |1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/MAINTAINERS b/MAINTAINERS
index 9a63714..206783d 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -110,6 +110,7 @@ Linux VFIO
 F: lib/librte_eal/linuxapp/eal/*vfio*

 Linux Xen
+M: Jijiang Liu 
 F: lib/librte_eal/linuxapp/xen_dom0/
 F: lib/librte_eal/linuxapp/eal/*xen*
 F: lib/librte_eal/linuxapp/eal/include/exec-env/rte_dom0_common.h
-- 
1.7.7.6

[dpdk-dev] [PATCH] maintainer: claim review for Xen Dom0 support

[Thomas Monjalon]

Hi Jijiang,

2015-02-05 10:31, Jijiang Liu:
> I will be a volunteer of reviewing the following files:
>lib/librte_eal/linuxapp/xen_dom0/*
>lib/librte_eal/linuxapp/eal/eal_xen_memory.c
>lib/librte_eal/linuxapp/eal/include/exec-env/rte_dom0_common.h

Actually, the section that you are subscribing is wider:
F: lib/librte_eal/linuxapp/xen_dom0/
F: lib/librte_eal/linuxapp/eal/*xen*
F: lib/librte_eal/linuxapp/eal/include/exec-env/rte_dom0_common.h
F: lib/librte_mempool/rte_dom0_mempool.c
F: lib/librte_pmd_xenvirt/
F: doc/guides/xen/
F: app/test-pmd/mempool_*
F: examples/vhost_xen/

>  Linux Xen
> +M: Jijiang Liu 

Was your intention to be responsible of the whole Xen area or
only of EAL for Xen?

-- 
Thomas

[dpdk-dev] [PATCH] maintainer: claim review for Xen Dom0 support

[Liu, Jijiang]

Hi Thomas,


> -Original Message-
> From: Thomas Monjalon [mailto:thomas.monjalon at 6wind.com]
> Sent: Thursday, February 05, 2015 4:25 PM
> To: Liu, Jijiang
> Cc: dev at dpdk.org
> Subject: Re: [dpdk-dev] [PATCH] maintainer: claim review for Xen Dom0
> support
> 
> Hi Jijiang,
> 
> 2015-02-05 10:31, Jijiang Liu:
> > I will be a volunteer of reviewing the following files:
> >lib/librte_eal/linuxapp/xen_dom0/*
> >lib/librte_eal/linuxapp/eal/eal_xen_memory.c
> >lib/librte_eal/linuxapp/eal/include/exec-env/rte_dom0_common.h
> 
> Actually, the section that you are subscribing is wider:
> F: lib/librte_eal/linuxapp/xen_dom0/
> F: lib/librte_eal/linuxapp/eal/*xen*
> F: lib/librte_eal/linuxapp/eal/include/exec-env/rte_dom0_common.h
> F: lib/librte_mempool/rte_dom0_mempool.c
> F: lib/librte_pmd_xenvirt/
> F: doc/guides/xen/
> F: app/test-pmd/mempool_*
> F: examples/vhost_xen/
> 
> >  Linux Xen
> > +M: Jijiang Liu 
> 
> Was your intention to be responsible of the whole Xen area or only of EAL for
> Xen?

I just talked this with Xie Huawei, and I know that he will be a volunteer for 
the following files.
 F: lib/librte_pmd_xenvirt/
 F: doc/guides/xen/
 F: app/test-pmd/mempool_*
 F: examples/vhost_xen/

so I will be responsible for the following files:
 F: lib/librte_eal/linuxapp/xen_dom0/
F: lib/librte_eal/linuxapp/eal/*xen*
 F: lib/librte_eal/linuxapp/eal/include/exec-env/rte_dom0_common.h
 F: lib/librte_mempool/rte_dom0_mempool.c


> --
> Thomas

[dpdk-dev] [PATCH v1] ixgbe: fix link issue in loopback mode

[Cunming Liang]

In loopback mode, it's expected force link up even when there's no cable 
connect.
But in codes, setup_sfp() rewrites the related register.
It causes in the case 'multispeed_fiber', it can't link up without cable 
connect. 

Signed-off-by: Cunming Liang 
---
 lib/librte_pmd_ixgbe/ixgbe_ethdev.c | 10 +-
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/lib/librte_pmd_ixgbe/ixgbe_ethdev.c 
b/lib/librte_pmd_ixgbe/ixgbe_ethdev.c
index b341dd0..d6d408e 100644
--- a/lib/librte_pmd_ixgbe/ixgbe_ethdev.c
+++ b/lib/librte_pmd_ixgbe/ixgbe_ethdev.c
@@ -1501,6 +1501,11 @@ ixgbe_dev_start(struct rte_eth_dev *dev)
goto error;
}

+   /* Skip link setup if loopback mode is enabled for 82599. */
+   if (hw->mac.type == ixgbe_mac_82599EB &&
+   dev->data->dev_conf.lpbk_mode == IXGBE_LPBK_82599_TX_RX)
+   goto skip_link_setup;
+
if (ixgbe_is_sfp(hw) && hw->phy.multispeed_fiber) {
err = hw->mac.ops.setup_sfp(hw);
if (err)
@@ -1510,11 +1515,6 @@ ixgbe_dev_start(struct rte_eth_dev *dev)
/* Turn on the laser */
ixgbe_enable_tx_laser(hw);

-   /* Skip link setup if loopback mode is enabled for 82599. */
-   if (hw->mac.type == ixgbe_mac_82599EB &&
-   dev->data->dev_conf.lpbk_mode == IXGBE_LPBK_82599_TX_RX)
-   goto skip_link_setup;
-
err = ixgbe_check_link(hw, &speed, &link_up, 0);
if (err)
goto error;
-- 
1.8.1.4

[dpdk-dev] [PATCH] fix testpmd show port info error

[Xuelin Shi]

Hi,

Anybody interested in this patch could have a review or comment on it?
I'm new here. Should I send this patch to some specific maintainer to make this 
more efficient?

Thanks,
Shi Xuelin

-Original Message-
From: xuelin.shi at freescale.com [mailto:xuelin@freescale.com] 
Sent: Monday, February 02, 2015 14:52
To: dev at dpdk.org
Cc: Shi Xuelin-B29237
Subject: [PATCH] fix testpmd show port info error

From: Xuelin Shi 

the port number type should be consistent with librte_cmdline, else there is 
potential endian issue.

Signed-off-by: Xuelin Shi 
---
 app/test-pmd/cmdline.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c index 
4beb404..488ac63 100644
--- a/app/test-pmd/cmdline.c
+++ b/app/test-pmd/cmdline.c
@@ -5568,7 +5568,7 @@ cmdline_parse_token_string_t cmd_showport_what =
TOKEN_STRING_INITIALIZER(struct cmd_showport_result, what,
 "info#stats#xstats#fdir#stat_qmap");
 cmdline_parse_token_num_t cmd_showport_portnum =
-   TOKEN_NUM_INITIALIZER(struct cmd_showport_result, portnum, INT32);
+   TOKEN_NUM_INITIALIZER(struct cmd_showport_result, portnum, INT8);

 cmdline_parse_inst_t cmd_showport = {
.f = cmd_showport_parsed,
--
1.9.1

[dpdk-dev] 答复: mmap failed: Cannot allocate memory when init dpdk eal

[zhangsha (A)]

Hi, haifeng
Dpdk will mmap all the hugepages (4096) of host in the first step. Then it sort 
them and unmap the hugepages unneeded. My demo got the mistake when the 1st 
step. 
Thanks!


--
???: Linhaifeng 
: 2015?1?30? 20:24
???: zhangsha (A); dev at dpdk.org
??: Re: [dpdk-dev] mmap failed: Cannot allocate memory when init dpdk eal



On 2015/1/30 19:40, zhangsha (A) wrote:
> Hi ?all
> 
> I am suffering from the problem mmap failed as followed when init dpdk eal.
> 
> Fri Jan 30 09:03:29 2015:EAL: Setting up memory...
> Fri Jan 30 09:03:34 2015:EAL: map_all_hugepages(): mmap failed: Cannot 
> allocate memory Fri Jan 30 09:03:34 2015:EAL: Failed to mmap 2 MB 
> hugepages Fri Jan 30 09:03:34 2015:EAL: Cannot init memory
> 
> Before I run the demo, the free hugepages of my host is :
> 
> cat /proc/meminfo
> MemTotal:   132117056 kB
> MemFree:122040292 kB
> Buffers:   10984 kB
> Cached:   123056 kB
> SwapCached:0 kB
> Active:   120812 kB
> Inactive:  85860 kB
> Active(anon):  79488 kB
> Inactive(anon):  364 kB
> Active(file):  41324 kB
> Inactive(file):85496 kB
> Unevictable:   23576 kB
> Mlocked:   23576 kB
> SwapTotal: 0 kB
> SwapFree:  0 kB
> Dirty:  2576 kB
> Writeback: 0 kB
> AnonPages: 96236 kB
> Mapped:19936 kB
> Shmem:   552 kB
> Slab: 101344 kB
> SReclaimable:  24164 kB
> SUnreclaim:77180 kB
> KernelStack:2544 kB
> PageTables: 4180 kB
> NFS_Unstable:  0 kB
> Bounce:0 kB
> WritebackTmp:  0 kB
> CommitLimit:61864224 kB
> Committed_AS: 585844 kB
> VmallocTotal:   34359738367 kB
> VmallocUsed:  518656 kB
> VmallocChunk:   34292133264 kB
> HardwareCorrupted: 0 kB
> AnonHugePages:  4096 kB
> HugePages_Total:4096
> HugePages_Free: 4096
> HugePages_Rsvd:0
> HugePages_Surp:0
> Hugepagesize:   2048 kB
> DirectMap4k:   96256 kB
> DirectMap2M: 6178816 kB
> DirectMap1G:127926272 kB
> 
> And after the demo executed, I got the hugepages like this:
> 
> cat /proc/meminfo
> MemTotal:   132117056 kB
> MemFree:117325180 kB
> Buffers:   33508 kB
> Cached:   721912 kB
> SwapCached:0 kB
> Active:  4217712 kB
> Inactive: 540956 kB
> Active(anon):4019068 kB
> Inactive(anon):   121136 kB
> Active(file): 198644 kB
> Inactive(file):   419820 kB
> Unevictable:   23908 kB
> Mlocked:   23908 kB
> SwapTotal: 0 kB
> SwapFree:  0 kB
> Dirty:  2856 kB
> Writeback: 0 kB
> AnonPages:   4035184 kB
> Mapped:   160292 kB
> Shmem:122100 kB
> Slab: 177908 kB
> SReclaimable:  64808 kB
> SUnreclaim:   113100 kB
> KernelStack:7560 kB
> PageTables:62128 kB
> NFS_Unstable:  0 kB
> Bounce:0 kB
> WritebackTmp:  0 kB
> CommitLimit:61864224 kB
> Committed_AS:8789664 kB
> VmallocTotal:   34359738367 kB
> VmallocUsed:  527296 kB
> VmallocChunk:   34292122604 kB
> HardwareCorrupted: 0 kB
> AnonHugePages:262144 kB
> HugePages_Total:4096
> HugePages_Free: 2048
> HugePages_Rsvd:0
> HugePages_Surp:0
> Hugepagesize:   2048 kB
> DirectMap4k:  141312 kB
> DirectMap2M: 9279488 kB
> DirectMap1G:124780544 kB
> 
> Only the hugepages beyond to node1 was mapped. I was told host(having 64bit 
> OS) cannot allocate memory while node0 has 2048 free hugepages,why?
> Dose anyone encountered the similar problem ever?
> Any response will be appreciated!
> Thanks!
> 
> 
> 
> 

How do you tell kernel not to allocate memory on node0?

I guess node0 and node1 both have 2048 hugepages and you want to mmap 4096 
hugepages.
So you can mmap 2048 hugepages on node1.After this step you cannot mmap any 
hugepage files because you tell kernel not to allocate memory on node0.


--
Regards,
Haifeng

[dpdk-dev] [PATCH v2 2/7] rte_sched: expand scheduler hierarchy for more VLAN's

[Ananyev, Konstantin]

> -Original Message-
> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Stephen Hemminger
> Sent: Thursday, February 05, 2015 6:05 AM
> To: dev at dpdk.org
> Cc: Stephen Hemminger
> Subject: [dpdk-dev] [PATCH v2 2/7] rte_sched: expand scheduler hierarchy for 
> more VLAN's
> 
> From: Stephen Hemminger 
> 
> The QoS subport is limited to 8 bits in original code.
> But customers demanded ability to support full number of VLAN's (4096)
> therefore use the full part of the tag field of mbuf.
> 
> Resize the pipe as well to allow for more pipes in future and
> avoid expensive bitfield access.
> 
> Signed-off-by: Stephen Hemminger 
> ---
> v2 use tag area rather than claiming reserved bit which isn't documented
> 
>  lib/librte_mbuf/rte_mbuf.h   |  5 -
>  lib/librte_sched/rte_sched.h | 38 --
>  2 files changed, 28 insertions(+), 15 deletions(-)

Acked-by: Konstantin Ananyev 

[dpdk-dev] [PATCH] fix testpmd show port info error

[Olivier MATZ]

Hi,

On 02/05/2015 10:26 AM, Xuelin Shi wrote:
> Hi,
>
> Anybody interested in this patch could have a review or comment on it?
> I'm new here. Should I send this patch to some specific maintainer to make 
> this more efficient?
>
> Thanks,
> Shi Xuelin
>
> -Original Message-
> From: xuelin.shi at freescale.com [mailto:xuelin.shi at freescale.com]
> Sent: Monday, February 02, 2015 14:52
> To: dev at dpdk.org
> Cc: Shi Xuelin-B29237
> Subject: [PATCH] fix testpmd show port info error
>
> From: Xuelin Shi 
>
> the port number type should be consistent with librte_cmdline, else there is 
> potential endian issue.
>
> Signed-off-by: Xuelin Shi 
> ---
>   app/test-pmd/cmdline.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c index 
> 4beb404..488ac63 100644
> --- a/app/test-pmd/cmdline.c
> +++ b/app/test-pmd/cmdline.c
> @@ -5568,7 +5568,7 @@ cmdline_parse_token_string_t cmd_showport_what =
>   TOKEN_STRING_INITIALIZER(struct cmd_showport_result, what,
>"info#stats#xstats#fdir#stat_qmap");
>   cmdline_parse_token_num_t cmd_showport_portnum =
> - TOKEN_NUM_INITIALIZER(struct cmd_showport_result, portnum, INT32);
> + TOKEN_NUM_INITIALIZER(struct cmd_showport_result, portnum, INT8);
>
>   cmdline_parse_inst_t cmd_showport = {
>   .f = cmd_showport_parsed,
> --

Thank you for reporting this. However I think it should be UINT8
instead of INT8.

According to http://dpdk.org/dev, I think a better title could be
"testpmd: fix port parsing in show port info command"

Apart from this,
Acked-by: Olivier Matz 

I think it would be less work for Thomas if you can submit
a v2 that includes these 2 changes.

Regards,
Olivier

[dpdk-dev] [PATCH net-next v2] hyper-v: allow access to vmbus from userspace driver

[Vitaly Kuznetsov]

Stephen Hemminger  writes:

> This is enables the hyper-v driver for DPDK .
> The hv_uio driver needs to access the shared vmbus monitor
> pages.

Why can't we just make vmbus_connection struct public?

>
> I would also like to put hv_uio in upstream kernel like other
> uio drivers, but need to get API accepted with DPDK first.
>
> Signed-off-by: Stas Egorov 
> Signed-off-by: Stephen Hemminger 
>
> ---
> v2 - simplify and rename to vmbus_get_monitor_pages
>
>  drivers/hv/connection.c |   20 +---
>  include/linux/hyperv.h  |3 +++
>  2 files changed, 20 insertions(+), 3 deletions(-)
>
> --- a/drivers/hv/connection.c 2015-02-03 10:58:51.751752450 -0800
> +++ b/drivers/hv/connection.c 2015-02-04 14:59:51.636194383 -0800
> @@ -64,6 +64,15 @@ static __u32 vmbus_get_next_version(__u3
>   }
>  }
>
> +void vmbus_get_monitor_pages(unsigned long *int_page,
> +  unsigned long monitor_pages[2])
> +{

Too weird to be a public interface in my opinion.

> + *int_page = (unsigned long)vmbus_connection.int_page;
> + monitor_pages[0] = (unsigned long)vmbus_connection.monitor_pages[0];
> + monitor_pages[1] = (unsigned long)vmbus_connection.monitor_pages[1];
> +}
> +EXPORT_SYMBOL_GPL(vmbus_get_monitor_pages);
> +
>  static int vmbus_negotiate_version(struct vmbus_channel_msginfo *msginfo,
>   __u32 version)
>  {
> @@ -347,10 +356,7 @@ static void process_chn_event(u32 relid)
>   else
>   bytes_to_read = 0;
>   } while (read_state && (bytes_to_read != 0));
> - } else {
> - pr_err("no channel callback for relid - %u\n", relid);
>   }
> -
>  }

This change wasn't mentioned in your commit msg.

>
>  /*
> --- a/include/linux/hyperv.h  2015-02-03 10:58:51.751752450 -0800
> +++ b/include/linux/hyperv.h  2015-02-04 15:00:26.388355012 -0800
> @@ -868,6 +868,9 @@ extern int vmbus_recvpacket_raw(struct v
>
>  extern void vmbus_ontimer(unsigned long data);
>
> +extern void vmbus_get_monitor_pages(unsigned long *int_page,
> + unsigned long monitor_pages[2]);
> +
>  /* Base driver object */
>  struct hv_driver {
>   const char *name;
> ___
> devel mailing list
> devel at linuxdriverproject.org
> http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

-- 
  Vitaly

[dpdk-dev] mmap fails with more than 40000 hugepages

[Damjan Marion (damarion)]

Hi,

I have system with 2 NUMA nodes and 256G RAM total. I noticed that DPDK crashes 
in rte_eal_init()
when number of available hugepages is around 4 or above.
Everything works fine with lower values (i.e. 3).

I also tried with allocating 4 on node0 and 0 on node1, same crash happens.


Any idea what might be causing this?

Thanks,

Damjan


$ cat /sys/devices/system/node/node[01]/hugepages/hugepages-2048kB/nr_hugepages
2
2

$ grep -i huge /proc/meminfo
AnonHugePages:706560 kB
HugePages_Total:   4
HugePages_Free:4
HugePages_Rsvd:0
HugePages_Surp:0
Hugepagesize:   2048 kB


$ sudo ~/src/dpdk/x86_64-native-linuxapp-gcc/app/testpmd -l 5-7 -n 3 
--socket-mem 512,512
EAL: Detected lcore 0 as core 0 on socket 0
EAL: Detected lcore 1 as core 1 on socket 0
EAL: Detected lcore 2 as core 2 on socket 0
EAL: Detected lcore 3 as core 3 on socket 0
EAL: Detected lcore 4 as core 4 on socket 0
EAL: Detected lcore 5 as core 5 on socket 0
EAL: Detected lcore 6 as core 6 on socket 0
EAL: Detected lcore 7 as core 7 on socket 0
EAL: Detected lcore 8 as core 0 on socket 1
EAL: Detected lcore 9 as core 1 on socket 1
EAL: Detected lcore 10 as core 2 on socket 1
EAL: Detected lcore 11 as core 3 on socket 1
EAL: Detected lcore 12 as core 4 on socket 1
EAL: Detected lcore 13 as core 5 on socket 1
EAL: Detected lcore 14 as core 6 on socket 1
EAL: Detected lcore 15 as core 7 on socket 1
EAL: Detected lcore 16 as core 0 on socket 0
EAL: Detected lcore 17 as core 1 on socket 0
EAL: Detected lcore 18 as core 2 on socket 0
EAL: Detected lcore 19 as core 3 on socket 0
EAL: Detected lcore 20 as core 4 on socket 0
EAL: Detected lcore 21 as core 5 on socket 0
EAL: Detected lcore 22 as core 6 on socket 0
EAL: Detected lcore 23 as core 7 on socket 0
EAL: Detected lcore 24 as core 0 on socket 1
EAL: Detected lcore 25 as core 1 on socket 1
EAL: Detected lcore 26 as core 2 on socket 1
EAL: Detected lcore 27 as core 3 on socket 1
EAL: Detected lcore 28 as core 4 on socket 1
EAL: Detected lcore 29 as core 5 on socket 1
EAL: Detected lcore 30 as core 6 on socket 1
EAL: Detected lcore 31 as core 7 on socket 1
EAL: Support maximum 128 logical core(s) by configuration.
EAL: Detected 32 lcore(s)
EAL: VFIO modules not all loaded, skip VFIO support...
EAL: Setting up memory...
EAL: Ask a virtual area of 0x80 bytes
EAL: Virtual area found at 0x7fae2a20 (size = 0x80)
EAL: Ask a virtual area of 0x760 bytes
EAL: Virtual area found at 0x7fae22a0 (size = 0x760)
EAL: Ask a virtual area of 0x140 bytes
EAL: Virtual area found at 0x7fae2140 (size = 0x140)
EAL: Ask a virtual area of 0x20 bytes
EAL: Virtual area found at 0x7fae2100 (size = 0x20)
EAL: Ask a virtual area of 0x40 bytes
EAL: Virtual area found at 0x7fae20a0 (size = 0x40)
EAL: Ask a virtual area of 0x20 bytes
EAL: Virtual area found at 0x7fae2060 (size = 0x20)
EAL: Ask a virtual area of 0x20 bytes
EAL: Virtual area found at 0x7fae2020 (size = 0x20)
EAL: Ask a virtual area of 0x20 bytes
EAL: Virtual area found at 0x7fae1fe0 (size = 0x20)
EAL: Ask a virtual area of 0x580 bytes
EAL: Virtual area found at 0x7fae1a40 (size = 0x580)
EAL: Ask a virtual area of 0x3b20 bytes
EAL: Virtual area found at 0x7faddf00 (size = 0x3b20)
EAL: Ask a virtual area of 0x40 bytes
EAL: Virtual area found at 0x7faddea0 (size = 0x40)
EAL: Ask a virtual area of 0x7c0 bytes
EAL: Virtual area found at 0x7fadd6c0 (size = 0x7c0)
EAL: Ask a virtual area of 0x20 bytes
EAL: Virtual area found at 0x7fadd680 (size = 0x20)
EAL: Ask a virtual area of 0x20 bytes
EAL: Virtual area found at 0x7fadd640 (size = 0x20)
EAL: Ask a virtual area of 0x20 bytes
EAL: Virtual area found at 0x7fadd600 (size = 0x20)
EAL: Ask a virtual area of 0x20 bytes
EAL: Virtual area found at 0x7fadd5c0 (size = 0x20)
EAL: Ask a virtual area of 0x20 bytes
EAL: Virtual area found at 0x7fadd580 (size = 0x20)
EAL: Ask a virtual area of 0x980 bytes
EAL: Virtual area found at 0x7fadcbe0 (size = 0x980)
EAL: Ask a virtual area of 0x1900 bytes
EAL: Virtual area found at 0x7fadb2c0 (size = 0x1900)
EAL: Ask a virtual area of 0x2440 bytes
EAL: Virtual area found at 0x7fad8e60 (size = 0x2440)
EAL: Ask a virtual area of 0xc80 bytes
EAL: Virtual area found at 0x7fad81c0 (size = 0xc80)
EAL: Ask a virtual area of 0x3200 bytes
EAL: Virtual area found at 0x7fad4fa0 (size = 0x3200)
EAL: Ask a virtual area of 0x3db80 bytes
EAL: Virtual area found at 0x7fa97400 (size = 0x3db80)
EAL: Ask a virtual area of 0xfa00 bytes
EAL: Virtual area found at 0x7fa879e0 (size = 0xfa00)
EAL: Ask a virtual area of 0x16840 bytes
EAL: Virtual area found at 0x7fa71180 (size = 0x16840)
EAL: Ask a virtual area of 0x7d00 bytes
EAL: Virtual area found at 0x7fa6946

[dpdk-dev] [PATCH 1/7] ether: add function to query for link state interrupt

[Neil Horman]

On Wed, Feb 04, 2015 at 09:49:07PM -0800, Stephen Hemminger wrote:
> On Thu, 5 Feb 2015 01:47:37 +
> Neil Horman  wrote:
> 
> > On Wed, Feb 04, 2015 at 05:13:23PM -0800, Stephen Hemminger wrote:
> > > From: Stephen Hemminger 
> > > 
> > > Allow application to query whether link state will work.
> > > This is also part of abstracting dependency on PCI.
> > > 
> > > Signed-off-by: Stephen Hemminger 
> > > ---
> > >  lib/librte_ether/rte_ethdev.c | 14 ++
> > >  lib/librte_ether/rte_ethdev.h | 12 
> > >  2 files changed, 26 insertions(+)
> > > 
> > > diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c
> > > index ea3a1fb..4d803d0 100644
> > > --- a/lib/librte_ether/rte_ethdev.c
> > > +++ b/lib/librte_ether/rte_ethdev.c
> > > @@ -1264,6 +1264,20 @@ rte_eth_link_get_nowait(uint8_t port_id, struct 
> > > rte_eth_link *eth_link)
> > >   }
> > >  }
> > >  
> > > +int
> > > +rte_eth_has_link_state(uint8_t port_id)
> > > +{
> > > + struct rte_eth_dev *dev;
> > > +
> > > + if (port_id >= nb_ports) {
> > > + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
> > > + return 0;
> > > + }
> > > + dev = &rte_eth_devices[port_id];
> > > +
> > > + return (dev->pci_dev->driver->drv_flags & RTE_PCI_DRV_INTR_LSC) != 0;
> > > +}
> > > +
> > >  void
> > >  rte_eth_stats_get(uint8_t port_id, struct rte_eth_stats *stats)
> > >  {
> > > diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
> > > index 1200c1c..9d43ca3 100644
> > > --- a/lib/librte_ether/rte_ethdev.h
> > > +++ b/lib/librte_ether/rte_ethdev.h
> > > @@ -2038,6 +2038,18 @@ extern void rte_eth_link_get_nowait(uint8_t 
> > > port_id,
> > >   struct rte_eth_link *link);
> > >  
> > >  /**
> > > + * Test whether device supports link state interrupt mode.
> > > + *
> > > + * @param port_id
> > > + *   The port identifier of the Ethernet device.
> > > + * @return
> > > + *   - (1) if link state interrupt is supported
> > > + *   - (0) if link state interrupt is not supported
> > > + */
> > > +extern int
> > > +rte_eth_has_link_state(uint8_t port_id);
> > > +
> > > +/**
> > >   * Retrieve the general I/O statistics of an Ethernet device.
> > >   *
> > >   * @param port_id
> > > -- 
> > > 2.1.4
> > > 
> > > 
> > I think if your intention is to expose this as an application (or other 
> > library
> > visible api), you'll need to add it to the appropriate version map file for 
> > the
> > link now.  Should just be a single line addition
> > 
> > Regards
> > Neil
> > 
> 
> Will add that after other changes are approved.
> Also looking at the map files they seem to have been auto generated
> because some symbols are exposed which are unnecessary (example 
> pci_driver_list)
It wasn't quite auto generated.  It was generated based on what the requisite
header files for a library exposed to the various applications that used them.
In your example above pci_driver_list is a global public variable, that is
referenced by applications that the DPDK ships (in this case the test_pci
application).  I completely agree that it shouldn't need to be versioned, as it
shouldn't be exposed at all, relying instead on accessor functions to reach it,
but as such, it is what it is.  By versioning it, we at least allow ourselves
the option to maintain backwards compatibility if the list type changes or we
wish to start hiding the symbol in future versions.

Neil

> 

[dpdk-dev] [PATCH v2 4/7] rte_sched: don't clear statistics when read

[Neil Horman]

On Wed, Feb 04, 2015 at 10:13:58PM -0800, Stephen Hemminger wrote:
> From: Stephen Hemminger 
> 
> Make rte_sched statistics API work like the ethernet statistics API.
> Don't auto-clear statistics.
> 
> Signed-off-by: Stephen Hemminger 
> ---
>  lib/librte_sched/rte_sched.c | 30 ++
>  lib/librte_sched/rte_sched.h | 29 +
>  2 files changed, 59 insertions(+)
> 
> diff --git a/lib/librte_sched/rte_sched.c b/lib/librte_sched/rte_sched.c
> index 8cb8bf1..d891e50 100644
> --- a/lib/librte_sched/rte_sched.c
> +++ b/lib/librte_sched/rte_sched.c
> @@ -935,6 +935,21 @@ rte_sched_subport_read_stats(struct rte_sched_port *port,
>  }
>  
>  int
> +rte_sched_subport_stats_reset(struct rte_sched_port *port,
> +   uint32_t subport_id)
> +{
> + struct rte_sched_subport *s;
> +
> + /* Check user parameters */
> + if (port == NULL || subport_id >= port->n_subports_per_port)
> + return -1;
> +
> + s = port->subport + subport_id;
> + memset(&s->stats, 0, sizeof(struct rte_sched_subport_stats));
> + return 0;
> +}
> +
> +int
>  rte_sched_queue_read_stats(struct rte_sched_port *port,
>   uint32_t queue_id,
>   struct rte_sched_queue_stats *stats,
> @@ -963,6 +978,21 @@ rte_sched_queue_read_stats(struct rte_sched_port *port,
>   return 0;
>  }
>  
> +int
> +rte_sched_queue_stats_reset(struct rte_sched_port *port,
> + uint32_t queue_id)
> +{
> + struct rte_sched_queue_extra *qe;
> +
> + /* Check user parameters */
> + if (port == NULL || queue_id >= rte_sched_port_queues_per_port(port))
> + return -1;
> +
> + qe = port->queue_extra + queue_id;
> + memset(&qe->stats, 0, sizeof(struct rte_sched_queue_stats));
> + return 0;
> +}
> +
>  static inline uint32_t
>  rte_sched_port_qindex(struct rte_sched_port *port, uint32_t subport, 
> uint32_t pipe, uint32_t traffic_class, uint32_t queue)
>  {
> diff --git a/lib/librte_sched/rte_sched.h b/lib/librte_sched/rte_sched.h
> index e9bf18a..3d007e4 100644
> --- a/lib/librte_sched/rte_sched.h
> +++ b/lib/librte_sched/rte_sched.h
> @@ -317,6 +317,21 @@ rte_sched_subport_read_stats(struct rte_sched_port *port,
>   struct rte_sched_subport_stats *stats,
>   uint32_t *tc_ov);
>  
> +
> +/**
> + * Hierarchical scheduler subport statistics reset
> + *
> + * @param port
> + *   Handle to port scheduler instance
> + * @param subport_id
> + *   Subport ID
> + * @return
> + *   0 upon success, error code otherwise
> + */
> +int
> +rte_sched_subport_stats_reset(struct rte_sched_port *port,
> +   uint32_t subport_id);
> +
>  /**
>   * Hierarchical scheduler queue statistics read
>   *
> @@ -338,6 +353,20 @@ rte_sched_queue_read_stats(struct rte_sched_port *port,
>   struct rte_sched_queue_stats *stats,
>   uint16_t *qlen);
>  
> +/**
> + * Hierarchical scheduler queue statistics reset
> + *
> + * @param port
> + *   Handle to port scheduler instance
> + * @param queue_id
> + *   Queue ID within port scheduler
> + * @return
> + *   0 upon success, error code otherwise
> + */
> +int
> +rte_sched_queue_stats_reset(struct rte_sched_port *port,
> + uint32_t queue_id);
> +
Both need to be added to the version map to expose them properly.
Neil

[dpdk-dev] mmap fails with more than 40000 hugepages

[Neil Horman]

On Thu, Feb 05, 2015 at 12:00:48PM +, Damjan Marion (damarion) wrote:
> Hi,
> 
> I have system with 2 NUMA nodes and 256G RAM total. I noticed that DPDK 
> crashes in rte_eal_init()
> when number of available hugepages is around 4 or above.
> Everything works fine with lower values (i.e. 3).
> 
> I also tried with allocating 4 on node0 and 0 on node1, same crash 
> happens.
> 
> 
> Any idea what might be causing this?
> 
> Thanks,
> 
> Damjan
> 
> 
> $ cat 
> /sys/devices/system/node/node[01]/hugepages/hugepages-2048kB/nr_hugepages
> 2
> 2
> 
> $ grep -i huge /proc/meminfo
> AnonHugePages:706560 kB
> HugePages_Total:   4
> HugePages_Free:4
> HugePages_Rsvd:0
> HugePages_Surp:0
> Hugepagesize:   2048 kB
> 
Whats your shmmax value set to? 4 2MB hugepages is way above the default
setting for how much shared ram a system will allow.  I've not done the math on
your logs below, but judging by the size of some of the mapped segments, I'm
betting your hitting the default limit of 4GB.

Neil

> 
> $ sudo ~/src/dpdk/x86_64-native-linuxapp-gcc/app/testpmd -l 5-7 -n 3 
> --socket-mem 512,512
> EAL: Detected lcore 0 as core 0 on socket 0
> EAL: Detected lcore 1 as core 1 on socket 0
> EAL: Detected lcore 2 as core 2 on socket 0
> EAL: Detected lcore 3 as core 3 on socket 0
> EAL: Detected lcore 4 as core 4 on socket 0
> EAL: Detected lcore 5 as core 5 on socket 0
> EAL: Detected lcore 6 as core 6 on socket 0
> EAL: Detected lcore 7 as core 7 on socket 0
> EAL: Detected lcore 8 as core 0 on socket 1
> EAL: Detected lcore 9 as core 1 on socket 1
> EAL: Detected lcore 10 as core 2 on socket 1
> EAL: Detected lcore 11 as core 3 on socket 1
> EAL: Detected lcore 12 as core 4 on socket 1
> EAL: Detected lcore 13 as core 5 on socket 1
> EAL: Detected lcore 14 as core 6 on socket 1
> EAL: Detected lcore 15 as core 7 on socket 1
> EAL: Detected lcore 16 as core 0 on socket 0
> EAL: Detected lcore 17 as core 1 on socket 0
> EAL: Detected lcore 18 as core 2 on socket 0
> EAL: Detected lcore 19 as core 3 on socket 0
> EAL: Detected lcore 20 as core 4 on socket 0
> EAL: Detected lcore 21 as core 5 on socket 0
> EAL: Detected lcore 22 as core 6 on socket 0
> EAL: Detected lcore 23 as core 7 on socket 0
> EAL: Detected lcore 24 as core 0 on socket 1
> EAL: Detected lcore 25 as core 1 on socket 1
> EAL: Detected lcore 26 as core 2 on socket 1
> EAL: Detected lcore 27 as core 3 on socket 1
> EAL: Detected lcore 28 as core 4 on socket 1
> EAL: Detected lcore 29 as core 5 on socket 1
> EAL: Detected lcore 30 as core 6 on socket 1
> EAL: Detected lcore 31 as core 7 on socket 1
> EAL: Support maximum 128 logical core(s) by configuration.
> EAL: Detected 32 lcore(s)
> EAL: VFIO modules not all loaded, skip VFIO support...
> EAL: Setting up memory...
> EAL: Ask a virtual area of 0x80 bytes
> EAL: Virtual area found at 0x7fae2a20 (size = 0x80)
> EAL: Ask a virtual area of 0x760 bytes
> EAL: Virtual area found at 0x7fae22a0 (size = 0x760)
> EAL: Ask a virtual area of 0x140 bytes
> EAL: Virtual area found at 0x7fae2140 (size = 0x140)
> EAL: Ask a virtual area of 0x20 bytes
> EAL: Virtual area found at 0x7fae2100 (size = 0x20)
> EAL: Ask a virtual area of 0x40 bytes
> EAL: Virtual area found at 0x7fae20a0 (size = 0x40)
> EAL: Ask a virtual area of 0x20 bytes
> EAL: Virtual area found at 0x7fae2060 (size = 0x20)
> EAL: Ask a virtual area of 0x20 bytes
> EAL: Virtual area found at 0x7fae2020 (size = 0x20)
> EAL: Ask a virtual area of 0x20 bytes
> EAL: Virtual area found at 0x7fae1fe0 (size = 0x20)
> EAL: Ask a virtual area of 0x580 bytes
> EAL: Virtual area found at 0x7fae1a40 (size = 0x580)
> EAL: Ask a virtual area of 0x3b20 bytes
> EAL: Virtual area found at 0x7faddf00 (size = 0x3b20)
> EAL: Ask a virtual area of 0x40 bytes
> EAL: Virtual area found at 0x7faddea0 (size = 0x40)
> EAL: Ask a virtual area of 0x7c0 bytes
> EAL: Virtual area found at 0x7fadd6c0 (size = 0x7c0)
> EAL: Ask a virtual area of 0x20 bytes
> EAL: Virtual area found at 0x7fadd680 (size = 0x20)
> EAL: Ask a virtual area of 0x20 bytes
> EAL: Virtual area found at 0x7fadd640 (size = 0x20)
> EAL: Ask a virtual area of 0x20 bytes
> EAL: Virtual area found at 0x7fadd600 (size = 0x20)
> EAL: Ask a virtual area of 0x20 bytes
> EAL: Virtual area found at 0x7fadd5c0 (size = 0x20)
> EAL: Ask a virtual area of 0x20 bytes
> EAL: Virtual area found at 0x7fadd580 (size = 0x20)
> EAL: Ask a virtual area of 0x980 bytes
> EAL: Virtual area found at 0x7fadcbe0 (size = 0x980)
> EAL: Ask a virtual area of 0x1900 bytes
> EAL: Virtual area found at 0x7fadb2c0 (size = 0x1900)
> EAL: Ask a virtual area of 0x2440 bytes
> EAL: Virtual area found at 0x7fad8e60 (size = 0x2440)
> EAL: Ask a virtual area of

[dpdk-dev] mmap fails with more than 40000 hugepages

[Damjan Marion (damarion)]

> On 05 Feb 2015, at 13:59, Neil Horman  wrote:
> 
> On Thu, Feb 05, 2015 at 12:00:48PM +, Damjan Marion (damarion) wrote:
>> Hi,
>> 
>> I have system with 2 NUMA nodes and 256G RAM total. I noticed that DPDK 
>> crashes in rte_eal_init()
>> when number of available hugepages is around 4 or above.
>> Everything works fine with lower values (i.e. 3).
>> 
>> I also tried with allocating 4 on node0 and 0 on node1, same crash 
>> happens.
>> 
>> 
>> Any idea what might be causing this?
>> 
>> Thanks,
>> 
>> Damjan
>> 
>> 
>> $ cat 
>> /sys/devices/system/node/node[01]/hugepages/hugepages-2048kB/nr_hugepages
>> 2
>> 2
>> 
>> $ grep -i huge /proc/meminfo
>> AnonHugePages:706560 kB
>> HugePages_Total:   4
>> HugePages_Free:4
>> HugePages_Rsvd:0
>> HugePages_Surp:0
>> Hugepagesize:   2048 kB
>> 
> Whats your shmmax value set to? 4 2MB hugepages is way above the default
> setting for how much shared ram a system will allow.  I've not done the math 
> on
> your logs below, but judging by the size of some of the mapped segments, I'm
> betting your hitting the default limit of 4GB.

$ cat /proc/sys/kernel/shmmax
33554432

$ sysctl -w kernel.shmmax=8589934592
kernel.shmmax = 8589934592

same crash :(

Thanks,

Damjan

[dpdk-dev] mmap fails with more than 40000 hugepages

[Jay Rolette]

On Thu, Feb 5, 2015 at 6:00 AM, Damjan Marion (damarion)  wrote:

> Hi,
>
> I have system with 2 NUMA nodes and 256G RAM total. I noticed that DPDK
> crashes in rte_eal_init()
> when number of available hugepages is around 4 or above.
> Everything works fine with lower values (i.e. 3).
>
> I also tried with allocating 4 on node0 and 0 on node1, same crash
> happens.
>
>
> Any idea what might be causing this?
>

Any reason you can't switch to using 1GB hugepages? You'll get better
performance and your init time will be shorter. The systems we run on are
similar (256GB, 2 NUMA nodes) and that works fine for us.

Not directly related, but if you have to stick with 2MB hugepages, you
might want to take a look at a patch I submitted that fixes the O(n^2)
algorithm used in initializing hugepages.

Jay

[dpdk-dev] mmap fails with more than 40000 hugepages

[Damjan Marion (damarion)]

On 05 Feb 2015, at 14:22, Jay Rolette mailto:rolette 
at infiniteio.com>> wrote:

On Thu, Feb 5, 2015 at 6:00 AM, Damjan Marion (damarion) mailto:damarion at cisco.com>> wrote:
Hi,

I have system with 2 NUMA nodes and 256G RAM total. I noticed that DPDK crashes 
in rte_eal_init()
when number of available hugepages is around 4 or above.
Everything works fine with lower values (i.e. 3).

I also tried with allocating 4 on node0 and 0 on node1, same crash happens.


Any idea what might be causing this?

Any reason you can't switch to using 1GB hugepages? You'll get better 
performance and your init time will be shorter. The systems we run on are 
similar (256GB, 2 NUMA nodes) and that works fine for us.

Yes, unfortunately some other consumers are needing smaller pages


Not directly related, but if you have to stick with 2MB hugepages, you might 
want to take a look at a patch I submitted that fixes the O(n^2) algorithm used 
in initializing hugepages.

I tried it hoping that it will change something, no luck?

Thanks,

Damjan

[dpdk-dev] DPDK testpmd forwarding performace degradation

[Alexander Belyakov]

On Thu, Jan 29, 2015 at 3:43 PM, Alexander Belyakov 
wrote:

>
>
> On Wed, Jan 28, 2015 at 3:24 PM, Alexander Belyakov 
> wrote:
>
>>
>>
>> On Tue, Jan 27, 2015 at 7:21 PM, De Lara Guarch, Pablo <
>> pablo.de.lara.guarch at intel.com> wrote:
>>
>>>
>>>
>>> > On Tue, Jan 27, 2015 at 10:51 AM, Alexander Belyakov
>>>
>>> >  wrote:
>>>
>>> >
>>>
>>> > Hi Pablo,
>>>
>>> >
>>>
>>> > On Mon, Jan 26, 2015 at 5:22 PM, De Lara Guarch, Pablo
>>>
>>> >  wrote:
>>>
>>> > Hi Alexander,
>>>
>>> >
>>>
>>> > > -Original Message-
>>>
>>> > > From: dev [mailto:dev-bounces at dpdk.org ] On
>>> Behalf Of Alexander
>>>
>>> > Belyakov
>>>
>>> > > Sent: Monday, January 26, 2015 10:18 AM
>>>
>>> > > To: dev at dpdk.org
>>>
>>> > > Subject: [dpdk-dev] DPDK testpmd forwarding performace degradation
>>>
>>> > >
>>>
>>> > > Hello,
>>>
>>> > >
>>>
>>> > > recently I have found a case of significant performance degradation
>>> for our
>>>
>>> > > application (built on top of DPDK, of course). Surprisingly, similar
>>> issue
>>>
>>> > > is easily reproduced with default testpmd.
>>>
>>> > >
>>>
>>> > > To show the case we need simple IPv4 UDP flood with variable UDP
>>>
>>> > payload
>>>
>>> > > size. Saying "packet length" below I mean: Eth header length (14
>>> bytes) +
>>>
>>> > > IPv4 header length (20 bytes) + UPD header length (8 bytes) + UDP
>>> payload
>>>
>>> > > length (variable) + CRC (4 bytes). Source IP addresses and ports are
>>>
>>> > selected
>>>
>>> > > randomly for each packet.
>>>
>>> > >
>>>
>>> > > I have used DPDK with revisions 1.6.0r2 and 1.7.1. Both show the same
>>>
>>> > issue.
>>>
>>> > >
>>>
>>> > > Follow "Quick start" guide (http://dpdk.org/doc/quick-start) to
>>> build and
>>>
>>> > > run testpmd. Enable testpmd forwarding ("start" command).
>>>
>>> > >
>>>
>>> > > Table below shows measured forwarding performance depending on
>>>
>>> > packet
>>>
>>> > > length:
>>>
>>> > >
>>>
>>> > > No. -- UDP payload length (bytes) -- Packet length (bytes) --
>>> Forwarding
>>>
>>> > > performance (Mpps) -- Expected theoretical performance (Mpps)
>>>
>>> > >
>>>
>>> > > 1. 0 -- 64 -- 14.8 -- 14.88
>>>
>>> > > 2. 34 -- 80 -- 12.4 -- 12.5
>>>
>>> > > 3. 35 -- 81 -- 6.2 -- 12.38 (!)
>>>
>>> > > 4. 40 -- 86 -- 6.6 -- 11.79
>>>
>>> > > 5. 49 -- 95 -- 7.6 -- 10.87
>>>
>>> > > 6. 50 -- 96 -- 10.7 -- 10.78 (!)
>>>
>>> > > 7. 60 -- 106 -- 9.4 -- 9.92
>>>
>>> > >
>>>
>>> > > At line number 3 we have added 1 byte of UDP payload (comparing to
>>>
>>> > > previous
>>>
>>> > > line) and got forwarding performance halved! 6.2 Mpps against 12.38
>>> Mpps
>>>
>>> > > of
>>>
>>> > > expected theoretical maximum for this packet size.
>>>
>>> > >
>>>
>>> > > That is the issue.
>>>
>>> > >
>>>
>>> > > Significant performance degradation exists up to 50 bytes of UDP
>>> payload
>>>
>>> > > (96 bytes packet length), where it jumps back to theoretical maximum.
>>>
>>> > >
>>>
>>> > > What is happening between 80 and 96 bytes packet length?
>>>
>>> > >
>>>
>>> > > This issue is stable and 100% reproducible. At this point I am not
>>> sure if
>>>
>>> > > it is DPDK or NIC issue. These tests have been performed on Intel(R)
>>> Eth
>>>
>>> > > Svr Bypass Adapter X520-LR2 (X520LR2BP).
>>>
>>> > >
>>>
>>> > > Is anyone aware of such strange behavior?
>>>
>>> > I cannot reproduce the issue using two ports on two different 82599EB
>>> NICs,
>>>
>>> > using 1.7.1 and 1.8.0.
>>>
>>> > I always get either same or better linerate as I increase the packet
>>> size.
>>>
>>> >
>>>
>>> > Thank you for trying to reproduce the issue.
>>>
>>> >
>>>
>>> > Actually, have you tried using 1.8.0?
>>>
>>> >
>>>
>>> > I feel 1.8.0 is little bit immature and might require some post-release
>>>
>>> > patching. Even tespmd from this release is not forwarding packets
>>> properly
>>>
>>> > on my setup. It is up and running without visible errors/warnings,
>>> TX/RX
>>>
>>> > counters are ticking but I can not see any packets at the output.
>>>
>>>
>>>
>>> This is strange. Without  changing anything, forwarding works perfectly
>>> for me
>>>
>>> (so, RTE_LIBRTE_IXGBE_RX_ALLOW_BULK_ALLOC is enabled).
>>>
>>>
>>>
>>> >Please note, both 1.6.0r2 and 1.7.1 releases work (on the same setup)
>>> out-of-the-box just
>>>
>>> > fine with only exception of this mysterious performance drop.
>>>
>>> > So it will take some time to figure out what is wrong with dpdk-1.8.0.
>>>
>>> > Meanwhile we could focus on stable dpdk-1.7.1.
>>>
>>> >
>>>
>>> > Managed to get testpmd from dpdk-1.8.0 to work on my setup.
>>>
>>> > Unfortunately I had to disable RTE_LIBRTE_IXGBE_RX_ALLOW_BULK_ALLOC,
>>>
>>> > it is new comparing to 1.7.1 and somehow breaks testpmd forwarding. By
>>> the
>>>
>>> > way, simply disabling RTE_LIBRTE_IXGBE_RX_ALLOW_BULK_ALLOC in
>>>
>>> > common_linuxapp config file breaks the build - had to make
>>> quick'n'dirty fix
>>>
>>> > in struct igb_rx_queue as well.
>>>
>>> >
>>>
>>> > Anyway, issue is still here.
>>>
>>> >
>>>
>>> > Forwarding 80 bytes packets at 12.4 Mp

[dpdk-dev] mmap fails with more than 40000 hugepages

[Neil Horman]

On Thu, Feb 05, 2015 at 01:20:01PM +, Damjan Marion (damarion) wrote:
> 
> > On 05 Feb 2015, at 13:59, Neil Horman  wrote:
> > 
> > On Thu, Feb 05, 2015 at 12:00:48PM +, Damjan Marion (damarion) wrote:
> >> Hi,
> >> 
> >> I have system with 2 NUMA nodes and 256G RAM total. I noticed that DPDK 
> >> crashes in rte_eal_init()
> >> when number of available hugepages is around 4 or above.
> >> Everything works fine with lower values (i.e. 3).
> >> 
> >> I also tried with allocating 4 on node0 and 0 on node1, same crash 
> >> happens.
> >> 
> >> 
> >> Any idea what might be causing this?
> >> 
> >> Thanks,
> >> 
> >> Damjan
> >> 
> >> 
> >> $ cat 
> >> /sys/devices/system/node/node[01]/hugepages/hugepages-2048kB/nr_hugepages
> >> 2
> >> 2
> >> 
> >> $ grep -i huge /proc/meminfo
> >> AnonHugePages:706560 kB
> >> HugePages_Total:   4
> >> HugePages_Free:4
> >> HugePages_Rsvd:0
> >> HugePages_Surp:0
> >> Hugepagesize:   2048 kB
> >> 
> > Whats your shmmax value set to? 4 2MB hugepages is way above the default
> > setting for how much shared ram a system will allow.  I've not done the 
> > math on
> > your logs below, but judging by the size of some of the mapped segments, I'm
> > betting your hitting the default limit of 4GB.
> 
> $ cat /proc/sys/kernel/shmmax
> 33554432
> 
> $ sysctl -w kernel.shmmax=8589934592
> kernel.shmmax = 8589934592
> 
> same crash :(
> 
> Thanks,
> 
> Damjan

What about the shmmni and shmmax values.  The shmmax value will also need to be
set to at least 80G (more if you have other shared memory needs), and shmmni
will need to be larger than 40,000 to handle all the segments your creating.
Neil

[dpdk-dev] mmap fails with more than 40000 hugepages

[Jay Rolette]

On Thu, Feb 5, 2015 at 7:36 AM, Damjan Marion (damarion)  wrote:
>
>  On 05 Feb 2015, at 14:22, Jay Rolette  wrote:
>
>   Not directly related, but if you have to stick with 2MB hugepages, you
> might want to take a look at a patch I submitted that fixes the O(n^2)
> algorithm used in initializing hugepages.
>
>
>  I tried it hoping that it will change something, no luck?
>

I suggested that patch only to speed up your init time. There's nothing in
it that will help with the crash you are seeing.

[dpdk-dev] mmap fails with more than 40000 hugepages

[Damjan Marion (damarion)]

> On 05 Feb 2015, at 15:59, Neil Horman  wrote:
> 
> On Thu, Feb 05, 2015 at 01:20:01PM +, Damjan Marion (damarion) wrote:
>> 
>>> On 05 Feb 2015, at 13:59, Neil Horman  wrote:
>>> 
>>> On Thu, Feb 05, 2015 at 12:00:48PM +, Damjan Marion (damarion) wrote:
 Hi,
 
 I have system with 2 NUMA nodes and 256G RAM total. I noticed that DPDK 
 crashes in rte_eal_init()
 when number of available hugepages is around 4 or above.
 Everything works fine with lower values (i.e. 3).
 
 I also tried with allocating 4 on node0 and 0 on node1, same crash 
 happens.
 
 
 Any idea what might be causing this?
 
 Thanks,
 
 Damjan
 
 
 $ cat 
 /sys/devices/system/node/node[01]/hugepages/hugepages-2048kB/nr_hugepages
 2
 2
 
 $ grep -i huge /proc/meminfo
 AnonHugePages:706560 kB
 HugePages_Total:   4
 HugePages_Free:4
 HugePages_Rsvd:0
 HugePages_Surp:0
 Hugepagesize:   2048 kB
 
>>> Whats your shmmax value set to? 4 2MB hugepages is way above the default
>>> setting for how much shared ram a system will allow.  I've not done the 
>>> math on
>>> your logs below, but judging by the size of some of the mapped segments, I'm
>>> betting your hitting the default limit of 4GB.
>> 
>> $ cat /proc/sys/kernel/shmmax
>> 33554432
>> 
>> $ sysctl -w kernel.shmmax=8589934592
>> kernel.shmmax = 8589934592
>> 
>> same crash :(
>> 
>> Thanks,
>> 
>> Damjan
> 
> What about the shmmni and shmmax values.  The shmmax value will also need to 
> be
> set to at least 80G (more if you have other shared memory needs), and shmmni
> will need to be larger than 40,000 to handle all the segments your creating.
> Neil

Hmm, if that is the reason, then 3 will also not work. as both values are 
set by default to much much lower value.

Anyway, I tried:

$ sysctl kernel.shmmni
kernel.shmmni = 10

$ sysctl kernel.shmmax
kernel.shmmax = 274877906944

$ cat /sys/devices/system/node/node[01]/hugepages/hugepages-2048kB/nr_hugepages
2
2

$ sudo ~/src/dpdk/x86_64-native-linuxapp-gcc/app/testpmd -l 5-7 -n 3 
--socket-mem 512,512
EAL: Detected lcore 0 as core 0 on socket 0
EAL: Detected lcore 1 as core 1 on socket 0
EAL: Detected lcore 2 as core 2 on socket 0
EAL: Detected lcore 3 as core 3 on socket 0
EAL: Detected lcore 4 as core 4 on socket 0
EAL: Detected lcore 5 as core 5 on socket 0
EAL: Detected lcore 6 as core 6 on socket 0
EAL: Detected lcore 7 as core 7 on socket 0
EAL: Detected lcore 8 as core 0 on socket 1
EAL: Detected lcore 9 as core 1 on socket 1
EAL: Detected lcore 10 as core 2 on socket 1
EAL: Detected lcore 11 as core 3 on socket 1
EAL: Detected lcore 12 as core 4 on socket 1
EAL: Detected lcore 13 as core 5 on socket 1
EAL: Detected lcore 14 as core 6 on socket 1
EAL: Detected lcore 15 as core 7 on socket 1
EAL: Detected lcore 16 as core 0 on socket 0
EAL: Detected lcore 17 as core 1 on socket 0
EAL: Detected lcore 18 as core 2 on socket 0
EAL: Detected lcore 19 as core 3 on socket 0
EAL: Detected lcore 20 as core 4 on socket 0
EAL: Detected lcore 21 as core 5 on socket 0
EAL: Detected lcore 22 as core 6 on socket 0
EAL: Detected lcore 23 as core 7 on socket 0
EAL: Detected lcore 24 as core 0 on socket 1
EAL: Detected lcore 25 as core 1 on socket 1
EAL: Detected lcore 26 as core 2 on socket 1
EAL: Detected lcore 27 as core 3 on socket 1
EAL: Detected lcore 28 as core 4 on socket 1
EAL: Detected lcore 29 as core 5 on socket 1
EAL: Detected lcore 30 as core 6 on socket 1
EAL: Detected lcore 31 as core 7 on socket 1
EAL: Support maximum 128 logical core(s) by configuration.
EAL: Detected 32 lcore(s)
EAL: VFIO modules not all loaded, skip VFIO support...
EAL: Setting up memory...
EAL: Ask a virtual area of 0x8000 bytes
EAL: Virtual area found at 0x7fe28000 (size = 0x8000)
EAL: Ask a virtual area of 0x8000 bytes
EAL: Virtual area found at 0x7fe1c000 (size = 0x8000)
EAL: Ask a virtual area of 0x20 bytes
EAL: Virtual area found at 0x7fe335c0 (size = 0x20)
EAL: Ask a virtual area of 0x940 bytes
EAL: Virtual area found at 0x7fe32c60 (size = 0x940)
EAL: Ask a virtual area of 0x5c0 bytes
EAL: Virtual area found at 0x7fe32680 (size = 0x5c0)
EAL: Ask a virtual area of 0x38e0 bytes
EAL: Virtual area found at 0x7fe24700 (size = 0x38e0)
EAL: Ask a virtual area of 0x20 bytes
EAL: Virtual area found at 0x7fe32640 (size = 0x20)
EAL: Ask a virtual area of 0x800 bytes
EAL: Virtual area found at 0x7fe31e20 (size = 0x800)
EAL: Ask a virtual area of 0x81580 bytes
EAL: Virtual area found at 0x7fd9aa60 (size = 0x81580)
EAL: Ask a virtual area of 0x1f40 bytes
EAL: Virtual area found at 0x7fd98b00 (size = 0x1f40)
EAL: Ask a virtual area of 0xfd40 bytes
EAL: Virtual area found at 0x7fd88da0 (size = 0xfd40)
EAL: Ask a virtual area of 0x20 by

[dpdk-dev] mmap fails with more than 40000 hugepages

[Neil Horman]

On Thu, Feb 05, 2015 at 03:41:25PM +, Damjan Marion (damarion) wrote:
> 
> > On 05 Feb 2015, at 15:59, Neil Horman  wrote:
> > 
> > On Thu, Feb 05, 2015 at 01:20:01PM +, Damjan Marion (damarion) wrote:
> >> 
> >>> On 05 Feb 2015, at 13:59, Neil Horman  wrote:
> >>> 
> >>> On Thu, Feb 05, 2015 at 12:00:48PM +, Damjan Marion (damarion) wrote:
>  Hi,
>  
>  I have system with 2 NUMA nodes and 256G RAM total. I noticed that DPDK 
>  crashes in rte_eal_init()
>  when number of available hugepages is around 4 or above.
>  Everything works fine with lower values (i.e. 3).
>  
>  I also tried with allocating 4 on node0 and 0 on node1, same crash 
>  happens.
>  
>  
>  Any idea what might be causing this?
>  
>  Thanks,
>  
>  Damjan
>  
>  
>  $ cat 
>  /sys/devices/system/node/node[01]/hugepages/hugepages-2048kB/nr_hugepages
>  2
>  2
>  
>  $ grep -i huge /proc/meminfo
>  AnonHugePages:706560 kB
>  HugePages_Total:   4
>  HugePages_Free:4
>  HugePages_Rsvd:0
>  HugePages_Surp:0
>  Hugepagesize:   2048 kB
>  
> >>> Whats your shmmax value set to? 4 2MB hugepages is way above the 
> >>> default
> >>> setting for how much shared ram a system will allow.  I've not done the 
> >>> math on
> >>> your logs below, but judging by the size of some of the mapped segments, 
> >>> I'm
> >>> betting your hitting the default limit of 4GB.
> >> 
> >> $ cat /proc/sys/kernel/shmmax
> >> 33554432
> >> 
> >> $ sysctl -w kernel.shmmax=8589934592
> >> kernel.shmmax = 8589934592
> >> 
> >> same crash :(
> >> 
> >> Thanks,
> >> 
> >> Damjan
> > 
> > What about the shmmni and shmmax values.  The shmmax value will also need 
> > to be
> > set to at least 80G (more if you have other shared memory needs), and shmmni
> > will need to be larger than 40,000 to handle all the segments your creating.
> > Neil
> 
> Hmm, if that is the reason, then 3 will also not work. as both values are 
> set by default to much much lower value.
> 
> Anyway, I tried:
> 
> $ sysctl kernel.shmmni
> kernel.shmmni = 10
> 
> $ sysctl kernel.shmmax
> kernel.shmmax = 274877906944
> 
> $ cat 
> /sys/devices/system/node/node[01]/hugepages/hugepages-2048kB/nr_hugepages
> 2
> 2
> 
Hmm, then you probably need to start looking at other failure possibilities.
The segment that failed was requesting a size of about 38Gb.  Is it possible
that your virtual memory map doesn't actually have a hole that large?  That
would indicate a bug in the eal init path, since it searches for that.  Time to
start adding some printf and dumping your /proc/pid/maps file

Neil

> $ sudo ~/src/dpdk/x86_64-native-linuxapp-gcc/app/testpmd -l 5-7 -n 3 
> --socket-mem 512,512
> EAL: Detected lcore 0 as core 0 on socket 0
> EAL: Detected lcore 1 as core 1 on socket 0
> EAL: Detected lcore 2 as core 2 on socket 0
> EAL: Detected lcore 3 as core 3 on socket 0
> EAL: Detected lcore 4 as core 4 on socket 0
> EAL: Detected lcore 5 as core 5 on socket 0
> EAL: Detected lcore 6 as core 6 on socket 0
> EAL: Detected lcore 7 as core 7 on socket 0
> EAL: Detected lcore 8 as core 0 on socket 1
> EAL: Detected lcore 9 as core 1 on socket 1
> EAL: Detected lcore 10 as core 2 on socket 1
> EAL: Detected lcore 11 as core 3 on socket 1
> EAL: Detected lcore 12 as core 4 on socket 1
> EAL: Detected lcore 13 as core 5 on socket 1
> EAL: Detected lcore 14 as core 6 on socket 1
> EAL: Detected lcore 15 as core 7 on socket 1
> EAL: Detected lcore 16 as core 0 on socket 0
> EAL: Detected lcore 17 as core 1 on socket 0
> EAL: Detected lcore 18 as core 2 on socket 0
> EAL: Detected lcore 19 as core 3 on socket 0
> EAL: Detected lcore 20 as core 4 on socket 0
> EAL: Detected lcore 21 as core 5 on socket 0
> EAL: Detected lcore 22 as core 6 on socket 0
> EAL: Detected lcore 23 as core 7 on socket 0
> EAL: Detected lcore 24 as core 0 on socket 1
> EAL: Detected lcore 25 as core 1 on socket 1
> EAL: Detected lcore 26 as core 2 on socket 1
> EAL: Detected lcore 27 as core 3 on socket 1
> EAL: Detected lcore 28 as core 4 on socket 1
> EAL: Detected lcore 29 as core 5 on socket 1
> EAL: Detected lcore 30 as core 6 on socket 1
> EAL: Detected lcore 31 as core 7 on socket 1
> EAL: Support maximum 128 logical core(s) by configuration.
> EAL: Detected 32 lcore(s)
> EAL: VFIO modules not all loaded, skip VFIO support...
> EAL: Setting up memory...
> EAL: Ask a virtual area of 0x8000 bytes
> EAL: Virtual area found at 0x7fe28000 (size = 0x8000)
> EAL: Ask a virtual area of 0x8000 bytes
> EAL: Virtual area found at 0x7fe1c000 (size = 0x8000)
> EAL: Ask a virtual area of 0x20 bytes
> EAL: Virtual area found at 0x7fe335c0 (size = 0x20)
> EAL: Ask a virtual area of 0x940 bytes
> EAL: Virtual area found at 0x7fe32c60 (size = 0x940)
> EAL: Ask a virtual area of 0x5c0 bytes
> EAL: Virt

[dpdk-dev] [PATCH net-next v2] hyper-v: allow access to vmbus from userspace driver

[Stephen Hemminger]

On Thu, 05 Feb 2015 12:01:03 +0100
Vitaly Kuznetsov  wrote:

> Stephen Hemminger  writes:
> 
> > This is enables the hyper-v driver for DPDK .
> > The hv_uio driver needs to access the shared vmbus monitor
> > pages.
> 
> Why can't we just make vmbus_connection struct public?
> 
> >
> > I would also like to put hv_uio in upstream kernel like other
> > uio drivers, but need to get API accepted with DPDK first.
> >
> > Signed-off-by: Stas Egorov 
> > Signed-off-by: Stephen Hemminger 
> >
> > ---
> > v2 - simplify and rename to vmbus_get_monitor_pages
> >
> >  drivers/hv/connection.c |   20 +---
> >  include/linux/hyperv.h  |3 +++
> >  2 files changed, 20 insertions(+), 3 deletions(-)
> >
> > --- a/drivers/hv/connection.c   2015-02-03 10:58:51.751752450 -0800
> > +++ b/drivers/hv/connection.c   2015-02-04 14:59:51.636194383 -0800
> > @@ -64,6 +64,15 @@ static __u32 vmbus_get_next_version(__u3
> > }
> >  }
> >
> > +void vmbus_get_monitor_pages(unsigned long *int_page,
> > +unsigned long monitor_pages[2])
> > +{
> 
> Too weird to be a public interface in my opinion.
> 
> > +   *int_page = (unsigned long)vmbus_connection.int_page;
> > +   monitor_pages[0] = (unsigned long)vmbus_connection.monitor_pages[0];
> > +   monitor_pages[1] = (unsigned long)vmbus_connection.monitor_pages[1];
> > +}
> > +EXPORT_SYMBOL_GPL(vmbus_get_monitor_pages);
> > +
> >  static int vmbus_negotiate_version(struct vmbus_channel_msginfo *msginfo,
> > __u32 version)
> >  {
> > @@ -347,10 +356,7 @@ static void process_chn_event(u32 relid)
> > else
> > bytes_to_read = 0;
> > } while (read_state && (bytes_to_read != 0));
> > -   } else {
> > -   pr_err("no channel callback for relid - %u\n", relid);
> > }
> > -
> >  }
> 
> This change wasn't mentioned in your commit msg.

Was in earlier commit. Need to prevent driver from logging error
when device is not claimed directly by existing kernel devices.