[dpdk-dev] How to know corresponding device from port number
> [BR] Firstly, to identify the ring PMD's vs the ethernet device PMDs you can > use the information in the rte_eth_dev structure. For each device x, (0 <= x > <=5), if you check rte_eth_devices[x], the ring pmd's will have a NULL driver > pointer and the pci address given in the pci_dev structure will be all-zeros. > As for distinguishing two different ring ethdevs from each other, I'm not > aware of any way to do this, they will just have different eth_dev indexes. Thanks, I understand there is no way to distinguish rings. It will be a problem in following case. If someone wants to implement forwarding application that receives packets from ETH_A and send those to ETH_B. Also above application is split to 3 processes like following. [ETH_A]-->Process_A --> [Ring_A] --> Process_B --> [Ring_B] --> Process_C --> [ETH_B] (All 3 processes are implemented using PMD) At present, to implement Process_B might be difficult or tricky because ring can't be distinguished. I guess all virtual eth device like ring and pcap should have MAC address. And It should be possible to specify MAC address from command line. If so, DPDK application can distinguish all ports using MAC address even if port corresponds virtual eth device. Thanks, Tetsuya Mukawa
[dpdk-dev] Regarding VM live migration with SRIOV
Hi, Let me be more specific. Does DPDK support hot plugin/plugout of PCI devices ? What typically needs to be done if this is to be achieved inside an application. Typically, the NIC PF or VF appears to the DPDK application as a PCI device which is probed at startup. Now what happens if I insert a new VF dynamically and want to use it inside the DPDK application (while it is already running), how should this typically be done ? [hotplugin] And what happens if the DPDK application is in control of a PCI device and that PCI device is suddenly removed ? How can the application detect this and stop doing data transfer on this and sort of unload it ? [hotplugout] If the above can be coded inside the DPDK app, then we can think of live VM migration with SRIOV -- just hotplugin and plugout the VF's. Regards -Prashant -Original Message- From: dev [mailto:dev-boun...@dpdk.org] On Behalf Of Prashant Upadhyaya Sent: Monday, November 25, 2013 7:32 PM To: dev at dpdk.org Subject: [dpdk-dev] Regarding VM live migration with SRIOV Hi guys, I have a VM on top of QEMU/KVM hypervisor. Guest and Host are both Fedora 18. I am using 82599 NIC with SRIOV based VF's in the VM. In VM I am running a DPDK based application which uses the VF. Now I have to do a live migration of the running VM from one physical machine to the other. Right, so has anybody tried it before in the above environment. If yes, would love to hear from you and learn from your experience of doing it instead of making the same mistakes and learning the hard way. Regards -Prashant === Please refer to http://www.aricent.com/legal/email_disclaimer.html for important disclosures regarding this electronic communication. === === Please refer to http://www.aricent.com/legal/email_disclaimer.html for important disclosures regarding this electronic communication. ===
[dpdk-dev] Regarding VM live migration with SRIOV
Hi Stephen, The rte_eal_pci_probe is typically called at the startup. Now let's say a DPDK application is running with a PCI device (doing tx and rx) and I remove that PCI device underneath (hot plugout) So how does the application now know that the device is gone ? Is it that rte_eal_pci_probe should be called periodically from, let's say, the slow control path of the DPDK application ? Regards -Prashant -Original Message- From: Stephen Hemminger [mailto:step...@networkplumber.org] Sent: Wednesday, November 27, 2013 10:16 AM To: Prashant Upadhyaya Cc: dev at dpdk.org Subject: Re: [dpdk-dev] Regarding VM live migration with SRIOV On Wed, 27 Nov 2013 10:09:09 +0530 Prashant Upadhyaya wrote: > Hi, > > Let me be more specific. > Does DPDK support hot plugin/plugout of PCI devices ? > What typically needs to be done if this is to be achieved inside an > application. > > Typically, the NIC PF or VF appears to the DPDK application as a PCI device > which is probed at startup. > Now what happens if I insert a new VF dynamically and want to use it > inside the DPDK application (while it is already running), how should > this typically be done ? [hotplugin] And what happens if the DPDK > application is in control of a PCI device and that PCI device is > suddenly removed ? How can the application detect this and stop doing > data transfer on this and sort of unload it ? [hotplugout] > > If the above can be coded inside the DPDK app, then we can think of live VM > migration with SRIOV -- just hotplugin and plugout the VF's. > > Regards > -Prashant > The current implementation does look like it supports hotplug. All devices are discovered during rte_eal_pci_probe. === Please refer to http://www.aricent.com/legal/email_disclaimer.html for important disclosures regarding this electronic communication. ===
[dpdk-dev] Cannot run l3fwd with the problem of " nb_tx_queues"
Hi, I also have the same question in l3fwd example. I guess may DPDK don't support the chip-set so that we got this error msg. In my system, I have 2 kinds of chip-set, 82580 & 82574L. If I use l3fwd -c 3 -n 1 -- -p 3 --config '(0,0,0),(1,0,1)' in 82574L, it will get the error msg. But when I change my card with 82580 and use the same command to execute DPDK l3fwd. It's working. So, maybe the root cause is the DPDK does not support your chip-set and send out this error msg. On 11/14/2013 04:31 PM, Jose Gavine Cueto wrote: > Hi, > > Your nic (port 0) is capable of 1 txq only see error: > > " port_id=0 nb_tx_queues=2 > 1" > > I think that caused the error. > > Cheers, > Pepe > > > On Thu, Nov 14, 2013 at 1:21 PM, yuxuan zhang gmail.com>wrote: > >> Hi Everyone: >> >> I meet a problem with running the DPDK1.3 example l3fwd in the VMware. I >> have two 82545EM NIC to run the l3fwd and the OS is Cent6.4. But l3fwd >> cannot be configured port correctly .I start the l3fwd with following >> command: >> >> ./build/l3fwd -c 0x3 -n 2 -- -p 0x3 --config="(0,0,0),(1,0,1)" >> >> However, I get following errors with debug info: >> >> Initializing port 0 ... Creating queues: nb_rxq=1 nb_txq=2... PMD: >> rte_eth_dev_configure: ethdev port_id=0 nb_tx_queues=2 > 1 >> EAL: Error - exiting with code: 1 >> Cause: Cannot configure device: err=-22, port=0 >> >> And I have known that the reason for this problem produced in the >> **rte_eth_dev_configure function**. However, I do not kown how to fix it. >> >> I would really appreciate If someone could provide some help to solve this >> issue. >> > > -- Best Regards, Zachary ? This email may contain confidential information. Please do not use or disclose it in any way and delete it if you are not the intended recipient.
[dpdk-dev] Cannot run l3fwd with the problem of " nb_tx_queues"
Hello, 27/11/2013 11:58, Zachary.Jen (???) : 6305 : > I also have the same question in l3fwd example. > I guess may DPDK don't support the chip-set so that we got this error msg. > > In my system, I have 2 kinds of chip-set, 82580 & 82574L. > If I use l3fwd -c 3 -n 1 -- -p 3 --config '(0,0,0),(1,0,1)' in 82574L, > it will get the error msg. > But when I change my card with 82580 and use the same command to execute > DPDK l3fwd. > It's working. > > So, maybe the root cause is the DPDK does not support your chip-set and > send out this error msg. No, it's only a problem of hardware queues availability. 82574 has only 1 queue. -- Thomas
[dpdk-dev] Cannot run l3fwd with the problem of " nb_tx_queues"
Hi yuxuan, Yes, by experience I've successfully run dpdk with a virtualbox vm using an e1000 NIC. I'm not totally sure about the capabilities of the e1000 NIC but based on the error you've mentioned, it seems that it can only support 1 TX queue. "Now, I can use only one logic core to run the app and the performance of l3fwd is undesirable." This shows that dpdk can run successfully with 1 TX queue only because probably your e1000 NIC only supports 1 queue. Cheers, Pepe On Wed, Nov 27, 2013 at 6:58 PM, Zachary.Jen (???) : 6305 < Zachary.Jen at cas-well.com> wrote: > Hi, > > I also have the same question in l3fwd example. > I guess may DPDK don't support the chip-set so that we got this error msg. > > In my system, I have 2 kinds of chip-set, 82580 & 82574L. > If I use l3fwd -c 3 -n 1 -- -p 3 --config '(0,0,0),(1,0,1)' in 82574L, > it will get the error msg. > But when I change my card with 82580 and use the same command to execute > DPDK l3fwd. > It's working. > > So, maybe the root cause is the DPDK does not support your chip-set and > send out this error msg. > > On 11/14/2013 04:31 PM, Jose Gavine Cueto wrote: > > Hi, > > > > Your nic (port 0) is capable of 1 txq only see error: > > > > " port_id=0 nb_tx_queues=2 > 1" > > > > I think that caused the error. > > > > Cheers, > > Pepe > > > > > > On Thu, Nov 14, 2013 at 1:21 PM, yuxuan zhang >wrote: > > > >> Hi Everyone: > >> > >> I meet a problem with running the DPDK1.3 example l3fwd in the VMware. I > >> have two 82545EM NIC to run the l3fwd and the OS is Cent6.4. But l3fwd > >> cannot be configured port correctly .I start the l3fwd with following > >> command: > >> > >> ./build/l3fwd -c 0x3 -n 2 -- -p 0x3 --config="(0,0,0),(1,0,1)" > >> > >> However, I get following errors with debug info: > >> > >> Initializing port 0 ... Creating queues: nb_rxq=1 nb_txq=2... PMD: > >> rte_eth_dev_configure: ethdev port_id=0 nb_tx_queues=2 > 1 > >> EAL: Error - exiting with code: 1 > >> Cause: Cannot configure device: err=-22, port=0 > >> > >> And I have known that the reason for this problem produced in the > >> **rte_eth_dev_configure function**. However, I do not kown how to fix > it. > >> > >> I would really appreciate If someone could provide some help to solve > this > >> issue. > >> > > > > > > -- > Best Regards, > Zachary > ? This email may contain > confidential information. Please do not use or disclose it in any way and > delete it if you are not the intended recipient. > -- To stop learning is like to stop loving.
[dpdk-dev] Cannot run l3fwd with the problem of " nb_tx_queues"
Hi, The problem is that E1000 NIC only support 1 TX queue ,the dpdk will change the max TX queue number in eth_em_infos_get() function the when you call the rte_eth_dev_configure() function .If you want to run l3fwd program on e1000 NIC correctly, you should modify the l3fwd by using only one TX queue per lcore. 2013/11/27 Jose Gavine Cueto > Hi yuxuan, > > Yes, by experience I've successfully run dpdk with a virtualbox vm using an > e1000 NIC. I'm not totally sure about the capabilities of the e1000 NIC > but based on the error you've mentioned, it seems that it can only support > 1 TX queue. > > "Now, I can use only one logic core to run the app and the performance of > l3fwd is undesirable." > > This shows that dpdk can run successfully with 1 TX queue only because > probably your e1000 NIC only supports 1 queue. > > Cheers, > Pepe > > > On Wed, Nov 27, 2013 at 6:58 PM, Zachary.Jen (??) : 6305 < > Zachary.Jen at cas-well.com> wrote: > > > Hi, > > > > I also have the same question in l3fwd example. > > I guess may DPDK don't support the chip-set so that we got this error > msg. > > > > In my system, I have 2 kinds of chip-set, 82580 & 82574L. > > If I use l3fwd -c 3 -n 1 -- -p 3 --config '(0,0,0),(1,0,1)' in 82574L, > > it will get the error msg. > > But when I change my card with 82580 and use the same command to execute > > DPDK l3fwd. > > It's working. > > > > So, maybe the root cause is the DPDK does not support your chip-set and > > send out this error msg. > > > > On 11/14/2013 04:31 PM, Jose Gavine Cueto wrote: > > > Hi, > > > > > > Your nic (port 0) is capable of 1 txq only see error: > > > > > > " port_id=0 nb_tx_queues=2 > 1" > > > > > > I think that caused the error. > > > > > > Cheers, > > > Pepe > > > > > > > > > On Thu, Nov 14, 2013 at 1:21 PM, yuxuan zhang > >wrote: > > > > > >> Hi Everyone: > > >> > > >> I meet a problem with running the DPDK1.3 example l3fwd in the > VMware. I > > >> have two 82545EM NIC to run the l3fwd and the OS is Cent6.4. But l3fwd > > >> cannot be configured port correctly .I start the l3fwd with following > > >> command: > > >> > > >> ./build/l3fwd -c 0x3 -n 2 -- -p 0x3 --config="(0,0,0),(1,0,1)" > > >> > > >> However, I get following errors with debug info: > > >> > > >> Initializing port 0 ... Creating queues: nb_rxq=1 nb_txq=2... > PMD: > > >> rte_eth_dev_configure: ethdev port_id=0 nb_tx_queues=2 > 1 > > >> EAL: Error - exiting with code: 1 > > >> Cause: Cannot configure device: err=-22, port=0 > > >> > > >> And I have known that the reason for this problem produced in the > > >> **rte_eth_dev_configure function**. However, I do not kown how to fix > > it. > > >> > > >> I would really appreciate If someone could provide some help to solve > > this > > >> issue. > > >> > > > > > > > > > > -- > > Best Regards, > > Zachary > > ???C???Y???K???N?? > > This email may contain > > confidential information. Please do not use or disclose it in any way and > > delete it if you are not the intended recipient. > > > > > > -- > To stop learning is like to stop loving. >
[dpdk-dev] Sporadic errors while initializing NICs in example applications, dpdk-1.5.0r1
Looks like I finally found the reason. After applying this patch I can
no longer reproduce the error.
diff --git a/lib/librte_pmd_ixgbe/ixgbe/ixgbe_82599.c
b/lib/librte_pmd_ixgbe/ixgbe/ixgbe_82599.c
index db07789..5f825fa 100644
--- a/lib/librte_pmd_ixgbe/ixgbe/ixgbe_82599.c
+++ b/lib/librte_pmd_ixgbe/ixgbe/ixgbe_82599.c
@@ -2347,7 +2347,7 @@ s32 ixgbe_reset_pipeline_82599(struct ixgbe_hw *hw)
/* Write AUTOC register with toggled LMS[2] bit and Restart_AN */
IXGBE_WRITE_REG(hw, IXGBE_AUTOC, autoc_reg ^
IXGBE_AUTOC_LMS_1G_AN);
/* Wait for AN to leave state 0 */
- for (i = 0; i < 10; i++) {
+ for (i = 0; i < 100; i++) {
msec_delay(4);
anlp1_reg = IXGBE_READ_REG(hw, IXGBE_ANLP1);
if (anlp1_reg & IXGBE_ANLP1_AN_STATE_MASK)
On 11/22/2013 04:48 PM, Thomas Monjalon wrote:
> Hello,
>
> 22/11/2013 13:29, Dmitry Vyal :
>> EAL: PCI device :02:00.0 on NUMA socket -1
>> EAL: probe driver: 8086:10fb rte_ixgbe_pmd
>> EAL: PCI memory mapped at 0x7f6b83687000
>> EAL: PCI memory mapped at 0x7f6b83683000
>> EAL: PCI device :02:00.1 on NUMA socket -1
>> EAL: probe driver: 8086:10fb rte_ixgbe_pmd
>> EAL: PCI memory mapped at 0x7f6b83663000
>> EAL: PCI memory mapped at 0x7f6b8365f000
> [...]
>> EAL: Error - exiting with code: 1
>> Cause: No Ethernet ports - bye
>>
>> Any ideas how to investigate this?
> Could you try this patch in order to see the root cause of your issue ?
>
> --- a/lib/librte_pmd_ixgbe/ixgbe_logs.h
> +++ b/lib/librte_pmd_ixgbe/ixgbe_logs.h
> @@ -34,41 +34,44 @@
> #ifndef _IXGBE_LOGS_H_
> #define _IXGBE_LOGS_H_
>
> +#define PMD_LOG(level, fmt, args...) \
> + RTE_LOG(level, PMD, "%s(): " fmt "\n", __func__, ##args)
> +
> #ifdef RTE_LIBRTE_IXGBE_DEBUG_INIT
> -#define PMD_INIT_LOG(level, fmt, args...) \
> - RTE_LOG(level, PMD, "%s(): " fmt "\n", __func__, ## args)
> +#define PMD_INIT_LOG(level, fmt, args...) PMD_LOG(level, fmt, ##args)
> #define PMD_INIT_FUNC_TRACE() PMD_INIT_LOG(DEBUG, " >>")
> #else
> -#define PMD_INIT_LOG(level, fmt, args...) do { } while(0)
> +#define PMD_INIT_LOG(level, fmt, args...) \
> + (void)(RTE_LOG_##level <= RTE_LOG_ERR ? PMD_LOG(level, fmt, ##args) : 0)
> #define PMD_INIT_FUNC_TRACE() do { } while(0)
> #endif
>
> #ifdef RTE_LIBRTE_IXGBE_DEBUG_RX
> -#define PMD_RX_LOG(level, fmt, args...) \
> - RTE_LOG(level, PMD, "%s(): " fmt "\n", __func__, ## args)
> +#define PMD_RX_LOG(level, fmt, args...) PMD_LOG(level, fmt, ##args)
> #else
> -#define PMD_RX_LOG(level, fmt, args...) do { } while(0)
> +#define PMD_RX_LOG(level, fmt, args...) \
> + (void)(RTE_LOG_##level <= RTE_LOG_ERR ? PMD_LOG(level, fmt, ##args) : 0)
> #endif
>
> #ifdef RTE_LIBRTE_IXGBE_DEBUG_TX
> -#define PMD_TX_LOG(level, fmt, args...) \
> - RTE_LOG(level, PMD, "%s(): " fmt "\n", __func__, ## args)
> +#define PMD_TX_LOG(level, fmt, args...) PMD_LOG(level, fmt, ##args)
> #else
> -#define PMD_TX_LOG(level, fmt, args...) do { } while(0)
> +#define PMD_TX_LOG(level, fmt, args...) \
> + (void)(RTE_LOG_##level <= RTE_LOG_ERR ? PMD_LOG(level, fmt, ##args) : 0)
> #endif
>
> #ifdef RTE_LIBRTE_IXGBE_DEBUG_TX_FREE
> -#define PMD_TX_FREE_LOG(level, fmt, args...) \
> - RTE_LOG(level, PMD, "%s(): " fmt "\n", __func__, ## args)
> +#define PMD_TX_FREE_LOG(level, fmt, args...) PMD_LOG(level, fmt, ##args)
> #else
> -#define PMD_TX_FREE_LOG(level, fmt, args...) do { } while(0)
> +#define PMD_TX_FREE_LOG(level, fmt, args...) \
> + (void)(RTE_LOG_##level <= RTE_LOG_ERR ? PMD_LOG(level, fmt, ##args) : 0)
> #endif
>
> #ifdef RTE_LIBRTE_IXGBE_DEBUG_DRIVER
> -#define PMD_DRV_LOG(level, fmt, args...) \
> - RTE_LOG(level, PMD, "%s(): " fmt "\n", __func__, ## args)
> +#define PMD_DRV_LOG(level, fmt, args...) PMD_LOG(level, fmt, ##args)
> #else
> -#define PMD_DRV_LOG(level, fmt, args...) do { } while(0)
> +#define PMD_DRV_LOG(level, fmt, args...) \
> + (void)(RTE_LOG_##level <= RTE_LOG_ERR ? PMD_LOG(level, fmt, ##args) : 0)
> #endif
>
> #endif /* _IXGBE_LOGS_H_ */
>
[dpdk-dev] Sporadic errors while initializing NICs in example applications, dpdk-1.5.0r1
I had exactly same problem and fixed it with same patch since release
1.4. But somehow it stopped to appear even without the patch. I have
no idea what I have done since both the kernel and the driver have
been updated during these days.
On Wed, Nov 27, 2013 at 4:06 PM, Dmitry Vyal wrote:
> Looks like I finally found the reason. After applying this patch I can no
> longer reproduce the error.
>
> diff --git a/lib/librte_pmd_ixgbe/ixgbe/ixgbe_82599.c
> b/lib/librte_pmd_ixgbe/ixgbe/ixgbe_82599.c
> index db07789..5f825fa 100644
> --- a/lib/librte_pmd_ixgbe/ixgbe/ixgbe_82599.c
> +++ b/lib/librte_pmd_ixgbe/ixgbe/ixgbe_82599.c
> @@ -2347,7 +2347,7 @@ s32 ixgbe_reset_pipeline_82599(struct ixgbe_hw *hw)
> /* Write AUTOC register with toggled LMS[2] bit and Restart_AN */
> IXGBE_WRITE_REG(hw, IXGBE_AUTOC, autoc_reg ^ IXGBE_AUTOC_LMS_1G_AN);
> /* Wait for AN to leave state 0 */
> - for (i = 0; i < 10; i++) {
> + for (i = 0; i < 100; i++) {
> msec_delay(4);
> anlp1_reg = IXGBE_READ_REG(hw, IXGBE_ANLP1);
> if (anlp1_reg & IXGBE_ANLP1_AN_STATE_MASK)
>
> On 11/22/2013 04:48 PM, Thomas Monjalon wrote:
>>
>> Hello,
>>
>> 22/11/2013 13:29, Dmitry Vyal :
>>>
>>> EAL: PCI device :02:00.0 on NUMA socket -1
>>> EAL: probe driver: 8086:10fb rte_ixgbe_pmd
>>> EAL: PCI memory mapped at 0x7f6b83687000
>>> EAL: PCI memory mapped at 0x7f6b83683000
>>> EAL: PCI device :02:00.1 on NUMA socket -1
>>> EAL: probe driver: 8086:10fb rte_ixgbe_pmd
>>> EAL: PCI memory mapped at 0x7f6b83663000
>>> EAL: PCI memory mapped at 0x7f6b8365f000
>>
>> [...]
>>>
>>> EAL: Error - exiting with code: 1
>>> Cause: No Ethernet ports - bye
>>>
>>> Any ideas how to investigate this?
>>
>> Could you try this patch in order to see the root cause of your issue ?
>>
>> --- a/lib/librte_pmd_ixgbe/ixgbe_logs.h
>> +++ b/lib/librte_pmd_ixgbe/ixgbe_logs.h
>> @@ -34,41 +34,44 @@
>> #ifndef _IXGBE_LOGS_H_
>> #define _IXGBE_LOGS_H_
>> +#define PMD_LOG(level, fmt, args...) \
>> + RTE_LOG(level, PMD, "%s(): " fmt "\n", __func__, ##args)
>> +
>> #ifdef RTE_LIBRTE_IXGBE_DEBUG_INIT
>> -#define PMD_INIT_LOG(level, fmt, args...) \
>> - RTE_LOG(level, PMD, "%s(): " fmt "\n", __func__, ## args)
>> +#define PMD_INIT_LOG(level, fmt, args...) PMD_LOG(level, fmt, ##args)
>> #define PMD_INIT_FUNC_TRACE() PMD_INIT_LOG(DEBUG, " >>")
>> #else
>> -#define PMD_INIT_LOG(level, fmt, args...) do { } while(0)
>> +#define PMD_INIT_LOG(level, fmt, args...) \
>> + (void)(RTE_LOG_##level <= RTE_LOG_ERR ? PMD_LOG(level, fmt,
>> ##args) : 0)
>> #define PMD_INIT_FUNC_TRACE() do { } while(0)
>> #endif
>> #ifdef RTE_LIBRTE_IXGBE_DEBUG_RX
>> -#define PMD_RX_LOG(level, fmt, args...) \
>> - RTE_LOG(level, PMD, "%s(): " fmt "\n", __func__, ## args)
>> +#define PMD_RX_LOG(level, fmt, args...) PMD_LOG(level, fmt, ##args)
>> #else
>> -#define PMD_RX_LOG(level, fmt, args...) do { } while(0)
>> +#define PMD_RX_LOG(level, fmt, args...) \
>> + (void)(RTE_LOG_##level <= RTE_LOG_ERR ? PMD_LOG(level, fmt,
>> ##args) : 0)
>> #endif
>> #ifdef RTE_LIBRTE_IXGBE_DEBUG_TX
>> -#define PMD_TX_LOG(level, fmt, args...) \
>> - RTE_LOG(level, PMD, "%s(): " fmt "\n", __func__, ## args)
>> +#define PMD_TX_LOG(level, fmt, args...) PMD_LOG(level, fmt, ##args)
>> #else
>> -#define PMD_TX_LOG(level, fmt, args...) do { } while(0)
>> +#define PMD_TX_LOG(level, fmt, args...) \
>> + (void)(RTE_LOG_##level <= RTE_LOG_ERR ? PMD_LOG(level, fmt,
>> ##args) : 0)
>> #endif
>> #ifdef RTE_LIBRTE_IXGBE_DEBUG_TX_FREE
>> -#define PMD_TX_FREE_LOG(level, fmt, args...) \
>> - RTE_LOG(level, PMD, "%s(): " fmt "\n", __func__, ## args)
>> +#define PMD_TX_FREE_LOG(level, fmt, args...) PMD_LOG(level, fmt, ##args)
>> #else
>> -#define PMD_TX_FREE_LOG(level, fmt, args...) do { } while(0)
>> +#define PMD_TX_FREE_LOG(level, fmt, args...) \
>> + (void)(RTE_LOG_##level <= RTE_LOG_ERR ? PMD_LOG(level, fmt,
>> ##args) : 0)
>> #endif
>> #ifdef RTE_LIBRTE_IXGBE_DEBUG_DRIVER
>> -#define PMD_DRV_LOG(level, fmt, args...) \
>> - RTE_LOG(level, PMD, "%s(): " fmt "\n", __func__, ## args)
>> +#define PMD_DRV_LOG(level, fmt, args...) PMD_LOG(level, fmt, ##args)
>> #else
>> -#define PMD_DRV_LOG(level, fmt, args...) do { } while(0)
>> +#define PMD_DRV_LOG(level, fmt, args...) \
>> + (void)(RTE_LOG_##level <= RTE_LOG_ERR ? PMD_LOG(level, fmt,
>> ##args) : 0)
>> #endif
>> #endif /* _IXGBE_LOGS_H_ */
>>
>
[dpdk-dev] Sporadic errors while initializing NICs in example applications, dpdk-1.5.0r1
27/11/2013 15:06, Dmitry Vyal :
> Looks like I finally found the reason. After applying this patch I can
> no longer reproduce the error.
>
> --- a/lib/librte_pmd_ixgbe/ixgbe/ixgbe_82599.c
> +++ b/lib/librte_pmd_ixgbe/ixgbe/ixgbe_82599.c
> /* Wait for AN to leave state 0 */
> - for (i = 0; i < 10; i++) {
> + for (i = 0; i < 100; i++) {
> msec_delay(4);
> anlp1_reg = IXGBE_READ_REG(hw, IXGBE_ANLP1);
> if (anlp1_reg & IXGBE_ANLP1_AN_STATE_MASK)
It's probably due to a frequency scaling.
The timer based is initialized when DPDK initialize
and the CPU can change its frequency, breaking next timers.
The fix is to control the CPU frequency.
Please try this, without your patch:
for g in /sys/devices/system/cpu/*/cpufreq/scaling_governor; do echo
performance >$g; done
The right fix for applications (examples and testpmd included)
could be to call rte_power_init(). Patches are welcomed.
Thank you for your analysis
--
Thomas
[dpdk-dev] Increasing number of txd and rxd from 256 to 1024 for virtio-net-pmd-1.1
Can you share your virtio driver with me ?
Do you mean to create multiple queues, each has 256 txd/rxd ? The packets
could be stored into the freeslots in those queues. But how can the virtio
pmd codes feed the slots down to the hardware to deliver them ?
The other question is that I was using vhost-net on the KVM host. This
supposed to be transparent to the DPDK + virtio pmd codes. But this cause
problem in the packet delivery ?
Thanks
On Tue, Nov 26, 2013 at 10:26 PM, Stephen Hemminger <
stephen at networkplumber.org> wrote:
> On Tue, 26 Nov 2013 21:15:02 -0800
> James Yu wrote:
>
> > Running one directional traffic from Spirent traffic generator to l2fwd
> > running inside a guest OS on a RHEL 6.2 KVM host, I encountered
> performance
> > issue and need to increase the number of rxd and txd from 256 to 1024.
> > There was not enough freeslots for packets to be transmitted in this
> routine
> > virtio_send_packet(){
> >
> > if (tq->freeslots < nseg + 1) {
> > return -1;
> > }
> >
> > }
> >
> > How do I solve the performance issue by one of the following
> > 1. increase the number of rxd and txd from 256 to 1024
> > This should prevent packets could not be stored into the ring due
> > to lack of freeslots. But l2fwd fails to run and indicate the number must
> > be equal to 256.
> > 2. increase the MAX_PKT_BURST
> > But this is not ideal since it will increase the delay while
> > improving the throughput
> > 3. other mechanism that you know can improve it ?
> > Is there any other approach to have enough freeslots to store the
> > packets before passing down to PCI ?
> >
> >
> > Thanks
> >
> > James
> >
> >
> > This is the performance numbers I measured on the l2fwd printout for the
> > receiving part. I added codes inside l2fwd to do tx part.
> >
>
> > vhost-net is enabled on KVM host, # of cache buffer 4096, Ubuntu 12.04.3
> > LTS (3.2.0-53-generic); kvm 1.2.0, libvirtd: 0.9.8
> > 64 Bytes/pkt from Spirent @ 223k pps, running test for 10 seconds.
> >
>
> > DPDK 1.3 + virtio + 256 txd/rxd + nice -19 priority (l2fwd, guest kvm
> > process)
> > bash command: nice -n -19
> > /root/dpdk/dpdk-1.3.1r2/examples/l2fwd/build/l2fwd -c 3 -n 1 -b
> 000:00:03.0
> > -b 000:00:07.0 -b 000:00:0a.0 -b 000:00:09.0 -d
> > /root/dpdk/virtio-net-pmd-1.1/librte_pmd_virtio.so -- -q 1 -p 1
> >
>
> > Spirent -> l2fwd (receiving 10G) (RX on KVM guest)
> > MAX_PKT_BURST 10seconds (<1% loss) Packets Per Second
> >
> ---
> > 32 74k pps
> > 64 80k pps
> > 128 126kpps
> > 256 133kpps
> >
> > l2fw -> Spirent (10G port) (transmitting) (using one-directional one port
> > (port 0) setup)
> > MAX_PKT_BURST < 1% packet loss
> > 32 88kpp
> >
> >
> > **
> > The same test run on e1000 ports
> >
> >
>
> > DPDK 1.3 + e1000 + 1024 txd/rxd + nice -19 priority (l2fwd, guest kvm
> > process)
> > bash command: nice -n -19
> > /root/dpdk/dpdk-1.3.1r2/examples/l2fwd/build/l2fwd -c 3 -n 1 -b
> 000:00:03.0
> > -b 000:00:07.0 -b 000:00:0a.0 -b 000:00:09.0 -- -q 1 -p 1
> >
>
> > Spirent -> l2fwd (RECEIVING 10G)
> > MAX_PKT_BURST <= 1% packet loss
> > 32 110k pps
> >
> > l2fw -> Spirent (10G port) (TRANSMITTING) (using one-directional one port
> > (port 0) setup)
> > MAX_PKT_BURST pkts transmitted on l2fwd
> > 32171k pps (0% dropped)
> > 240 203k pps (6% dropped, 130k pps received
> on
> > eth6 (assumed on Spirent)) **
> > **: not enough freeslots in tx ring
> > ==> this indicate the effects of small txd/rxd (256) when more traffic is
> > generated, the packets can not
> > be sent due to lack of freeslots in tx ring. I guess this is the
> > symptom occurs in the virtio_net
>
> The number of slots with virtio is a parameter negotiated with the host.
> So unless the host (KVM) gives the device more slots, then it won't work.
> I have a better virtio driver and one of the features being added is
> multiqueue
> and merged TX buffer support which would give a bigger queue.
>
>
