[dpdk-dev] generic load balancing
Prashant, I assume your use case is not of one IP/UDP/TCP - or if it is, you are dealing with a single tuple that is not evenly distributed. You have a few options with the NIC that I can think of. 1) Use a single core to RX each port's frames and use your own software solution to RR to worker rings. There is an example of this in the Load Balancer sample application. 2) If your packets/frames have an evenly distributed field in the first 64 bytes of the frame, you can use the 2 byte match feature of flow director to send to different queues (with multiple match signatures). This will give even distribution, but not round robin behavior. 3) Modify the RSS redirection table for the NIC in the order you desire. I am unsure how often this can happen, or if there are performance issues with reprogramming it. Definitely would need some experimentation. What is it you are trying to achieve with Round Robin? A distribution of packets to multiple cores for processing, or something else? Without knowing the use case, my main suggestion is to use the LB sample application - that way you can distribute in any way you please. Thanks, Bryan Benson Original message From: Prashant Upadhyaya Date:12/04/2013 9:30 PM (GMT-08:00) To: Stephen Hemminger Cc: dev at dpdk.org Subject: Re: [dpdk-dev] generic load balancing Hi Stepher, The awfulness depends upon the 'usecase' I have eg. a usecase where I want this roundrobin behaviour. I just want the NIC to give me a facility to use this. Regards -Prashant -Original Message- From: Stephen Hemminger [mailto:step...@networkplumber.org] Sent: Thursday, December 05, 2013 10:25 AM To: Prashant Upadhyaya Cc: Fran?ois-Fr?d?ric Ozog; Michael Quicquaro; dev at dpdk.org Subject: Re: [dpdk-dev] generic load balancing Round robin would actually be awful for any protocol because it would cause out of order packets. That is why flow based algorithms like flow director and RSS work much better. On Wed, Dec 4, 2013 at 8:31 PM, Prashant Upadhyaya wrote: > Hi, > > It's a real pity that Intel 82599 NIC (and possibly others) don't have a > simple round robin scheduling of packets on the configured queues. > > I have requested Intel earlier, and using this forum requesting again -- > please please put this facility in the NIC that if I drop N queues there and > configure the NIC for some round robin scheduling on queues, then NIC should > simply put the received packets one by one on queue 1, then on > queue2,,then on queueN, and then back on queue 1. > The above is very useful in lot of load balancing cases. > > Regards > -Prashant > > > -Original Message- > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Fran?ois-Fr?d?ric > Ozog > Sent: Thursday, December 05, 2013 2:35 AM > To: 'Michael Quicquaro' > Cc: dev at dpdk.org > Subject: Re: [dpdk-dev] generic load balancing > > Hi, > > As far as I can tell, this is really hardware dependent. Some hash functions > allow uplink and downlink packets of the same "session" to go to the same > queue (I know Chelsio can do this). > > For the Intel card, you may find what you want in: > http://www.intel.com/content/www/us/en/ethernet-controllers/82599-10-g > be-con > troller-datasheet.html > > Other cards require NDA or other agreements to get details of RSS. > > If you have a performance problem, may I suggest you use kernel 3.10 then > monitor system activity with "perf" command. For instance you can start with > "perf top -a" this will give you nice information. Then your creativity will > do the rest ;-) You may be surprised what comes on the top hot points... > (the most unexpected hot function I found here was Linux syscall > gettimeofday!!!) > > Fran?ois-Fr?d?ric > >> -Message d'origine- >> De : dev [mailto:dev-bounces at dpdk.org] De la part de Michael >> Quicquaro Envoy? : mercredi 4 d?cembre 2013 18:53 ? : dev at dpdk.org Objet : >> [dpdk-dev] generic load balancing >> >> Hi all, >> I am writing a dpdk application that will receive packets from one >> interface and process them. It does not forward packets in the > traditional >> sense. However, I do need to process them at full line rate and >> therefore need more than one core. The packets can be somewhat >> generic in nature > and >> can be nearly identical (especially at the beginning of the packet). >> I've used the rxonly function of testpmd as a model. >> >> I've run into problems in processing a full line rate of data since >> the nature of the data causes all the data to be presented to only one core. > I >> get a large percentage of dropped packets (shows up as Rx-Errors in >> "port >> stats") because of this. I've tried modifying the data so that >> packets have different UDP ports and that seems to work when I use >> --rss-udp >> >> My questions are: >> 1) Is there a way to configure RSS so that it alternates packets to >> all configured cores regardless of the packet data? >> >> 2) Where i
[dpdk-dev] generic load balancing
Prashant, 1) I thought the same, but I was pleasantly surprised at how much a single core can RX and distribute (from a single 10G port). It was a while back, but in my experimentation with well distributed incoming flows, I found nearly identical bottleneck points between polling using one core and using RSS for multiple cores. 2) Here are a few links - The flex byte offset looks like it is global for each port, but the value it matches is part of each in the signature, as is the output port. Adding the sig: http://dpdk.org/doc/api/rte__ethdev_8h.html#a4ef515ffe18b57bed5493bcea90f16d7 Filter definition: http://dpdk.org/doc/api/structrte__fdir__filter.html - Now, I have not yet used the flow director features, so YMMV. Also, I vaguely remember hearing a bit about a performance issue when using more than 4 output queues, due to the number of PCI-e lanes. I don't recall the exact details, so if you see something like that when using more than 4 queues, don't be surprised (but let us know what you find!). I hope this helps, - Bryan From: Prashant Upadhyaya [prashant.upadhy...@aricent.com] Sent: Thursday, December 05, 2013 6:16 AM To: Benson, Bryan; Stephen Hemminger Cc: dev at dpdk.org Subject: RE: [dpdk-dev] generic load balancing Hi Bryan, Regarding your 1st point, the single core becomes the rx bottleneck which is clearly not desirable. I am not sure regarding how to use the stuff you mentioned in 2nd point, is there some DPDK api which lets me configure this, kindly let me know. Regards -Prashant From: Benson, Bryan [mailto:bmben...@amazon.com] Sent: Thursday, December 05, 2013 1:14 PM To: Prashant Upadhyaya; Stephen Hemminger Cc: dev at dpdk.org Subject: RE: [dpdk-dev] generic load balancing Prashant, I assume your use case is not of one IP/UDP/TCP - or if it is, you are dealing with a single tuple that is not evenly distributed. You have a few options with the NIC that I can think of. 1) Use a single core to RX each port's frames and use your own software solution to RR to worker rings. There is an example of this in the Load Balancer sample application. 2) If your packets/frames have an evenly distributed field in the first 64 bytes of the frame, you can use the 2 byte match feature of flow director to send to different queues (with multiple match signatures). This will give even distribution, but not round robin behavior. 3) Modify the RSS redirection table for the NIC in the order you desire. I am unsure how often this can happen, or if there are performance issues with reprogramming it. Definitely would need some experimentation. What is it you are trying to achieve with Round Robin? A distribution of packets to multiple cores for processing, or something else? Without knowing the use case, my main suggestion is to use the LB sample application - that way you can distribute in any way you please. Thanks, Bryan Benson Original message From: Prashant Upadhyaya Date:12/04/2013 9:30 PM (GMT-08:00) To: Stephen Hemminger Cc: dev at dpdk.org<mailto:dev at dpdk.org> Subject: Re: [dpdk-dev] generic load balancing Hi Stepher, The awfulness depends upon the 'usecase' I have eg. a usecase where I want this roundrobin behaviour. I just want the NIC to give me a facility to use this. Regards -Prashant -Original Message- From: Stephen Hemminger [mailto:step...@networkplumber.org] Sent: Thursday, December 05, 2013 10:25 AM To: Prashant Upadhyaya Cc: Fran?ois-Fr?d?ric Ozog; Michael Quicquaro; dev at dpdk.org<mailto:dev at dpdk.org> Subject: Re: [dpdk-dev] generic load balancing Round robin would actually be awful for any protocol because it would cause out of order packets. That is why flow based algorithms like flow director and RSS work much better. On Wed, Dec 4, 2013 at 8:31 PM, Prashant Upadhyaya mailto:prashant.upadhyaya at aricent.com>> wrote: > Hi, > > It's a real pity that Intel 82599 NIC (and possibly others) don't have a > simple round robin scheduling of packets on the configured queues. > > I have requested Intel earlier, and using this forum requesting again -- > please please put this facility in the NIC that if I drop N queues there and > configure the NIC for some round robin scheduling on queues, then NIC should > simply put the received packets one by one on queue 1, then on > queue2,,then on queueN, and then back on queue 1. > The above is very useful in lot of load balancing cases. > > Regards > -Prashant > > > -Original Message- > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Fran?ois-Fr?d?ric > Ozog > Sent: Thursday, December 05, 2013 2:35 AM > To: 'Michael Quicquaro' > Cc: dev at dpdk.org<mailto:dev at dpdk.org> > Subject: Re: [dpdk-dev] generic load balancing > > Hi, > > As far as I can tell, this is re
[dpdk-dev] PCI device mapping to socket
All, Does anyone know of a way I can find out which socket a PCI device/bridge is tied up to? I have looked into dmidecode and lspci to no avail, but I may be missing something. We are looking at putting multiple NICs into a single dual socket server. This is so that I can tie specific NIC ports to the proper socket to take advantage of DDIO. Thank you, Bryan Benson Amazon Web Services
[dpdk-dev] RX checksum offloading
All,
Has anyone had any experience with hardware checksum offloads not working? It
appears that the combination of settings we are using in our application may be
the crux of the issue, as offloading test-pmd application in csum mode
correctly identifies bad packets via the ol_flags. I am a bit stumped in
tracking it down, any help is greatly appreciated!
We are using the 82599 NIC & DPDK 1.3 (with RSC disabled).
Settings:
/* Ethernet Configuration */
static struct rte_eth_conf port_conf = {
.rxmode = {
.split_hdr_size = 0,
.header_split = 0, /**< Header Split disabled */
.hw_ip_checksum = 1, /**< IP/UDP/TCP checksum offload enabled. */
.hw_vlan_filter = 0, /**< VLAN filtering disabled */
.jumbo_frame= 0, /**< Jumbo Frame Support disabled */
.hw_strip_crc = 1, /**< CRC stripped by hardware */
},
.rx_adv_conf = {
.rss_conf = {
.rss_key = NULL,
.rss_hf = 0x00,
},
},
.txmode = {
},
};
Thank you,
Bryan Benson
Amazon Web Services
[dpdk-dev] RX checksum offloading
All, The receive checksum issue seems to be caused by using a RX_FREE_THRESH value that is 32 or larger, as validated by the test-pmd application. I used 2 different packet types - bad IP checksum sent to port 0 and bad tcp checksum to port 1. The way I was sending the packets did not vary between the various tests. Below is a brief summary of the testing - additional gory text is attached. 0, 8, 16, 24, 28, 30, 31 are okay. Values above 32 are not, as tested with 32, 64 and 128. I will continue researching this - The version I used to test this is the vanilla version of DPDK 1.3, but with the RSC disable patch applied (helps when there are many ACKs). Thanks, Bryan Benson [bmbenson]~/1.3.1.1/DPDK% sudo ./x86_64-default-linuxapp-gcc/app/testpmd -c 0xFF00FF00 -n 4 -b :06:00.0 -- --portmask=0x3 --nb-cores=2 --enable-rx-cksum --disable-hw-vlan --disable-rss --crc-strip --rxd=1024 --txd=1024 -i ... testpmd> set fwd csum Set csum packet forwarding mode testpmd> start csum packet forwarding - CRC stripping enabled - packets/burst=16 nb forwarding cores=2 - nb forwarding ports=2 RX queues=1 - RX desc=1024 - RX free threshold=0 RX threshold registers: pthresh=8 hthresh=8 wthresh=4 TX queues=1 - TX desc=1024 - TX free threshold=0 TX threshold registers: pthresh=36 hthresh=0 wthresh=0 TX RS bit threshold=0 - TXQ flags=0x0 testpmd> stop Telling cores to stop... Waiting for lcores to finish... -- Forward statistics for port 0 -- RX-packets: 490511 RX-dropped: 0 RX-total: 490511 Bad-ipcsum: 490496 Bad-l4csum: 0 TX-packets: 488720 TX-dropped: 0 TX-total: 488720 -- Forward statistics for port 1 -- RX-packets: 488804 RX-dropped: 0 RX-total: 488804 Bad-ipcsum: 0 Bad-l4csum: 488704 TX-packets: 490511 TX-dropped: 0 TX-total: 490511 +++ Accumulated forward statistics for all ports+++ RX-packets: 979315 RX-dropped: 0 RX-total: 979315 TX-packets: 979231 TX-dropped: 0 TX-total: 979231 Done. [bmbenson]~/1.3.1.1/DPDK% sudo ./x86_64-default-linuxapp-gcc/app/testpmd -c 0xFF00FF00 -n 4 -b :06:00.0 -- --portmask=0x3 --nb-cores=2 --enable-rx-cksum --disable-hw-vlan --disable-rss --crc-strip --rxd=1024 --txd=1024 --rxfreet=32 -i ... testpmd> set fwd csum Set csum packet forwarding mode testpmd> start csum packet forwarding - CRC stripping enabled - packets/burst=16 nb forwarding cores=2 - nb forwarding ports=2 RX queues=1 - RX desc=1024 - RX free threshold=32 RX threshold registers: pthresh=8 hthresh=8 wthresh=4 TX queues=1 - TX desc=1024 - TX free threshold=0 TX threshold registers: pthresh=36 hthresh=0 wthresh=0 TX RS bit threshold=0 - TXQ flags=0x0 testpmd> stop Telling cores to stop... Waiting for lcores to finish... -- Forward statistics for port 0 -- RX-packets: 378894 RX-dropped: 0 RX-total: 378894 Bad-ipcsum: 0 Bad-l4csum: 0 TX-packets: 381197 TX-dropped: 0 TX-total: 381197 -- Forward statistics for port 1 -- RX-packets: 381197 RX-dropped: 0 RX-total: 381197 Bad-ipcsum: 0 Bad-l4csum: 0 TX-packets: 378894 TX-dropped: 0 TX-total: 378894 +++ Accumulated forward statistics for all ports+++ RX-packets: 760091 RX-dropped: 0 RX-total: 760091 TX-packets: 760091 TX-dropped: 0 TX-total: 760091
[dpdk-dev] RX checksum offloading
- speed 1 Mbps - full-duplex Done testpmd> set fwd csum Set csum packet forwarding mode testpmd> start csum packet forwarding - CRC stripping enabled - packets/burst=16 nb forwarding cores=2 - nb forwarding ports=2 RX queues=1 - RX desc=1024 - RX free threshold=128 RX threshold registers: pthresh=8 hthresh=8 wthresh=4 TX queues=1 - TX desc=1024 - TX free threshold=0 TX threshold registers: pthresh=36 hthresh=0 wthresh=0 TX RS bit threshold=0 - TXQ flags=0x0 testpmd> stop Telling cores to stop... Waiting for lcores to finish... -- Forward statistics for port 0 -- RX-packets: 372239 RX-dropped: 0 RX-total: 372239 Bad-ipcsum: 0 Bad-l4csum: 0 TX-packets: 375310 TX-dropped: 0 TX-total: 375310 -- Forward statistics for port 1 -- RX-packets: 375310 RX-dropped: 0 RX-total: 375310 Bad-ipcsum: 0 Bad-l4csum: 0 TX-packets: 372239 TX-dropped: 0 TX-total: 372239 +++ Accumulated forward statistics for all ports+++ RX-packets: 747549 RX-dropped: 0 RX-total: 747549 TX-packets: 747549 TX-dropped: 0 TX-total: 747549 Done. ________ From: dev [dev-boun...@dpdk.org] on behalf of Benson, Bryan Sent: Wednesday, November 06, 2013 7:09 PM To: dev at dpdk.org Subject: Re: [dpdk-dev] RX checksum offloading All, The receive checksum issue seems to be caused by using a RX_FREE_THRESH value that is 32 or larger, as validated by the test-pmd application. I used 2 different packet types - bad IP checksum sent to port 0 and bad tcp checksum to port 1. The way I was sending the packets did not vary between the various tests. Below is a brief summary of the testing - additional gory text is attached. 0, 8, 16, 24, 28, 30, 31 are okay. Values above 32 are not, as tested with 32, 64 and 128. I will continue researching this - The version I used to test this is the vanilla version of DPDK 1.3, but with the RSC disable patch applied (helps when there are many ACKs). Thanks, Bryan Benson [bmbenson]~/1.3.1.1/DPDK% sudo ./x86_64-default-linuxapp-gcc/app/testpmd -c 0xFF00FF00 -n 4 -b :06:00.0 -- --portmask=0x3 --nb-cores=2 --enable-rx-cksum --disable-hw-vlan --disable-rss --crc-strip --rxd=1024 --txd=1024 -i ... testpmd> set fwd csum Set csum packet forwarding mode testpmd> start csum packet forwarding - CRC stripping enabled - packets/burst=16 nb forwarding cores=2 - nb forwarding ports=2 RX queues=1 - RX desc=1024 - RX free threshold=0 RX threshold registers: pthresh=8 hthresh=8 wthresh=4 TX queues=1 - TX desc=1024 - TX free threshold=0 TX threshold registers: pthresh=36 hthresh=0 wthresh=0 TX RS bit threshold=0 - TXQ flags=0x0 testpmd> stop Telling cores to stop... Waiting for lcores to finish... -- Forward statistics for port 0 -- RX-packets: 490511 RX-dropped: 0 RX-total: 490511 Bad-ipcsum: 490496 Bad-l4csum: 0 TX-packets: 488720 TX-dropped: 0 TX-total: 488720 -- Forward statistics for port 1 -- RX-packets: 488804 RX-dropped: 0 RX-total: 488804 Bad-ipcsum: 0 Bad-l4csum: 488704 TX-packets: 490511 TX-dropped: 0 TX-total: 490511 +++ Accumulated forward statistics for all ports+++ RX-packets: 979315 RX-dropped: 0 RX-total: 979315 TX-packets: 979231 TX-dropped: 0 TX-total: 979231 Done. [bmbenson]~/1.3.1.1/DPDK% sudo ./x86_64-default-linuxapp-gcc/app/testpmd -c 0xFF00FF00 -n 4 -b :06:00.0 -- --portmask=0x3 --nb-cores=2 --enable-rx-cksum --disable-hw-vlan --disable-rss --crc-strip --rxd=1024 --txd=1024 --rxfreet=32 -i ... testpmd> set fwd csum Set csum packet forwarding mode testpmd> start csum packet forwarding - CRC stripping enabled - packets/burst=16 nb forwarding cores=2 - nb forwarding ports=2 RX queues=1 - RX desc=1024 - RX free threshold=32 RX threshold registers: pthresh=8 hthresh=8 wthresh=4 TX queues=1 - TX desc=1024 - TX free threshold=0 TX threshold registers: pthresh=36 hthresh=0 wthresh=0 TX RS bit threshold=0 - TXQ flags=0x0 testpmd> stop Telling cores to stop..
[dpdk-dev] TX IP checksum offloading
Hi Daniel, I posted some findings yesterday that seem relevant. Try using the ol flags when you use an RX free threshold below 32. I am looking into what can be done to fix it for values of 32 and above. http://dpdk.org/ml/archives/dev/2013-November/000724.html Thanks, Bryan Benson From: dev [dev-bounces at dpdk.org] on behalf of Daniel Kaminsky [daniel.kamin...@infinitelocality.com] Sent: Thursday, November 07, 2013 4:52 AM To: dev at dpdk.org Subject: [dpdk-dev] TX IP checksum offloading Hi, Did anyone had an experience using the PKT_TX_IP_CKSUM flag? I have an application that generates IP traffic but whenever I try to set this flag on (*m->ol_flags = PKT_TX_IP_CKSUM)* the rte_mbuf nothing is sent out. Retrieving that statistics from the ethernet device show zero for opackets and oerrors. I'm using ixgbe driver and DPDK 1.5.0-8 (from Intel) distribution. Thanks, Daniel
[dpdk-dev] RX checksum offloading
All,
I have prepared a patch for the issue. If it is not in the correct format,
please advise. I also changed the array to be uint32_t to be consistent with
the other functions that pull the status flags.
== PATCH BEGIN ==
>From 2ab73e88d92044c1d840f896d87e9a8cbcf29ce4 Mon Sep 17 00:00:00 2001
From: Bryan Benson
Date: Thu, 7 Nov 2013 21:38:57 +
Subject: [PATCH] Do not unset the status bits in the scan_hw_ring function
when counting the number of done descriptors.
---
inteldpdk/DPDK/lib/librte_pmd_ixgbe/ixgbe_rxtx.c | 13 +++--
1 file changed, 7 insertions(+), 6 deletions(-)
diff --git a/inteldpdk/DPDK/lib/librte_pmd_ixgbe/ixgbe_rxtx.c
b/inteldpdk/DPDK/lib/librte_pmd_ixgbe/ixgbe_rxtx.c
index 2be9979..c4abf9a 100755
--- a/inteldpdk/DPDK/lib/librte_pmd_ixgbe/ixgbe_rxtx.c
+++ b/inteldpdk/DPDK/lib/librte_pmd_ixgbe/ixgbe_rxtx.c
@@ -1037,7 +1037,8 @@ ixgbe_rx_scan_hw_ring(struct igb_rx_queue *rxq)
struct igb_rx_entry *rxep;
struct rte_mbuf *mb;
uint16_t pkt_len;
- int s[LOOK_AHEAD], nb_dd;
+ uint32_t s[LOOK_AHEAD];
+ int nb_dd;
int i, j, nb_rx = 0;
@@ -1060,12 +1061,12 @@ ixgbe_rx_scan_hw_ring(struct igb_rx_queue *rxq)
for (j = LOOK_AHEAD-1; j >= 0; --j)
s[j] = rxdp[j].wb.upper.status_error;
- /* Clear everything but the status bits (LSB) */
- for (j = 0; j < LOOK_AHEAD; ++j)
- s[j] &= IXGBE_RXDADV_STAT_DD;
+ nb_dd = 0;
+ /* add to nd_dd when the status bit is set (LSB) */
+ for (j = 0; j < LOOK_AHEAD; ++j) {
+ nb_dd += s[j] & IXGBE_RXDADV_STAT_DD;
+ }
- /* Compute how many status bits were set */
- nb_dd = s[0]+s[1]+s[2]+s[3]+s[4]+s[5]+s[6]+s[7];
nb_rx += nb_dd;
/* Translate descriptor info to mbuf format */
--
1.7.9.5
== PATCH END ==
A validation text email will follow.
Thank you,
Bryan Benson
Amazon Web Services
From: Thomas Monjalon [thomas.monja...@6wind.com]
Sent: Thursday, November 07, 2013 3:44 AM
To: Benson, Bryan
Cc: dev at dpdk.org; LiuFeng
Subject: Re: [dpdk-dev] RX checksum offloading
07/11/2013 10:06, Thomas Monjalon :
> 07/11/2013 04:44, Benson, Bryan :
> > RX queues=1 - RX desc=1024 - RX free threshold=16
> >
> > -- Forward statistics for port 0
> >
> > RX-total: 543761 Bad-ipcsum: 543558 Bad-l4csum: 0
> >
> > -- Forward statistics for port 1
> >
> > RX-total: 542226 Bad-ipcsum: 0 Bad-l4csum: 542005
> >
> > RX queues=1 - RX desc=1024 - RX free threshold=32
> >
> > -- Forward statistics for port 0
> >
> > RX-total: 378894 Bad-ipcsum: 0 Bad-l4csum: 0
> >
> > -- Forward statistics for port 1
> >
> > RX-total: 381197 Bad-ipcsum: 0 Bad-l4csum: 0
>
> Excellent report, thank you !
>
> We need to fix it now :)
LiuFeng had seen this issue with additonal comments:
http://www.dpdk.org/ml/archives/dev/2013-September/000529.html
--
Thomas
[dpdk-dev] RX checksum offloading
All, below is the validation of the submitted patch - I used the default rx free thresh (0) and 64 & it now works for both cases. Thank you! Bryan Benson -- Validation -- [bmbenson]~/1.3.1.1/DPDK% sudo ./x86_64-default-linuxapp-gcc/app/testpmd -c 0xFF00FF00 -n 4 -b :06:00.0 -- --portmask=0x3 --nb-cores=2 --enable-rx-cksum --disable-hw-vlan --disable-rss --crc-strip --rxd=1024 --txd=1024 -i ... ... testpmd> set fwd csum Set csum packet forwarding mode testpmd> start csum packet forwarding - CRC stripping enabled - packets/burst=16 nb forwarding cores=2 - nb forwarding ports=2 RX queues=1 - RX desc=1024 - RX free threshold=0 RX threshold registers: pthresh=8 hthresh=8 wthresh=4 TX queues=1 - TX desc=1024 - TX free threshold=0 TX threshold registers: pthresh=36 hthresh=0 wthresh=0 TX RS bit threshold=0 - TXQ flags=0x0 testpmd> stop Telling cores to stop... Waiting for lcores to finish... -- Forward statistics for port 0 -- RX-packets: 505103 RX-dropped: 0 RX-total: 505103 Bad-ipcsum: 505088 Bad-l4csum: 0 TX-packets: 508688 TX-dropped: 0 TX-total: 508688 -- Forward statistics for port 1 -- RX-packets: 508944 RX-dropped: 0 RX-total: 508944 Bad-ipcsum: 0 Bad-l4csum: 508672 TX-packets: 505103 TX-dropped: 0 TX-total: 505103 +++ Accumulated forward statistics for all ports+++ RX-packets: 1014047RX-dropped: 0 RX-total: 1014047 TX-packets: 1013791TX-dropped: 0 TX-total: 1013791 Done. testpmd> quit [bmbenson]~/1.3.1.1/DPDK% sudo ./x86_64-default-linuxapp-gcc/app/testpmd -c 0xFF00FF00 -n 4 -b :06:00.0 -- --portmask=0x3 --nb-cores=2 --enable-rx-cksum --disable-hw-vlan --disable-rss --crc-strip --rxd=1024 --txd=1024 --rxfreet=64 -i ... ... testpmd> set fwd csum Set csum packet forwarding mode testpmd> start csum packet forwarding - CRC stripping enabled - packets/burst=16 nb forwarding cores=2 - nb forwarding ports=2 RX queues=1 - RX desc=1024 - RX free threshold=64 RX threshold registers: pthresh=8 hthresh=8 wthresh=4 TX queues=1 - TX desc=1024 - TX free threshold=0 TX threshold registers: pthresh=36 hthresh=0 wthresh=0 TX RS bit threshold=0 - TXQ flags=0x0 testpmd> stop Telling cores to stop... Waiting for lcores to finish... -- Forward statistics for port 0 -- RX-packets: 314382 RX-dropped: 0 RX-total: 314382 Bad-ipcsum: 314368 Bad-l4csum: 0 TX-packets: 317453 TX-dropped: 0 TX-total: 317453 -- Forward statistics for port 1 -- RX-packets: 317492 RX-dropped: 0 RX-total: 317492 Bad-ipcsum: 0 Bad-l4csum: 317440 TX-packets: 314382 TX-dropped: 0 TX-total: 314382 +++ Accumulated forward statistics for all ports+++ RX-packets: 631874 RX-dropped: 0 RX-total: 631874 TX-packets: 631835 TX-dropped: 0 TX-total: 631835 From: Thomas Monjalon [thomas.monja...@6wind.com] Sent: Thursday, November 07, 2013 3:44 AM To: Benson, Bryan Cc: dev at dpdk.org; LiuFeng Subject: Re: [dpdk-dev] RX checksum offloading 07/11/2013 10:06, Thomas Monjalon : > 07/11/2013 04:44, Benson, Bryan : > > RX queues=1 - RX desc=1024 - RX free threshold=16 > > > > -- Forward statistics for port 0 > > > > RX-total: 543761 Bad-ipcsum: 543558 Bad-l4csum: 0 > > > > -- Forward statistics for port 1 > > > > RX-total: 542226 Bad-ipcsum: 0 Bad-l4csum: 542005 > > > > RX queues=1 - RX desc=1024 - RX free threshold=32 > > > > -- Forward statistics for port 0 > > > > RX-total: 378894 Bad-ipcsum: 0 Bad-l4csum: 0 > > > > -- Forward statistics for port 1 > > > > RX-total: 381197 Bad-ipcsum: 0 Bad-l4csum: 0 > > Excellent report, thank you ! > > We need to fix it now :) LiuFeng had seen this issue with additonal comments: http://www.dpdk.org/ml/archives/dev/2013-September/000529.html -- Thomas
[dpdk-dev] 82599 Checksum offloading for UDP
All, We are using RX checksum offloading for L3 and L4 on the 82599 NIC. One odd behavior we have noticed is that UDP packets with a zero UDP checksum are being marked as invalid. The UDP checksum field for IPv4 + UDP is optional. Is there a way to toggle the behavior? I have been reviewing the DataSheet and have not seen a way so far. Thank you, Bryan Benson Amazon Web Services
[dpdk-dev] problem with rte_pktmbuf_prepend (possible bug?)
Whoa, be careful, we used a custom init function and had a nasty bug because we assumed it was called on mbuf_free as well. The rte_pktmbuf_init function pointer passed into mempool create is only used at pool initialization time, not when an mbuf is freed - A reference to the obj_initi function is not stored anywhere for future use during mbuf_free. Some of the fields are reset when the NIC has completed the send of the mbufs, but it does not use a custom function ptr. Thanks, Bryan Benson From: dev [dev-bounces at dpdk.org] on behalf of Periklis Akritidis [akri...@niometrics.com] Sent: Tuesday, February 18, 2014 6:19 AM To: Mario Gianni Cc: dev at dpdk.org Subject: Re: [dpdk-dev] problem with rte_pktmbuf_prepend (possible bug?) Hi Mario, Are you passing rte_pktmbuf_init as the obj_init argument to rte_mempool_create? It is called when the mbuf if freed and it will reset the fields. I vaguely remember I had the same issue at some point and resolved it somehow. This comes to mind. Cheers, Periklis On 18 Feb, 2014, at 6:27 pm, Mario Gianni wrote: > Hi all, I'm experimenting some code with DPDK v1.5.0 and I have the following > problem: > > I have a thread that receives packets from NIC, once I received a packet I > want to prepend some data to it and I try to do so through the function > rte_pktmbuf_prepend() > then the packet is enqueued in a ring buffer where it will be used by a > client thread before being dropped through the function rte_pktmbuf_free() > called by the client thread. > > Now, if I try to send packets to this program I have the following behaviour: > In a first time it seems to work correctly, then after a certain number of > received packets (approximately the same number as the number of mbufs > present in the mempool) if I call the rte_pktmbuf_headroom it returns that > the headroom is shrinking more than the expected, until after a certain > number of packets the headroom goes to zero. > > It seems like that when I call the rte_pktmbuf_free() function it doesn't > reset the data position inside the mbuf, so when I call for a second time the > mbuf the headroom continues to shrink until it finishes. > > > Do you have any idea of this strange behaviour?Could it be a bug in the > prepend/free function? > > > Thank you, > > Mario
