Hi Hamid, I didn't see any attachment but I think there is a solution. My first question is, when you created the mempool, how much did you defined the memory cache per CPU? In the example program this number is fairly small (32) and I think that increasing it to something much bigger (e.g. 512) will significantly improve the CPU scalability.
Regards, Daniel Kaminsky On Wed, Apr 30, 2014 at 7:56 AM, Jayakumar, Muthurajan < muthurajan.jayakumar at intel.com> wrote: > Hi, > Please find the attached paper http://kfall.net/ucbpage/papers/snc.pdf > Figures 4 and 5 refers about the degradation when the # of queues are > increased. > It refers sweet spot as 2 to 4 queues. > > Have you please verified with smaller # of queues please? > > Thanks, > > > -----Original Message----- > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Hamid Ramazani > Sent: Tuesday, April 29, 2014 9:42 PM > To: Thomas Monjalon; dev at dpdk.org > Subject: [dpdk-dev] packet loss: multi-queue (RSS enabled) > > Hi, > I tried a lot (more than a week) to solve the problem myself and not to > bother the list, but I didn't succeed. Maybe other people have the same > problem. > > I have a simple program attached. It is intended for simple packet > capturing; captures from interface and writes to memory, and frees the > memory in the next loop. > > I have a 10G 82599EB Intel SFI/SFP+ network interface for capturing > packets. > As you may know, this network card supports up to 128 RSS queues. > > This is just a test, so the packets being sent at 820Kpps (kilo packet per > second). Each packet is 1500B (fixed size); it is 9.16 Gbit per second. > Of course when the packet per seconds goes up and packet size goes down > (e.g. 400B per packet), it gets much worse. > > When using one queue to receive, I receive all the packets, with no loss. > When I use more than one queue (e.g. 8 queues), with each thread running > on a dedicated core, I have a considerable amount of loss. > > Please note that: > 1. The computer has 12 * 2.67GHz cores, and it does nothing else but > capturing packets. The CPU is Intel Xeon X5650. > 2. The operating system is Ubuntu 12.04.3 LTS > > Attached file includes: > main.h > main.c > Makefile > ./run.sh > > It is configured to be run with 8 queues. > > If you want to change the number of receive queues, please: > 1. in main.c, change the value assigned to nb_rx_q_of_dev to the desired > value. > 2. change core mask in run.sh file (since there is SKIP_MASTER, you should > give a core containing one more CPU than given number of queues). > > I think there might be following problems: > 1. the port configuration is not fine. > 2. freeing memory has a considerable amount of overhead, and may be I > shouldn't do that. But If I don't the pool will be full, won't be? Is there > any other way? > > Please help. > Thanks a lot in advance for your help and comments. > > All the Best, > Hamid >
