Thanks a lot Jeff for your detailed explanation. I still have open question left. I would be grateful if someone would share their insight on it.
I have performed experiments to vary both the MAX_BURST_SIZE (originally set as 32) and BURST_TX_DRAIN_US (originally set as 100 usec) in l3fwd main.c. While I vary the MAX_BURST_SIZE (1, 8, 16, 32, 64, and 128) and fix BURST_TX_DRAIN_US=100 usec, I see a low average latency when sending a burst of packets less than or equal to the MAX_BURST_SIZE. For example, when MAX_BURST_SIZE is 32, if I send a burst of 32 packets or less, then I get around 10 usec of latency. When it goes over it, it starts to get higher average latency, which make total sense. My main question are the following. When I start sending continuous packet at a rate of 14.88 Mpps for 64B packets, it shows consistently receiving an average latency of 150 usec, no matter what MAX_BURST_SIZE. My guess is that the latency should be bounded by BURST_TX_DRAIN_US, which is fixed at 100 usec. Would you share your thought on this issue please? Thanks, Jun On Thu, May 22, 2014 at 7:06 PM, Shaw, Jeffrey B <jeffrey.b.shaw at intel.com>wrote: > Hello, > > > I measured a roundtrip latency (using Spirent traffic generator) of > sending 64B packets over a 10GbE to DPDK, and DPDK does nothing but simply > forward back to the incoming port (l3fwd without any lookup code, i.e., > dstport = port_id). > > However, to my surprise, the average latency was around 150 usec. (The > packet drop rate was only 0.001%, i.e., 283 packets/sec dropped) Another > test I did was to measure the latency due to sending only a single 64B > packet, and the latency I measured is ranging anywhere from 40 usec to 100 > usec. > > 40-100usec seems very high. > The l3fwd application does some internal buffering before transmitting the > packets. It buffers either 32 packets, or waits up to 100us (hash-defined > as BURST_TX_DRAIN_US), whichever comes first. > Try either removing this timeout, or sending a burst of 32 packets at > time. Or you could try with testpmd, which should have reasonably low > latency out of the box. > > There is also a section in the Release Notes (8.6 How can I tune my > network application to achieve lower latency?) which provides some pointers > for getting lower latency if you are willing to give up top-rate throughput. > > Thanks, > Jeff >
