Hi Alexander, Please confirm if the patch works for you.
@Wang, are you saying that without the patch the NIC does not fan out the messages properly on all the receive queues ? So what exactly happens ? Regards -Prashant -----Original Message----- From: dev [mailto:dev-boun...@dpdk.org] On Behalf Of Alexander Belyakov Sent: Monday, November 04, 2013 1:51 AM To: Wang, Shawn Cc: dev at dpdk.org Subject: Re: [dpdk-dev] Surprisingly high TCP ACK packets drop counter Hi, thanks for the patch and explanation. We have tried DPDK 1.3 and 1.5 - both have the same issue. Regards, Alexander On Fri, Nov 1, 2013 at 6:54 PM, Wang, Shawn <xingbow at amazon.com> wrote: > Hi: > > We had the same problem before. It turned out that RSC (receive side > coalescing) is enabled by default in DPDK. So we write this na?ve > patch to disable it. This patch is based on DPDK 1.3. Not sure 1.5 has > changed it or not. > After this patch, ACK rate should go back to 14.5Mpps. For details, > you can refer to Intel? 82599 10 GbE Controller Datasheet. (7.11 > Receive Side Coalescing). > > From: xingbow <xingbow at amazon.com> > Date: Wed, 21 Aug 2013 11:35:23 -0700 > Subject: [PATCH] Disable RSC in ixgbe_dev_rx_init function in file > > ixgbe_rxtx.c > > --- > > DPDK/lib/librte_pmd_ixgbe/ixgbe/ixgbe_type.h | 2 +- > DPDK/lib/librte_pmd_ixgbe/ixgbe_rxtx.c | 7 +++++++ > 2 files changed, 8 insertions(+), 1 deletion(-) > > diff --git a/DPDK/lib/librte_pmd_ixgbe/ixgbe/ixgbe_type.h > b/DPDK/lib/librte_pmd_ixgbe/ixgbe/ixgbe_type.h > index 7fffd60..f03046f 100644 > > --- a/DPDK/lib/librte_pmd_ixgbe/ixgbe/ixgbe_type.h > > +++ b/DPDK/lib/librte_pmd_ixgbe/ixgbe/ixgbe_type.h > > @@ -1930,7 +1930,7 @@ enum { > > #define IXGBE_RFCTL_ISCSI_DIS 0x00000001 > #define IXGBE_RFCTL_ISCSI_DWC_MASK 0x0000003E > #define IXGBE_RFCTL_ISCSI_DWC_SHIFT 1 > -#define IXGBE_RFCTL_RSC_DIS 0x00000010 > > +#define IXGBE_RFCTL_RSC_DIS 0x00000020 > > #define IXGBE_RFCTL_NFSW_DIS 0x00000040 > #define IXGBE_RFCTL_NFSR_DIS 0x00000080 > #define IXGBE_RFCTL_NFS_VER_MASK 0x00000300 > diff --git a/DPDK/lib/librte_pmd_ixgbe/ixgbe_rxtx.c > b/DPDK/lib/librte_pmd_ixgbe/ixgbe_rxtx.c > index 07830b7..ba6e05d 100755 > > --- a/DPDK/lib/librte_pmd_ixgbe/ixgbe_rxtx.c > > +++ b/DPDK/lib/librte_pmd_ixgbe/ixgbe_rxtx.c > > @@ -3007,6 +3007,7 @@ ixgbe_dev_rx_init(struct rte_eth_dev *dev) > > uint64_t bus_addr; > uint32_t rxctrl; > uint32_t fctrl; > + uint32_t rfctl; > > uint32_t hlreg0; > uint32_t maxfrs; > uint32_t srrctl; > @@ -3033,6 +3034,12 @@ ixgbe_dev_rx_init(struct rte_eth_dev *dev) > > fctrl |= IXGBE_FCTRL_PMCF; > IXGBE_WRITE_REG(hw, IXGBE_FCTRL, fctrl); > > + /* Disable RSC */ > + RTE_LOG(INFO, PMD, "Disable RSC\n"); > + rfctl = IXGBE_READ_REG(hw, IXGBE_RFCTL); > + rfctl |= IXGBE_RFCTL_RSC_DIS; > + IXGBE_WRITE_REG(hw, IXGBE_RFCTL, rfctl); > + > > /* > * Configure CRC stripping, if any. > */ > -- > > > Thanks. > Wang, Xingbo > > > > > On 11/1/13 6:43 AM, "Alexander Belyakov" <abelyako at gmail.com> wrote: > > >Hello, > > > >we have simple test application on top of DPDK which sole purpose is > >to forward as much packets as possible. Generally we easily achieve > >14.5Mpps with two 82599EB (one as input and one as output). The only > >suprising exception is forwarding pure TCP ACK flood when performace > >always drops to approximately 7Mpps. > > > >For simplicity consider two different types of traffic: > >1) TCP SYN flood is forwarded at 14.5Mpps rate, > >2) pure TCP ACK flood is forwarded only at 7Mpps rate. > > > >Both SYN and ACK packets have exactly the same length. > > > >It is worth to mention, this forwarding application looks at Ethernet > >and IP headers, but never deals with L4 headers. > > > >We tracked down issue to RX circuit. To be specific, there are 4 RX > >queues initialized on input port and rte_eth_stats_get() shows > >uniform packet distribution (q_ipackets) among them, while q_errors > >remain zero for all queues. The only drop counter quickly increasing > >in the case of pure ACK flood is ierrors, while rx_nombuf remains zero. > > > >We tried different kinds of traffic generators, but always got the > >same > >result: 7Mpps (instead of expected 14Mpps) for TCP packets with ACK > >flag bit set while all other flag bits dropped. Source IPs and ports > >are selected randomly. > > > >Please let us know if anyone is aware of such strange behavior and > >where should we look at to narrow down the problem. > > > >Thanks in advance, > >Alexander Belyakov > > =============================================================================== Please refer to http://www.aricent.com/legal/email_disclaimer.html for important disclosures regarding this electronic communication. ===============================================================================
