Thanks for answer, Vladimir! So we need look for x16 NIC if we want achieve 40GE line rate...
On Wed, Jul 1, 2015 at 3:06 PM, Vladimir Medvedkin <medvedkinv at gmail.com> wrote: > Hi Pavel, > > Looks like you ran into pcie bottleneck. So let's calculate xl710 rx only > case. > Assume we have 32byte descriptors (if we want more offload). > DMA makes one pcie transaction with packet payload, one descriptor writeback > and one memory request for free descriptors for every 4 packets. For > Transaction Layer Packet (TLP) there is 30 bytes overhead (4 PHY + 6 DLL + > 16 header + 4 ECRC). So for 1 rx packet dma sends 30 + 64(packet itself) + > 30 + 32 (writeback descriptor) + (16 / 4) (read request for new > descriptors). Note that we do not take into account PCIe ACK/NACK/FC Update > DLLP. So we have 160 bytes per packet. One lane PCIe 3.0 transmits 1 byte in > 1 ns, so x8 transmits 8 bytes in 1 ns. 1 packet transmits in 20 ns. Thus > in theory pcie 3.0 x8 may transfer not more than 50mpps. > Correct me if I'm wrong. > > Regards, > Vladimir > > > 2015-06-29 18:41 GMT+03:00 Pavel Odintsov <pavel.odintsov at gmail.com>: >> >> Hello, Andrew! >> >> What NIC have you used? Is it XL710? >> >> On Mon, Jun 29, 2015 at 6:38 PM, Andrew Theurer <atheurer at redhat.com> >> wrote: >> > >> > >> > On Mon, Jun 29, 2015 at 10:06 AM, Keunhong Lee <dlrmsghd at gmail.com> >> > wrote: >> >> >> >> I have not used XL710 or i40e. >> >> I have no opinion for those NICs. >> >> >> >> Keunhong. >> >> >> >> 2015-06-29 15:59 GMT+09:00 Pavel Odintsov <pavel.odintsov at gmail.com>: >> >> >> >> > Hello! >> >> > >> >> > Lee, thank you so much for sharing your experience! What do you think >> >> > about 40GE version of 82599? >> >> > >> >> > On Mon, Jun 29, 2015 at 2:35 AM, Keunhong Lee <dlrmsghd at gmail.com> >> >> > wrote: >> >> > > DISCLAIMER: This information is not verified. This is truly my >> >> > > personal >> >> > > opinion. >> >> > > >> >> > > As I know, intel 82599 is the only 10G NIC which supports line rate >> >> > > with >> >> > > minimum sized packets (64 byte). >> >> > > According to our internal tests, Mellanox's 40G NICs even support >> >> > > less >> >> > than >> >> > > 30Mpps. >> >> > > I think 40 Mpps is the hardware capacity. >> > >> > >> > This is approximately what I see as well. >> > >> >> >> >> > > >> >> > > Keunhong. >> >> > > >> >> > > >> >> > > >> >> > > 2015-06-28 19:34 GMT+09:00 Pavel Odintsov >> >> > > <pavel.odintsov at gmail.com>: >> >> > >> >> >> > >> Hello, folks! >> >> > >> >> >> > >> We have execute bunch of tests for receive data with Intel XL710 >> >> > >> 40GE >> >> > >> NIC. We want to achieve wire speed on this platform for traffic >> >> > >> capture. >> >> > >> >> >> > >> But we definitely can't do it. We tried with different versions of >> >> > >> DPDK: 1.4, 1.6, 1.8, 2.0. And have not success. >> >> > >> >> >> > >> We achieved only 40Mpps and could do more. >> >> > >> >> >> > >> Could anybody help us with this issue? Looks like this NIC's could >> >> > >> not >> >> > >> work on wire speed :( >> >> > >> >> >> > >> Platform: Intel Xeon E5 e5 2670 + XL 710. >> >> > >> >> >> > >> -- >> >> > >> Sincerely yours, Pavel Odintsov >> >> > > >> >> > > >> >> > >> >> > >> >> > >> >> > -- >> >> > Sincerely yours, Pavel Odintsov >> >> > >> > >> > -Andrew >> > >> > >> >> >> >> -- >> Sincerely yours, Pavel Odintsov > > -- Sincerely yours, Pavel Odintsov
