On Thu, Dec 10, 2015 at 4:40 PM, Adrian Chadd <adrian.ch...@gmail.com> wrote:
> On 10 December 2015 at 10:29, Denis Pearson <dennix.pear...@gmail.com> > wrote: > > On Thu, Dec 10, 2015 at 2:18 PM, Eggert, Lars <l...@netapp.com> wrote: > > > >> On 2015-10-26, at 18:40, Eggert, Lars <l...@netapp.com> wrote: > >> > On 2015-10-26, at 17:08, Pieper, Jeffrey E < > jeffrey.e.pie...@intel.com> > >> wrote: > >> >> As a caveat, this was using default netperf message sizes. > >> > > >> > I get the same ~3 Gb/s with the default netperf sizes and driver > 1.4.5. > >> > >> Now there is version 1.4.8 on the Intel website, but it doesn't change > >> things for me. > >> > > > > I had the opportunity to see similar numbers and behavior while using > XL710 > > 1.4.3 as of FreeBSD r291085 while in DPDK poll mode, but driver 1.2.8 as > of > > r292035 was providing expected numbers. While removing rxcsum/txcsum did > > not provide differences, fully removing RSS + disabling rx/cxsum support > > provided better numbers. > > Can someone debug this a bit more? (My kit with ixl NICs in it is > still not up and available. :( ) > > Device RSS, even without kernel RSS enabled, shouldn't cause a massive > performance drop. If it is then something else odd is going on. > Do you have a diff where you removed things? > I can probably find out a snapshot with the code at the time and extract a diff, yes. I just don't know how it worths wasting the time when the problem is not reproducible on the current 1.4.8 driver which will hopefully get into -CURRENT (if it's not already there?). And it's much more specific, the performance drop happened on dpdk poll mode, not the usual kernel operation so a simple diff only pointing out the changes for the driver to actually build and run without rss will still require a testlab and different ways to generate traffic. This is why I suggested a transceiver change or replug first. Anyway RSS performance dropping problem is far from a FreeBSD specific problem, while researching I could find the exact same complaints on Windows users starting from windows 8 while having RSS@4 or RSS@16 or RSS completely disabled, some times with acceptable results only when it was disabled (despiste the fact that MiniportInterruptDPC was using a whole CPU when RSS was off results were still better). So I guess this is just a side effect of when it's just good to have NIC features turned off. The reason, I'm not an engineer to answer, but I would guess it's related to other NIC features also doing something with the packet or any sort of errors netstat or driver status may not tell. I was able to see the problem even with low pps rates and big packet sizes, as well as avg pkt size of 768bytes so I don't think it's any sort of card resource starvation. I can manage to have the whole lab up and running by the weekend if you want to investigate and compare, just ping me off list. > > -adrian > > > However now with driver 1.4.8 and the same set of hardware setup, except > > for a different transceiver, I can get 36Gbps/24Mpps with no further > > tweaks, so if you can replace your transceiver, shall be a different test > > as a starting point. > _______________________________________________ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
