--- On Wed, 8/12/09, Pyun YongHyeon <pyu...@gmail.com> wrote:
> From: Pyun YongHyeon <pyu...@gmail.com> > Subject: Re: nfe taskq performance issues > To: "Peter Steele" <pste...@webmail.maxiscale.com> > Cc: freebsd-net@freebsd.org > Date: Wednesday, August 12, 2009, 5:35 PM > On Thu, Jul 23, 2009 at 08:58:07AM > -0700, Peter Steele wrote: > > We've been hitting serious nfe taskq performance > issues during stress > > tests and in doing some research on the problem we > came across this old > > email: > > > > > > > > From: Ivan Voras <ivo...@freebsd.org> > > Date: April 28, 2009 3:53:14 AM PDT > > To: freebsd-thre...@freebsd.org > > Cc: freebsd-net@freebsd.org, > freebsd-performa...@freebsd.org > > Subject: Re: FreeBSD 7.1 taskq em performance > > > > > > I have been hitting some barrier with FreeBSD 7.1 > network performance. > > I > > > have written an application which contains two > kernel threads that > > takes > > > mbufs directly from a network interface and > forwards to another > > network > > > interface. This idea is to simulate different > network environment. > > > > > > I have been using FreeBSD 6.4 amd64 and tested > with an Ixia box > > > (specialised hardware firing very high packet > rate). The PC was a > > Core2 2.6 > > > GHz with dual ports Intel PCIE Gigabit network > card. It can manage up > > to 1.2 > > > million pps. > > > > > > I have a higher spec PC with FreeBSD 7.1 amd64 > and Quadcore 2.3 GHz > > and > > > PCIE Gigabit network card. The performance can > only achieve up to 600k > > pps. > > > I notice the 'taskq em0' and 'taskq em1' is solid > 100% CPU but it is > > not in > > > FreeBSD 6.4. > > > > > > > > In our case we are running FreeBSD 7.0, but we are > seeing our boxes > > experience serious thread starvation issues as the > nfe0 cpu percentage > > climbs steadily while cpu idle time drops at times to > 0 percent. This > > email thread mentioned a patch for the em driver > here: > > > > > > > > http://people.yandex-team.ru/~wawa/ > > <http://people.yandex-team.ru/%7Ewawa/> > > > > > > > > > Does anyone know if this patch will work with the nfe > driver? > > > > That's for em(4). > > AFAIK all nfe(4) controllers lacks intelligent interrupts > moderation so driver should be prepared to handle > excessive > interrupt loads. I'm not sure whether NVIDIA ethernet > controllers > really lacks efficient interrupt mitigation mechanism but > it > seems Linux also faces the same hardware problem. > As you might know there is no publicly available data sheet > for > NVIDIA controllers so setting it right looks very hard to > me. > Try removing the INTR_MPSAFE flag from the bus_setup_intr() call. The entire point of using filters is to reduce lock contention. It might not solve the problem but its clearly an unnecessary potential bottleneck. Barney _______________________________________________ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
