--- On Wed, 12/2/09, Yuriy A. Korobko <administra...@shtorm.com> wrote:
> From: Yuriy A. Korobko <administra...@shtorm.com> > Subject: Re: Reducing number of interrupts from intel pro 1000 et adapter > To: "Barney Cordoba" <barney_cord...@yahoo.com> > Cc: freebsd-net@freebsd.org > Date: Wednesday, December 2, 2009, 11:05 AM > > > > > > I'd like to know a way to control tx interrupts > on intel > > > pro 1000 et > > > adapter with igb driver. Just installed one in > the router > > > and systat > > > shows 8-9k rx interrupts and 20k tx interrupts > from igb0 > > > and igb1 > > > adapters. Box is a router running freebsd 7.2 > release, I've > > > tried > > > default driver from kernel source and latest from > intel > > > site, effect is > > > the same with automatic interrupt moderation > enabled and > > > disabled. I > > > have the same box with intel pro 1000 pt adapter > which > > > have > > > tx(rx)_int_delay sysctls in em driver, I was able > to reduce > > > number of > > > tx/rx interrupts to 7-8k per interface and got > much more > > > cpu idle > > > because of less context switches with same pps. > > > > > > > > I'm curious as to why you are doing a load test on a > single core system > > with a part that is clearly designed to be used on a > multicore system? > > > > Barney > > Box have core quad cpu: > > 87 processes: 7 running, 58 sleeping, 22 waiting > CPU 0: 0.0% user, 0.0% nice, 3.1% system, > 36.4% interrupt, 60.5% idle > CPU 1: 0.0% user, 0.0% nice, 15.5% system, > 41.1% interrupt, 43.4% idle > CPU 2: 0.0% user, 0.0% nice, 11.6% system, > 27.1% interrupt, 61.2% idle > CPU 3: 0.0% user, 0.0% nice, 2.3% system, > 41.5% interrupt, 56.2% idle > Mem: 192M Active, 206M Inact, 246M Wired, 716K Cache, 112M > Buf, 1356M > > Anyway, high interrupt rate fixed by > > border# cat /boot/loader.conf | grep igb > if_igb_load="YES" > hw.igb.rxd=4096 > hw.igb.txd=4096 > hw.igb.num_queues=1 > hw.igb.enable_aim=1 > hw.igb.low_latency=1000 > hw.igb.ave_latency=2000 > hw.igb.bulk_latency=4000 > hw.igb.rx_process_limit=200 > hw.igb.fc_setting=0 > hw.igb.lro=0 > border# > > Have no idea why, but it seems like setting dev.igb. > sysctls at runtime > just ignored by the driver. Now I have 1000-1200 rx and > around 2000 tx > interrupts per second from interface with 100 kpps flow in > each > direction, main cpu eater on the router is dummynet right > now, it will > be awesome if one day we can have it multithreaded. > > I'm using latest igb driver from intel site, default driver > from kernel > may not have this issue. > > Thanks. > > > _______________________________________________ > freebsd-net@freebsd.org > mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org" > Ah, dummynet. Explains much. I've never seen livelock on a quad core system. The igb driver in FreeBSD needs a complete and total overhaul. Don't waste alot of time tweeking it. Turn off AIM and hard code something that suits your network. Intel's aim algorithm is ill-conceived IMO. Barney _______________________________________________ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
