On Wed, Mar 10, 2010 at 02:45:47PM -0800, David Christensen wrote: > > > I successfully reproduced the issue with netperf on BCM5709. > > You can use UDP frame size 1 to trigger the issue. > > > > > Changing the high level design of bce_rx_intr() and > > > bce_rx_fill_chain() slightly to post a new buffer as each frame is > > > passed to the OS would likely avoid these gaps during > > bursts of small > > > frames but I'm not sure whether they'll have a negative > > impact on the > > > more common case of streams of MTU sized frames. I've > > considered this > > > in the past but never coded the change and tested the resulting > > > performance. > > > > > > > I guess this may slightly increase performance with additional > > bus_dma(9) overheads but I think one of reason of dropping > > frames under heavy UDP frames may come from lack of free RX > > descriptors. > > Because bce(4) just uses a single RX ring so the number of > > available RX buffers would be 512. However it seems it's not > > possible to increase the number of RX buffers per RX ring so > > the next possible approach would be switching to use multiple > > RX rings with RSS. Even though FreeBSD does not dynamically > > adjust loads among CPUs I guess using RSS would be the way to go. > > The bce(4) hardware supports a linked list of pages for RX > buffer descriptors. The stock build supports 2 pages (RX_PAGES) > with a total of 511 BD's per page. The hardware can support a > maximum of 64K BD's but that would be an unnecessarily large > amount of mbufs for an infrequent problem. >
Thanks for the info. I guess 2048 or 4096 BDs would be necessary to get satisfactory Rx performance. I'll have to experiment this. > The middle road would probably involve changing RX_PAGES from a > #define to a sysctl variable to allow tuning for specific > environments along with a change in bce_rx_intr() to fill the > ring after all frames have been processed or when more than > 256 BDs have been consumed, whichever comes first. > > RSS would be great as well though it wouldn't make a dent in > this case since RSS is only supported for TCP, not UDP. > Even though UDP is not supported in RSS, RSS can handle IP. This wouldn't distribute UDP load coming from a single host but if source IP address is different it may help, I guess. > Dave _______________________________________________ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
