On Monday, October 15, 2012 9:04:27 am John Baldwin wrote:
> On Monday, October 15, 2012 10:10:40 am Alexander V. Chernikov wrote:
> > On 13.10.2012 23:24, Jack Vogel wrote:
> > > On Sat, Oct 13, 2012 at 11:22 AM, Luigi Rizzo<ri...@iet.unipi.it> wrote:
> >
> > >>
> > >> one option could be (same as it is done in the timer
> > >> routine in dummynet) to build a list of all the packets
> > >> that need to be sent to if_input(), and then call
> > >> if_input with the entire list outside the lock.
> > >>
> > >> It would be even easier if we modify the various *_input()
> > >> routines to handle a list of mbufs instead of just one.
> >
> > Bulk processing is generally a good idea we probably should implement.
> > Probably starting from driver queue ending with marked mbufs
> > (OURS/forward/legacy processing (appletalk and similar))?
> >
> > This can minimize an impact for all
> > locks on RX side:
> > L2
> > * rx PFIL hook
> > L3 (both IPv4 and IPv6)
> > * global IF_ADDR_RLOCK (currently commented out)
> > * Per-interface ADDR_RLOCK
> > * PFIL hook
> >
> > From the first glance, there can be problems with:
> > * Increased latency (we should have some kind of rx_process_limit), but
> > still
> > * reader locks being acquired for much longer amount of time
> >
> > >>
> > >> cheers
> > >> luigi
> > >>
> > >> Very interesting idea Luigi, will have to get that some thought.
> > >
> > > Jack
> >
> > Returning to original post topic:
> >
> > Given
> > 1) we are currently binding ixgbe ithreads to CPU cores
> > 2) RX queue lock is used by (indirectly) in only 2 places:
> > a) ISR routine (msix or legacy irq)
> > b) taskqueue routine which is scheduled if some packets remains in RX
> > queue and rx_process_limit ended OR we need something to TX
> >
> > 3) in practice taskqueue routine is a nightmare for many people since
> > there is no way to stop "kernel {ix0 que}" thread eating 100% cpu after
> > some traffic burst happens: once it is called it starts to schedule
> > itself more and more replacing original ISR routine. Additionally,
> > increasing rx_process_limit does not help since taskqueue is called with
> > the same limit. Finally, currently netisr taskq threads are not bound to
> > any CPU which makes the process even more uncontrollable.
>
> I think part of the problem here is that the taskqueue in ixgbe(4) is
> bogusly rescheduled for TX handling. Instead, ixgbe_msix_que() should
> just start transmitting packets directly.
>
> I fixed this in igb(4) here:
>
> http://svnweb.freebsd.org/base?view=revision&revision=233708
>
> You can try this for ixgbe(4). It also comments out a spurious taskqueue
> reschedule from the watchdog handler that might also lower the taskqueue
> usage. You can try changing that #if 0 to an #if 1 to test just the txeof
> changes:
Is anyone able to test this btw to see if it improves things on ixgbe at all?
(I don't have any ixgbe hardware.)
--
John Baldwin
_______________________________________________
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
