Re: netif_tx_disable and lockless TX

2006-06-14 Thread jamal
On Wed, 2006-31-05 at 19:52 +0200, Robert Olsson wrote: > jamal writes: > > > Latency-wise: TX completion interrupt provides the best latency. > > Processing in the poll() -aka softirq- was almost close to the hardirq > > variant. So if you can make things run in a softirq such as transmit > >

Re: netif_tx_disable and lockless TX

2006-06-02 Thread Robert Olsson
Stephen Hemminger writes: > I also noticed that you really don't save much by doing TX cleaning at > hardirq, because in hardirq you need to do dev_kfree_irq and that causes > a softirq (for the routing case where users=1). So when routing it > doesn't make much difference, both methods ca

Re: netif_tx_disable and lockless TX

2006-06-01 Thread Stephen Hemminger
Robert Olsson wrote: jamal writes: > Latency-wise: TX completion interrupt provides the best latency. > Processing in the poll() -aka softirq- was almost close to the hardirq > variant. So if you can make things run in a softirq such as transmit > one, then the numbers will likely stay the s

Re: netif_tx_disable and lockless TX

2006-05-31 Thread Herbert Xu
On Wed, May 31, 2006 at 05:27:53PM -0700, Michael Chan wrote: > > Now that I think about it some more, we can eliminate most of the > tx_lock in tg3 without converting to xmit_lock. In most places, we get > the tx_lock after we call netif_tx_disable(), etc, and quiesce the chip. > So the tx_lock o

Re: netif_tx_disable and lockless TX

2006-05-31 Thread Michael Chan
On Thu, 2006-06-01 at 10:42 +1000, Herbert Xu wrote: > On Wed, May 31, 2006 at 04:01:06PM -0700, Michael Chan wrote: > > > > Oh, I thought your idea was to keep the LLTX and just replace tx_lock > > with xmit_lock in tg3. But I suppose we can also clear LLTX, remove the > > tx_lock in hard_start_x

Re: netif_tx_disable and lockless TX

2006-05-31 Thread Herbert Xu
On Wed, May 31, 2006 at 04:01:06PM -0700, Michael Chan wrote: > > Oh, I thought your idea was to keep the LLTX and just replace tx_lock > with xmit_lock in tg3. But I suppose we can also clear LLTX, remove the > tx_lock in hard_start_xmit and convert the rest to avoid changing > netpoll. That's w

Re: netif_tx_disable and lockless TX

2006-05-31 Thread Michael Chan
On Thu, 2006-06-01 at 10:25 +1000, Herbert Xu wrote: > On Wed, May 31, 2006 at 05:09:08PM -0700, David Miller wrote: > > From: "Michael Chan" <[EMAIL PROTECTED]> > > Date: Wed, 31 May 2006 14:20:29 -0700 > > > > > David, So do we want to fix this issue as proposed by Herbert to replace > > > tx_lo

Re: netif_tx_disable and lockless TX

2006-05-31 Thread Herbert Xu
On Wed, May 31, 2006 at 05:09:08PM -0700, David Miller wrote: > From: "Michael Chan" <[EMAIL PROTECTED]> > Date: Wed, 31 May 2006 14:20:29 -0700 > > > David, So do we want to fix this issue as proposed by Herbert to replace > > tx_lock with xmit_lock? It seems quite straightforward to do. For this

Re: netif_tx_disable and lockless TX

2006-05-31 Thread David Miller
From: "Michael Chan" <[EMAIL PROTECTED]> Date: Wed, 31 May 2006 14:20:29 -0700 > David, So do we want to fix this issue as proposed by Herbert to replace > tx_lock with xmit_lock? It seems quite straightforward to do. For this > change to work, netpoll also needs to be fixed up a bit to check for

Re: netif_tx_disable and lockless TX

2006-05-31 Thread Michael Chan
On Tue, 2006-05-30 at 22:11 -0700, David Miller wrote: > From: Herbert Xu <[EMAIL PROTECTED]> > Date: Wed, 31 May 2006 14:58:11 +1000 > > > Yes, TG3 does not disable IRQs when taking its TX lock. So do you see > > any problems with replacing the TG3 TX lock using xmit_lock? > > I don't see any.

Re: netif_tx_disable and lockless TX

2006-05-31 Thread Robert Olsson
jamal writes: > Latency-wise: TX completion interrupt provides the best latency. > Processing in the poll() -aka softirq- was almost close to the hardirq > variant. So if you can make things run in a softirq such as transmit > one, then the numbers will likely stay the same. I don't rememb

Re: netif_tx_disable and lockless TX

2006-05-31 Thread jamal
On Wed, 2006-31-05 at 22:40 +1000, Herbert Xu wrote: > On Wed, May 31, 2006 at 08:36:12AM -0400, jamal wrote: > > [..] > > > > Been done in the past, bad numbers especially in SMP for reasons of > > latency and likelihood that a tasklet will run in a totally different > > CPU. > > Was the bad lat

Re: netif_tx_disable and lockless TX

2006-05-31 Thread Herbert Xu
On Wed, May 31, 2006 at 08:36:12AM -0400, jamal wrote: > > > The IRQ handler would look like > > > > if (!spin_trylock(&dev->xmit_lock)) { > > tasklet_schedule(&priv->tx_completion_tasklet); > > return; > > } > > > > handle_tx_completion(); > > spin_unlock(

Re: netif_tx_disable and lockless TX

2006-05-31 Thread jamal
On Wed, 2006-31-05 at 22:06 +1000, Herbert Xu wrote: > On Wed, May 31, 2006 at 12:08:18AM -0700, David Miller wrote: > > > > I understand what you're saying. What I'm talking about is that in > > net channel drivers we might go back to IRQ disabling locks again. > > OK, let's assume that the TX

Re: netif_tx_disable and lockless TX

2006-05-31 Thread Herbert Xu
On Wed, May 31, 2006 at 12:08:18AM -0700, David Miller wrote: > > I understand what you're saying. What I'm talking about is that in > net channel drivers we might go back to IRQ disabling locks again. OK, let's assume that the TX completion will go back into the IRQ handler. I contend that we

Re: netif_tx_disable and lockless TX

2006-05-31 Thread David Miller
From: Herbert Xu <[EMAIL PROTECTED]> Date: Wed, 31 May 2006 16:31:52 +1000 > On Tue, May 30, 2006 at 11:26:26PM -0700, David Miller wrote: > > > > I don't think it will be worthwhile to keep NAPI around just for > > TX completion. Sure the dev_kfree_skb() will schedule software > > interrupt wor

Re: netif_tx_disable and lockless TX

2006-05-30 Thread Herbert Xu
On Tue, May 30, 2006 at 11:26:26PM -0700, David Miller wrote: > > I don't think it will be worthwhile to keep NAPI around just for > TX completion. Sure the dev_kfree_skb() will schedule software > interrupt work to do the actual free, but the TX ring walking > and dev_kfree_skb() calls will be i

Re: netif_tx_disable and lockless TX

2006-05-30 Thread David Miller
From: Herbert Xu <[EMAIL PROTECTED]> Date: Wed, 31 May 2006 15:14:51 +1000 > On Tue, May 30, 2006 at 10:11:17PM -0700, David Miller wrote: > > > > It will come back when net channels arrive :-) This is because net > > channel drivers will do all the work in hard IRQs and not in a NAPI or > > NAPI

Re: netif_tx_disable and lockless TX

2006-05-30 Thread Herbert Xu
On Tue, May 30, 2006 at 10:30:25PM -0700, Michael Chan wrote: > > I also don't see any problem. It looks like we don't have to set > xmit_lock_owner, right? Right, unless you want to call dev_queue_xmit recursively :) -- Visit Openswan at http://www.openswan.org/ Email: Herbert Xu ~{PmV>HI~} <[E

Re: netif_tx_disable and lockless TX

2006-05-30 Thread Michael Chan
David Miller wrote: > From: Herbert Xu <[EMAIL PROTECTED]> > Date: Wed, 31 May 2006 14:58:11 +1000 > > > Yes, TG3 does not disable IRQs when taking its TX lock. So do you see > > any problems with replacing the TG3 TX lock using xmit_lock? > > I don't see any. > I also don't see any problem.

Re: netif_tx_disable and lockless TX

2006-05-30 Thread Herbert Xu
On Tue, May 30, 2006 at 10:11:17PM -0700, David Miller wrote: > > It will come back when net channels arrive :-) This is because net > channel drivers will do all the work in hard IRQs and not in a NAPI or > NAPI-like context. I thought the current channel stuff is RX only. Is TX completion movi

Re: netif_tx_disable and lockless TX

2006-05-30 Thread David Miller
From: Herbert Xu <[EMAIL PROTECTED]> Date: Wed, 31 May 2006 14:58:11 +1000 > Hi Michael: > > On Tue, May 30, 2006 at 09:51:03PM -0700, Michael Chan wrote: > > > > > That's why I suggest that every NIC that uses this feature be forced > > > to do what TG3 does so only BH disabling is needed. Once

Re: netif_tx_disable and lockless TX

2006-05-30 Thread Herbert Xu
Hi Michael: On Tue, May 30, 2006 at 09:51:03PM -0700, Michael Chan wrote: > > > That's why I suggest that every NIC that uses this feature be forced > > to do what TG3 does so only BH disabling is needed. Once that's done > > they can just use xmit_lock and everyone will be happy again. > > As l

Re: netif_tx_disable and lockless TX

2006-05-30 Thread Michael Chan
Herbert Xu wrote: > On Tue, May 30, 2006 at 09:13:22PM -0700, David Miller wrote: > > As per the bug, I always keep thinking about changing how we > > do the lockless stuff by simply making xmit_lock a pointer. > > Drivers like tg3 can make it point to their local driver lock. > > This is equival

Re: netif_tx_disable and lockless TX

2006-05-30 Thread Herbert Xu
On Tue, May 30, 2006 at 09:13:22PM -0700, David Miller wrote: > > There were numbers posted for real drivers at one point, I > do remember that, and the gains were non-trivial. At least > enough to justify the feature. I wonder if there are other ways of achieving the same improvement. > As per

Re: netif_tx_disable and lockless TX

2006-05-30 Thread Roland Dreier
Herbert> However, lockless drivers do not take the xmit_lock so Herbert> this method is ineffective. Such drivers need to do Herbert> their own checking inside whatever locks that they do Herbert> take. For example, tg3 could get around this by checking Herbert> whether the qu

Re: netif_tx_disable and lockless TX

2006-05-30 Thread David Miller
From: Herbert Xu <[EMAIL PROTECTED]> Date: Wed, 31 May 2006 14:03:07 +1000 > I must say though that I'm becoming less and less impressed by the > lockless feature based on the number of problems that it has caused. > Does anyone have any hard figures as to its effectiveness (excluding > any stats

netif_tx_disable and lockless TX

2006-05-30 Thread Herbert Xu
Hi: It has occured to me that the function netif_tx_disable may be unsafe when used in drivers that do lockless transmission. In particular, the function relies on taking the xmit_lock to ensure that no further transmissions will occur until the queue is started again. However, lockless drivers