Re: skbuff truesize incorrect.

2014-05-27 Thread David Miller
From: David Laight Date: Tue, 27 May 2014 15:23:14 + > True, you still need to sort out how to handle multiple ethernet > frames in the same 4k page, and arbitrary page boundaries within a > frame. Multiple ethernet frames in the same 4K page is used by several ethernet drivers already. NIU

RE: skbuff truesize incorrect.

2014-05-27 Thread David Laight
From: David Miller > From: David Laight > Date: Fri, 23 May 2014 08:52:13 + > > > The hardware will put multiple ethernet frames into a single USB bulk data > > message. To handle this the driver generates a URB that is (hopefully) long > > enough for the longest USB message (typically 32k is

Re: skbuff truesize incorrect.

2014-05-23 Thread David Miller
From: David Laight Date: Fri, 23 May 2014 08:52:13 + > The hardware will put multiple ethernet frames into a single USB bulk data > message. To handle this the driver generates a URB that is (hopefully) long > enough for the longest USB message (typically 32k is assumed to be enough). > The U

Re: skbuff truesize incorrect.

2014-05-23 Thread David Miller
From: Jim Baxter Date: Fri, 23 May 2014 10:21:19 +0100 > On 22/05/14 21:58, David Miller wrote: >> From: Jim Baxter >> Date: Thu, 22 May 2014 21:21:55 +0100 >> >>> If there are multiple clones for an skb all pointing at the same data, >>> will that distort the memory used when they all have the

Re: skbuff truesize incorrect.

2014-05-23 Thread Eric Dumazet
On Fri, 2014-05-23 at 08:44 -0700, Rick Jones wrote: > If you are measuring performance with the likes of netperf, you should > be able to get an idea of the performance effect from the change in > service demand (CPU consumed per unit of work) even if the maximum > throughput remains capped.

Re: skbuff truesize incorrect.

2014-05-23 Thread Rick Jones
On 05/23/2014 02:33 AM, Bjørn Mork wrote: Jim Baxter writes: I'll create and test a patch for the cdc_ncm host driver unless someone else wants to do that. I haven't really played with the gadget driver before, so I'd prefer if someone knowing it (Jim maybe?) could take care of it. If not, th

RE: skbuff truesize incorrect.

2014-05-23 Thread Eric Dumazet
On Fri, 2014-05-23 at 15:30 +, David Laight wrote: > From: Eric Dumazet > ... > > TCP stack uses fast clones, and current stack gives them a truesize of > > 2048 + sizeof(sk_buff), while it really should be 2048 + > > 2*sizeof(sk_buff) > > > > Luckily, GSO/TSO tends to reduce the error, as skb

RE: skbuff truesize incorrect.

2014-05-23 Thread David Laight
From: Eric Dumazet ... > TCP stack uses fast clones, and current stack gives them a truesize of > 2048 + sizeof(sk_buff), while it really should be 2048 + > 2*sizeof(sk_buff) > > Luckily, GSO/TSO tends to reduce the error, as skbs overhead is lower. Doesn't that affect the tx side - where the tru

Re: skbuff truesize incorrect.

2014-05-23 Thread Jim Baxter
On 23/05/14 14:47, Eric Dumazet wrote: > On Fri, 2014-05-23 at 12:13 +0100, Jim Baxter wrote: > >> What are the side effects of changing the truesize, if the original >> uncloned skb has the full truesize then isn't the potential memory usage >> still counted for the avoidance of OOM? > > Nope. T

Re: skbuff truesize incorrect.

2014-05-23 Thread Eric Dumazet
On Fri, 2014-05-23 at 11:33 +0200, Bjørn Mork wrote: > Jim Baxter writes: > > >> I'll create and test a patch for the cdc_ncm host driver unless someone > >> else wants to do that. I haven't really played with the gadget driver > >> before, so I'd prefer if someone knowing it (Jim maybe?) could t

Re: skbuff truesize incorrect.

2014-05-23 Thread Eric Dumazet
On Fri, 2014-05-23 at 12:13 +0100, Jim Baxter wrote: > What are the side effects of changing the truesize, if the original > uncloned skb has the full truesize then isn't the potential memory usage > still counted for the avoidance of OOM? Nope. This can be disastrous. A malicious remote peer ca

Re: skbuff truesize incorrect.

2014-05-23 Thread Jim Baxter
On 23/05/14 11:45, David Laight wrote: > From: Bjørn Mork [mailto:bj...@mork.no] >> David Laight writes: > ... >>> Note that some of the usb ethernet drivers allocate large skb then >>> lie about the truesize. >> >> Hmm, which drivers are these? > > $ grep truesize linux/drivers/net/usb/* > > as

RE: skbuff truesize incorrect.

2014-05-23 Thread David Laight
From: Bjørn Mork [mailto:bj...@mork.no] > David Laight writes: ... > > Note that some of the usb ethernet drivers allocate large skb then > > lie about the truesize. > > Hmm, which drivers are these? $ grep truesize linux/drivers/net/usb/* asix_88179_178a, smsc95xx, sr9700. David

Re: skbuff truesize incorrect.

2014-05-23 Thread Bjørn Mork
David Laight writes: > Since USB bulk data are terminated by a short fragment there is actually > no need for the URB be long enough for the full message. Provided the > URB are multiples of the USB message size (1k for USB 3) the message > can be received into multiple URB - the driver just has

Re: skbuff truesize incorrect.

2014-05-23 Thread Bjørn Mork
Jim Baxter writes: >> I'll create and test a patch for the cdc_ncm host driver unless someone >> else wants to do that. I haven't really played with the gadget driver >> before, so I'd prefer if someone knowing it (Jim maybe?) could take care >> of it. If not, then I can always make an attempt u

RE: skbuff truesize incorrect.

2014-05-23 Thread David Laight
From: Jim Baxter > On 22/05/14 21:58, David Miller wrote: > > From: Jim Baxter > > Date: Thu, 22 May 2014 21:21:55 +0100 > > > >> If there are multiple clones for an skb all pointing at the same data, > >> will that distort the memory used when they all have the same truesize? > > > > What is dist

Re: skbuff truesize incorrect.

2014-05-23 Thread Jim Baxter
On 22/05/14 21:58, David Miller wrote: > From: Jim Baxter > Date: Thu, 22 May 2014 21:21:55 +0100 > >> If there are multiple clones for an skb all pointing at the same data, >> will that distort the memory used when they all have the same truesize? > > What is distorted about it? > > Each clone

Re: skbuff truesize incorrect.

2014-05-23 Thread Jim Baxter
> But although the problem is the same, I believe the driver in question > isn't the one I have been looking at recently. The posted code snippet > was from the NCM gadget driver (drivers/usb/gadget/f_ncm.c), isn't that > right Jim? Yes this is the NCM Gadget driver. > > Yes, judging by this dis

RE: skbuff truesize incorrect.

2014-05-23 Thread David Laight
From: Eric Dumazet > On Thu, 2014-05-22 at 20:07 +0100, Jim Baxter wrote: > > > > > I have been investigating a network issue with bursts of network traffic > > over USB CDC-NCM, the issue is that the kernel is dropping packets > > because sk_rcvqueues_full() returns true due to skb2->truesize is

Re: skbuff truesize incorrect.

2014-05-23 Thread Bjørn Mork
David Miller writes: > From: Eric Dumazet > Date: Thu, 22 May 2014 14:03:21 -0700 > >> On Thu, 2014-05-22 at 13:58 -0700, Eric Dumazet wrote: >> >>> I would set rx_max (rx_urb_size) to SKB_MAX_HEAD(0) so that you do not >>> use high order allocations. >> >> Correction, that would need SKB_MAX_H

Re: skbuff truesize incorrect.

2014-05-22 Thread David Miller
From: Eric Dumazet Date: Thu, 22 May 2014 14:03:21 -0700 > On Thu, 2014-05-22 at 13:58 -0700, Eric Dumazet wrote: > >> I would set rx_max (rx_urb_size) to SKB_MAX_HEAD(0) so that you do not >> use high order allocations. > > Correction, that would need SKB_MAX_HEAD(NET_SKB_PAD + NET_IP_ALIGN),

Re: skbuff truesize incorrect.

2014-05-22 Thread Eric Dumazet
On Thu, 2014-05-22 at 13:58 -0700, Eric Dumazet wrote: > I would set rx_max (rx_urb_size) to SKB_MAX_HEAD(0) so that you do not > use high order allocations. Correction, that would need SKB_MAX_HEAD(NET_SKB_PAD + NET_IP_ALIGN), because drivers/net/usb/usbnet.c calls __netdev_alloc_skb_ip_align().

Re: skbuff truesize incorrect.

2014-05-22 Thread David Miller
From: Jim Baxter Date: Thu, 22 May 2014 21:21:55 +0100 > If there are multiple clones for an skb all pointing at the same data, > will that distort the memory used when they all have the same truesize? What is distorted about it? Each clone references exactly that much backing memory. -- To uns

Re: skbuff truesize incorrect.

2014-05-22 Thread Eric Dumazet
On Thu, 2014-05-22 at 20:07 +0100, Jim Baxter wrote: > > I have been investigating a network issue with bursts of network traffic > over USB CDC-NCM, the issue is that the kernel is dropping packets > because sk_rcvqueues_full() returns true due to skb2->truesize is always > 32960 instead of SKB_

Re: skbuff truesize incorrect.

2014-05-22 Thread Jim Baxter
On 22/05/14 20:21, David Miller wrote: > From: Jim Baxter > Date: Thu, 22 May 2014 20:07:41 +0100 > >> My questions are: >> >> 1) Which buffer size does truesize represent, is it the total buffer or >> just the data related to the relevant skb? >> >> 2) If truesize is for the skb it is contained

Re: skbuff truesize incorrect.

2014-05-22 Thread Eric Dumazet
On Thu, 2014-05-22 at 21:21 +0100, Jim Baxter wrote: > OK, so it is the value of the memory that has been allocated for the SKB. > If there are multiple clones for an skb all pointing at the same data, > will that distort the memory used when they all have the same truesize? Its always better to

Re: skbuff truesize incorrect.

2014-05-22 Thread Jim Baxter
On 22/05/14 20:59, Eric Dumazet wrote: > On Thu, 2014-05-22 at 20:39 +0100, Jim Baxter wrote: > >> I now think that the correct solution here is to create a new smaller >> skb and copy the data from the sub packets into it. > > For low speed devices, this is indeed the best way. > > (this is cal

Re: skbuff truesize incorrect.

2014-05-22 Thread Eric Dumazet
On Thu, 2014-05-22 at 20:39 +0100, Jim Baxter wrote: > I now think that the correct solution here is to create a new smaller > skb and copy the data from the sub packets into it. For low speed devices, this is indeed the best way. (this is called copybreak in some nic drivers) -- To unsubscri

Re: skbuff truesize incorrect.

2014-05-22 Thread Jim Baxter
On 22/05/14 20:25, Vlad Yasevich wrote: > On 05/22/2014 03:07 PM, Jim Baxter wrote: >> >> skb2 = skb_clone(skb, GFP_ATOMIC); >> if (skb2 == NULL) >> goto err; >> >> if (!skb_pull(skb2, index)) { >> ret = -EOVERFLOW; >> goto err; >> } > > This assumes that yo

Re: skbuff truesize incorrect.

2014-05-22 Thread Vlad Yasevich
On 05/22/2014 03:07 PM, Jim Baxter wrote: > Hi, I was hoping you can help me with some questions. > > I have been investigating a network issue with bursts of network traffic > over USB CDC-NCM, the issue is that the kernel is dropping packets > because sk_rcvqueues_full() returns true due to skb2

Re: skbuff truesize incorrect.

2014-05-22 Thread David Miller
From: Jim Baxter Date: Thu, 22 May 2014 20:07:41 +0100 > My questions are: > > 1) Which buffer size does truesize represent, is it the total buffer or > just the data related to the relevant skb? > > 2) If truesize is for the skb it is contained within should it be > updated during the call to

skbuff truesize incorrect.

2014-05-22 Thread Jim Baxter
Hi, I was hoping you can help me with some questions. I have been investigating a network issue with bursts of network traffic over USB CDC-NCM, the issue is that the kernel is dropping packets because sk_rcvqueues_full() returns true due to skb2->truesize is always 32960 instead of SKB_TRUESIZE(s