>
> > It does not exist today AFAIK. The hope of such a solution will prevent
> > the inclusion of TOE technology that exists today?
>
>
> TOE has a solid track record of being a point-in-time solution with
> decreased features and increased maintenance headaches.
>
> That's what prevents its i
From: Guy Thornley <[EMAIL PROTECTED]>
Date: Mon, 22 Aug 2005 11:06:13 +1200
> Does the 2.4 kernel support chained sk_buff's along the receive path? (This
> is news to me!) Is there any driver that does this, that can be read as an
> instructive example?
It does, just that no driver takes advanta
On Fri, Aug 19, 2005 at 07:05:06PM +0200, Andi Kleen wrote:
> > Right. The other issue with jumbos frames (9000MTU) is that
> > the allocation needed is just over 2 pages for 4K page size
> > machines (common case). 3 page contig allocations tend to fail
> > once a server is heavily loaded and memo
All other things being equal, it is better not to put packets into the
network faster than it can drain them out. Large bursts increase delay
variation, and increase the probability that two or more packets in a
connection will be dropped within an RTT (not every box is implementing
AQM yet). Ne
On Sun, 21 Aug 2005, David S. Miller wrote:
> LRO will work, and it's the negative attitude of the TOE folks that
> inspires me to want to help out the LRO folks and ignore the TOE mania
> altogether.
Dave you critized the black and white attitude before. It seems that you
are the only one in th
IL PROTECTED]; [EMAIL PROTECTED]
> Subject: Re: [PATCH] TCP Offload (TOE) - Chelsio
>
> And -by the way- it seems that LRO is patented. Hope you made
> arrangements for Linux to use that technology? Are others
> vendors allowed to implement LRO in their hardware?
>
Ahh, I w
> If you have a flow cache, keyed on saddr/daddr/sport/dport then you
> can keep a growing LRO limit. For example, when a flow cache entry is
> created, use a LRO limit of 2 frames. Each time the LRO limit is
> reached, increase the LRO limit by one (until you hit the largest
> LRO supported, whi
LRO will just stop accumulating when out-of-sequence data arrives.
Nothing complicated at all.
Unless the NIC keeps state, it is not always able to know if data is out of
sequence.
The LRO timing is not complicated, the packet limit is simply a
linearly increasing value that just makes sure
On Sun, 2005-08-21 at 01:53 -0700, Wael Noureddine wrote:
> >> How do you intend on avoiding huge stretch ACKs?
> >
> > The implication is that stretch ACKs are bad, which is wrong.
> > Oh yes, that's right, you're the same person who earlier in this
> > thread tried to teach us that bursty TCPs ar
How do you intend on avoiding huge stretch ACKs?
The implication is that stretch ACKs are bad, which is wrong.
Oh yes, that's right, you're the same person who earlier in this
thread tried to teach us that bursty TCPs are non-standard :-)
Are you saying that burstiness is not an issue?
There i
From: "Wael Noureddine" <[EMAIL PROTECTED]>
Date: Sun, 21 Aug 2005 00:54:51 -0700
> > You could also tweak the LRO timeout in a similar fashion based upon
> > traffic patterns as well. In fact, extremely sophisticated things can
> > be done here to deal with the LRO timing as seen on WAN vs. LAN
From: "Wael Noureddine" <[EMAIL PROTECTED]>
Date: Sun, 21 Aug 2005 00:17:17 -0700
> How do you intend on avoiding huge stretch ACKs?
The implication is that stretch ACKs are bad, which is wrong.
Oh yes, that's right, you're the same person who earlier in this
thread tried to teach us that bursty
You could also tweak the LRO timeout in a similar fashion based upon
traffic patterns as well. In fact, extremely sophisticated things can
be done here to deal with the LRO timing as seen on WAN vs. LAN
streams.
The accurate statement is "extremely complicated things need to be done here
to de
Date: Sat, 20 Aug 2005 21:55:22 -0700 (PDT)
I bet the tricks that we hack into the TCP/IP stack for LSO and for
LRO will turn out to be more difficult to maintain than the proposed
TOE hooks.
LRO is going to be mostly transparent.
How do you intend on avoiding huge stretch ACKs?
As far as
Christoph Lameter wrote:
On Sat, 20 Aug 2005, David S. Miller wrote:
But by in large, if a stateless alternative ever exists to
get the same performance benefit as TOE, it will undoubtedly
be preferred by the Linux networking maintainers, by in large.
So you TOE guys are fighting more than an up
> -Original Message-
> From: David S. Miller [mailto:[EMAIL PROTECTED]
> Sent: Saturday, August 20, 2005 9:40 PM
>
> If you have a flow cache, keyed on saddr/daddr/sport/dport
> then you can keep a growing LRO limit. For example, when a
> flow cache entry is created, use a LRO limi
From: Christoph Lameter <[EMAIL PROTECTED]>
Date: Sat, 20 Aug 2005 21:55:22 -0700 (PDT)
> I bet the tricks that we hack into the TCP/IP stack for LSO and for
> LRO will turn out to be more difficult to maintain than the proposed
> TOE hooks.
LRO is going to be mostly transparent.
> As far as I c
On Sat, 20 Aug 2005, David S. Miller wrote:
> From: Christoph Lameter <[EMAIL PROTECTED]>
> Date: Sat, 20 Aug 2005 21:16:16 -0700 (PDT)
> > It does not exist today AFAIK. The hope of such a solution will prevent
> > the inclusion of TOE technology that exists today?
> What you say isn't exactly t
From: "Leonid Grossman" <[EMAIL PROTECTED]>
Date: Sat, 20 Aug 2005 21:17:19 -0400
> OLS.pdf at ftp ns1.s2io.com user: linuxdocs password: HALdocs
Looks good, the LRO bits.
It seems important that the OS can specify the LRO sizing limits.
Even better, it would help also to have a flow cache of so
Christoph Lameter wrote:
On Sat, 20 Aug 2005, David S. Miller wrote:
But by in large, if a stateless alternative ever exists to
get the same performance benefit as TOE, it will undoubtedly
be preferred by the Linux networking maintainers, by in large.
So you TOE guys are fighting more than an up
From: Christoph Lameter <[EMAIL PROTECTED]>
Date: Sat, 20 Aug 2005 21:16:16 -0700 (PDT)
> It does not exist today AFAIK. The hope of such a solution will prevent
> the inclusion of TOE technology that exists today?
What you say isn't exactly true, as Lenoid Grossman and others are
working on LRO
On Sat, 20 Aug 2005, David S. Miller wrote:
> But by in large, if a stateless alternative ever exists to
> get the same performance benefit as TOE, it will undoubtedly
> be preferred by the Linux networking maintainers, by in large.
> So you TOE guys are fighting more than an uphill battle.
It do
From: "Leonid Grossman" <[EMAIL PROTECTED]>
Date: Sat, 20 Aug 2005 21:17:19 -0400
> Which reminds me - some people noted in Ottawa that USO is arguably a
> misleading name for "UDP TSO", so better suggestions are welcome.
UDP Fragmentation Offload, aka. UFO ? :-)
-
To unsubscribe from this list:
etdev@vger.kernel.org; [EMAIL PROTECTED]
> Subject: Re: [PATCH] TCP Offload (TOE) - Chelsio
>
> From: "Leonid Grossman" <[EMAIL PROTECTED]>
> Date: Sat, 20 Aug 2005 17:43:11 -0400
>
> > BTW any comments on the LRO algorithm in my OLS slides are most
> > wel
> But by in large, if a stateless alternative ever exists to
> get the same performance benefit as TOE, it will undoubtedly
> be preferred by the Linux networking maintainers, by in large.
> So you TOE guys are fighting more than an uphill battle.
Nevertheless, this constitutes a reasonable starti
From: "Leonid Grossman" <[EMAIL PROTECTED]>
Date: Sat, 20 Aug 2005 17:43:11 -0400
> BTW any comments on the LRO algorithm in my OLS slides are most welcome;
> we are looking to extend the implementation in the next ASIC.
Pointer?
-
To unsubscribe from this list: send the line "unsubscribe netdev"
From: "Wael Noureddine" <[EMAIL PROTECTED]>
Date: Sat, 20 Aug 2005 15:43:06 -0700
> All good points. However, unlike LRO, TOE actually can also reduce
> per-byte costs on receive by allowing zero copy with DDP.
Combined with Intel's I/O AT stuff, LRO can potentially make
the per-byte costs transp
From: Christoph Lameter <[EMAIL PROTECTED]>
Date: Sat, 20 Aug 2005 15:02:00 -0700 (PDT)
> I worked through the alternatives last year including some of the large
> packet tricks that are not really standard conformant. None of these
> was really satisfactory and I ended up wasting a lot of my con
From: "Wael Noureddine" <[EMAIL PROTECTED]>
Date: Sat, 20 Aug 2005 13:46:33 -0700
> That's a good point. Why not offer both alternatives and let the
> customers decide what they want? Why would there be a veto against TOE
> if it can be supported non-intrusively and with virtually no changes to
>
> TSO and TOE both help significantly with the per-packet costs. They
are
> effectively equivalent here to using larger packets. Doing zero-copy
and
> checksum offloading helps with the per-byte costs, and is possible
today
> with stock Linux, and I believe most TOE implementations do. But TO
On Sat, 20 Aug 2005, David S. Miller wrote:
> Christoph, you're a really bright guy, perhaps you can sit and come up
> with some other ideas which would act as stateless alternatives to
> TOE? I bet you can do it, if you would simply try...
I worked through the alternatives last year including s
> Here is one idea. Do a reverse LSO, have a dynamic cache on
> the network card watching saddr/daddr/sport/dport flows, and
> accumulate as many in-order TCP packets as possible into one
> large R-LSO frame.
> This accumulation is timed out by a length and time parameter
> programmable in th
On Aug 20, 2005, at 1:57 PM, Christoph Lameter wrote:
On Fri, 19 Aug 2005, Andi Kleen wrote:
Hmm - but is a 9k or 16k packet on the wire not equivalent to a micro
burst?
(actually it is not that micro compared to 1.5k packets). At least
against
burstiness they don't help and make things even
> > We are discussing something that is not useful for todays
> network load
> > and not standardized. TOE is the only answer to offloading
> transfers of
> > data encountered in contemporary networks.
>
> It is talk like this that makes me want to not participate in such
> threads "TOE i
From: Christoph Lameter <[EMAIL PROTECTED]>
Date: Sat, 20 Aug 2005 10:57:51 -0700 (PDT)
> We are discussing something that is not useful for todays network load
> and not standardized. TOE is the only answer to offloading transfers of
> data encountered in contemporary networks.
It is talk like
On Fri, 19 Aug 2005, Andi Kleen wrote:
> Hmm - but is a 9k or 16k packet on the wire not equivalent to a micro burst?
> (actually it is not that micro compared to 1.5k packets). At least against
> burstiness they don't help and make things even worse because the bursts
> cannot be split up anymor
EMAIL PROTECTED]; [EMAIL PROTECTED];
> netdev@vger.kernel.org; [EMAIL PROTECTED]
> Subject: Re: [PATCH] TCP Offload (TOE) - Chelsio
>
> On Friday 19 August 2005 01:00 pm, Leonid Grossman wrote:
> > > -Original Message-
> > > > deployment of larger MTUs, b
I find the "no toe, no way" attitude strange.
I've seen a number of server applications that:
a] move a lot of data over TCP.. let's say around 1 Gbps over a hundred
concurrent flows.
b] spend a significant amount of cycles in the kernel stack doing this.
c] spend the rest of their cycles doing us
> Each TOE implementation is locked in time by the speed of the NIC.
> Given time, the network stack will -exceed- the speed of
> today's TOE NICs.
>
> You can see this with 100mbps TOE NICs, which are slower than today's
> software net stack, with today's software net stack being more
> featu
> On the spec website, the current results have it off.
That was because the old implementation violated the congestion
window. With David's new superTSO the next generation of benchmarks
will likely have it on again.
-Andi
-
To unsubscribe from this list: send the line "unsubscribe netdev" in
th
On Friday 19 August 2005 01:00 pm, Leonid Grossman wrote:
> > -Original Message-
> > > deployment of larger MTUs, but NIC vendors probably can.
>
> This is already done, both on the hardware and on the OS side.
(Sorry if this is getting a bit offtopic for netdev.)
I know of a number of sit
We all agree that there is a problem, and each is offering a solution. The
point is that claiming stateless offload is ideal and that TOE is just bad
is not objective. Each has its pros and cons. The performance benefits of
TOE have been demonstrated for many real applications, and we've seen qu
> Right. The other issue with jumbos frames (9000MTU) is that
> the allocation needed is just over 2 pages for 4K page size
> machines (common case). 3 page contig allocations tend to fail
> once a server is heavily loaded and memory gets fragmented.
That's just a driver bug. The driver should be
EMAIL PROTECTED]; netdev@vger.kernel.org; [EMAIL PROTECTED]
> Subject: Re: [PATCH] TCP Offload (TOE) - Chelsio
>
> > I'm personally not a big fan of TSO or TOE. They both add a lot of
> > complexity to the network stack, and have other downsides.
> The *bes
Andi Kleen wrote:
I'm personally not a big fan of TSO or TOE. They both add a lot of complexity
to the network stack, and have other downsides. The *best* way to solve
these problems is to engineer technologies to use larger packet sizes. Even
at 9k (or better yet 16k) the advantages of thes
> I'm personally not a big fan of TSO or TOE. They both add a lot of
> complexity
> to the network stack, and have other downsides. The *best* way to solve
> these problems is to engineer technologies to use larger packet sizes. Even
> at 9k (or better yet 16k) the advantages of these offloa
On Friday 19 August 2005 12:37 am, Wael Noureddine wrote:
> > The is no RFC violated by being "bursty". Show me the RFC where TCP
> > burstiness is "standardized". This is yet another strawman.
>
> You surely know this is a recurring theme in all congestion control RFCs
> (RFC2581 in particular),
> Now I can take you even less seriously. In RFC2581, they are talking
> about unloading a burst of data into a connection where there has been
> significant idle time since the most recent data send.
To be fair Linux would be using TSO in this case too and therefore
cause bursts. But it also wou
From: "Wael Noureddine" <[EMAIL PROTECTED]>
Subject: Re: [PATCH] TCP Offload (TOE) - Chelsio
Date: Thu, 18 Aug 2005 21:37:07 -0700
> > The is no RFC violated by being "bursty". Show me the RFC where TCP
> > burstiness is "standardized". This is ye
The is no RFC violated by being "bursty". Show me the RFC where TCP
burstiness is "standardized". This is yet another strawman.
You surely know this is a recurring theme in all congestion control RFCs
(RFC2581 in particular),
as well as in the "Known TCP Implementation Problems" RFC2525.
-
From: Christoph Lameter <[EMAIL PROTECTED]>
Date: Thu, 18 Aug 2005 20:58:39 -0700 (PDT)
> On Thu, 18 Aug 2005, David S. Miller wrote:
>
> > The same performance can be obtained with stateless offloads.
> > You continually ignore this possibility, as if TOE is the only
> > way.
>
> TCP is a state
From: "Wael Noureddine" <[EMAIL PROTECTED]>
Date: Thu, 18 Aug 2005 20:50:17 -0700
> Can you explain why TOE is a hack while stateless offload is not?
No knowledge of TCP internals necessary, that defines a clean
and maintainable barrier between the device and the network
stack. It also allows all
Christoph Lameter wrote:
We may have TOE in $40 network cards. In fact given the way things shape
up there is the possibility that it may become difficult to get NICs
without TOE next year.
People have been saying this every year.
Every year, we go through this argument.
Every year, people
On Thu, 18 Aug 2005, David S. Miller wrote:
> The same performance can be obtained with stateless offloads.
> You continually ignore this possibility, as if TOE is the only
> way.
TCP is a stateful protocol and what can be done with stateless
offloads is very limited.
-
To unsubscribe from this
On Thu, 18 Aug 2005, David S. Miller wrote:
> Wouldn't you rather have a commoditized $40.00USD gigabit network card
> that got TOE level performance? I guess that question's answer depends
> upon whether you have some financial state in a company doing TOE :-)
We may have TOE in $40 network car
Christoph Lameter wrote:
On Thu, 18 Aug 2005, David S. Miller wrote:
And once again it will be "niche", and very far from commodity.
A specialized optimization for a very small and specialized audience,
ie. not appropriate for Linux upstream.
The TOE method will gradually become standard si
From: Christoph Lameter <[EMAIL PROTECTED]>
Date: Thu, 18 Aug 2005 20:50:18 -0700 (PDT)
> The TOE method will gradually become standard simply because it allows
> performance that cannot be obtained now with existing hardware. And we
> may be at some speed boundary for the hardware given the lim
From: Digital Aryan <[EMAIL PROTECTED]>
Date: Fri, 19 Aug 2005 09:18:45 +0530
> And seeing what has happened during 100Mbit, 1Gbit and 10Gbit it seems
> reqirements for networking are always "one step ahead" and the cpu, memory,
> bus-bandwidth will take time to match the requirements. Had this ev
On Thu, 18 Aug 2005, David S. Miller wrote:
> And once again it will be "niche", and very far from commodity.
> A specialized optimization for a very small and specialized audience,
> ie. not appropriate for Linux upstream.
The TOE method will gradually become standard simply because it allows
p
> With stateless offloading schemes? Absolutely it is possible.
>
> Even without stateless offloading, if it can't be done today, then
> they will soon.
>
> This is what has always happened in the past, people were preaching
> for TOE back when 100Mbit ethernet was "new and fast". But you
> cer
This is what has always happened in the past, people were preaching
for TOE back when 100Mbit ethernet was "new and fast". But you
certainly don't see anyone trying to justify TOE for those link
speeds today. The same will happen for 1Gbit and 10Gbit links
a year or so from now, the cpu, memor
From: Christoph Lameter <[EMAIL PROTECTED]>
Date: Thu, 18 Aug 2005 20:39:41 -0700 (PDT)
> In that time frame people will have TOEs for even higher speeds.
And once again it will be "niche", and very far from commodity.
A specialized optimization for a very small and specialized audience,
ie. not
On Thu, 18 Aug 2005, David S. Miller wrote:
> This is what has always happened in the past, people were preaching
> for TOE back when 100Mbit ethernet was "new and fast". But you
> certainly don't see anyone trying to justify TOE for those link
> speeds today. The same will happen for 1Gbit and
From: Christoph Lameter <[EMAIL PROTECTED]>
Date: Thu, 18 Aug 2005 17:49:09 -0700 (PDT)
> > I am still very much against TOE going into the Linux networking
> > stack. There are ways to obtain TOE's performance without
> > necessitating stateful support in the cards, everything that's
> > worthwh
On Thu, 18 Aug 2005, David S. Miller wrote:
> The point remains that TOE creates an ENORMOUS support burdon
> upon us, and makes bugs harder to field even if we add the
> "TOE Taint" thing.
Simply switch off the TOE to see if its TOE or the OS stack.
TCP is fairly standard though this is a pretty
From: Timur Tabi <[EMAIL PROTECTED]>
Date: Thu, 18 Aug 2005 17:45:13 -0500
> I think a more accurate question would be, "what TCP/IP stack am I
> talking to, today?" You're making it sound as if TOE fundamentally
> changes the entire Linux kernel, when it only affects networking.
Networking is a
Jeff Garzik wrote:
1) RFC compliance differs based on whether you use a TOE NIC, or Linux
software stack. "What Linux am I talking to, today?"
I think a more accurate question would be, "what TCP/IP stack am I talking to, today?"
You're making it sound as if TOE fundamentally changes the ent
On 8/12/05, David S. Miller <[EMAIL PROTECTED]> wrote:
> From: Dimitris Michailidis <[EMAIL PROTECTED]>
> Date: Fri, 12 Aug 2005 10:00:12 -0700
>
> > On 8/12/05, David S. Miller <[EMAIL PROTECTED]> wrote:
> > > This would mean that every time we wish to change the data structures
> > > and interfa
From: Dimitris Michailidis <[EMAIL PROTECTED]>
Date: Fri, 12 Aug 2005 10:00:12 -0700
> On 8/12/05, David S. Miller <[EMAIL PROTECTED]> wrote:
> > This would mean that every time we wish to change the data structures
> > and interfaces for TCP socket lookup, your drivers would need to
> > change.
>
From: Dimitris Michailidis <[EMAIL PROTECTED]>
Date: Fri, 12 Aug 2005 10:22:47 -0700
> This is true. There is nothing fundamentally preventing both passive
> and active opens to check netfilter before OKing a connection. Once a
> connection is established, it's rather impractical to run each of
On 8/12/05, Mitchell Blank Jr <[EMAIL PROTECTED]> wrote:
> I'm fairly pessimistic about full TOE also, I just want to see the patch
> cleaned up a bit so we can see the exact impact it would have. The RX
> optimization work presented in the Neterion and Intel papers at OLS sounds a
> lot more int
On 8/12/05, David S. Miller <[EMAIL PROTECTED]> wrote:
>
> > - static was removed from functions '__tcp_inherit_port' & '__tcp_v4_hash'
> > because these are called outside of tcp_ipv4.c from the TOM driver.
>
> There is no way you're going to be allowed to call such deep TCP
> internals from you
I'm fairly pessimistic about full TOE also, I just want to see the patch
cleaned up a bit so we can see the exact impact it would have. The RX
optimization work presented in the Neterion and Intel papers at OLS sounds a
lot more interesting to me though.
However, I do want to comment on one state
David S. Miller wrote:
From: Scott Bardone <[EMAIL PROTECTED]>
Date: Thu, 11 Aug 2005 23:16:14 -0700
- static was removed from functions '__tcp_inherit_port' & '__tcp_v4_hash'
because these are called outside of tcp_ipv4.c from the TOM driver.
There is no way you're going to be allowed to c
The networking gurus can comment on the internals of your patch better than
I can. Just a few style notes though:
> +#ifdef CONFIG_TCP_OFFLOAD
> +#define NETIF_F_TCPIP_OFFLOAD65536 /* Can offload TCP/IP */
> +#endif
No need to protect this inside CONFIG_* option
> +/* TOE API */
> +#i
From: Scott Bardone <[EMAIL PROTECTED]>
Date: Thu, 11 Aug 2005 23:16:14 -0700
> - static was removed from functions '__tcp_inherit_port' & '__tcp_v4_hash'
> because these are called outside of tcp_ipv4.c from the TOM driver.
There is no way you're going to be allowed to call such deep TCP
intern
76 matches
Mail list logo
