On Wed, 2006-06-28 at 00:18 -0400, Jeff Garzik wrote: > Herbert Xu wrote: > > On Tue, Jun 27, 2006 at 11:24:25PM -0400, Jeff Garzik wrote: > >> I don't see how that position has changed? > >> > >> http://linux-net.osdl.org/index.php/TOE > > > > Well I must say that RDMA over TCP smells very much like TOE. They've > > got an ARP table, a routing table, and presumably a TCP stack. > > A PCI device that presents itself as a SCSI controller, but under the > hood is really iSCSI-over-TCP smells like TOE. Running a virtualized > Linux guest on top of a proprietary stack [which provides networking > services to guests] also smells like TOE. :) >
I wonder if the existing iSCSI solutions handle next host mac addr changes, pmtu changes, etc? They _can_, if they implement the entire ARP and ICMP suites in HW/FW. Just curious if these vendors also see merit in the netevent changes I'm proposing... > If a TOE vendors wants to do TOE in a way that is transparent to the > kernel, more power to them. Such non-Linux TCP stack solutions still > suffer many of the problems listed at the web page above, but at least > they impose no burden on kernel maintenance. > > i.e. we really _do not_ want to get into the habit of co-managing arp > tables, routing tables, filtering rules, and dozens of other such > resources with multiple remote, independent TCP stack. We have enough > complexity as it is today, coordinating between the random variations of > SMP, uniprocessor, and NUMA machines out there. Not to mention > competing with under-the-hood firmware actions (ASF) on NICs. > > As an aside, RDMA over TCP just seems silly. TCP was _not_ meant to do > the things that RDMA users want. The infiniband/RDMA programming model > is an ultra-low-latency polling model where one or two apps are allowed > to completely consume the machine, either busy-waiting or processing > messages. > With RDMA over TCP, you can get the same ultra-low-latency, interrupt coalescing or avoidance, and copy avoidance as with Infiniband (with newly arriving 10Gb RDMA NICs). The benefit over IB, IMO, is the fact that your infrastructure is all IP. If you're interested in a kernel-mode app that benfits from RDMA, then check out NFS-RDMA, which runs today transparently over Infiniband and RDMA/TCP devices using the Infiniband RDMA-CM and verbs with minor changes to support RDMA/TCP devices. http://sourceforge.net/projects/nfs-rdma/ Steve. - To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
