On Wed, Mar 23, 2022, 9:54 AM Templin (US), Fred L < fred.l.temp...@boeing.com> wrote:
> Hi Tom, > > > -----Original Message----- > > From: Tom Herbert [mailto:t...@herbertland.com] > > Sent: Wednesday, March 23, 2022 6:19 AM > > To: Templin (US), Fred L <fred.l.temp...@boeing.com> > > Cc: Eggert, Lars <l...@netapp.com>; int-area@ietf.org; l...@eggert.org > > Subject: Re: [Int-area] IP Parcels improves performance for end systems > > > > On Tue, Mar 22, 2022 at 10:38 AM Templin (US), Fred L > > <fred.l.temp...@boeing.com> wrote: > > > > > > Tom, see below: > > > > > > > -----Original Message----- > > > > From: Tom Herbert [mailto:t...@herbertland.com] > > > > Sent: Tuesday, March 22, 2022 10:00 AM > > > > To: Templin (US), Fred L <fred.l.temp...@boeing.com> > > > > Cc: Eggert, Lars <l...@netapp.com>; int-area@ietf.org > > > > Subject: Re: [Int-area] IP Parcels improves performance for end > systems > > > > > > > > On Tue, Mar 22, 2022 at 7:42 AM Templin (US), Fred L > > > > <fred.l.temp...@boeing.com> wrote: > > > > > > > > > > Lars, I did a poor job of answering your question. One of the most > important aspects of > > > > > > > > > > IP Parcels in relation to TSO and GSO/GRO is that transports get > to use a full 4MB buffer > > > > > > > > > > instead of the 64KB limit in current practices. This is possible > due to the IP Parcel jumbo > > > > > > > > > > payload option encapsulation which provides a 32-bit length field > instead of just a 16-bit. > > > > > > > > > > By allowing the transport to present the IP layer with a buffer of > up to 4MB, it reduces > > > > > > > > > > the overhead, minimizes system calls and interrupts, etc. > > > > > > > > > > > > > > > > > > > > So, yes, IP Parcels is very much about improving the performance > for end systems in > > > > > > > > > > comparison with current practice (GSO/GRO and TSO). > > > > > > > > Hi Fred, > > > > > > > > The nice thing about TSO/GSO/GRO is that they don't require any > > > > changes to the protocol as just implementation techniques, also > > > > they're one sided opitmizations meaning for instance that TSO can be > > > > used at the sender without requiring GRO to be used at the receiver. > > > > My understanding is that IP parcels requires new protocol that would > > > > need to be implemented on both endpoints and possibly in some > routers. > > > > > > It is not entirely true that the protocol needs to be implemented on > both > > > endpoints . Sources that send IP Parcels send them into a > Parcel-capable path > > > which ends at either the final destination or a router for which the > next hop is > > > not Parcel-capable. If the Parcel-capable path extends all the way to > the final > > > destination, then the Parcel is delivered to the destination which > knows how > > > to deal with it. If the Parcel-capable path ends at a router somewhere > in the > > > middle, the router opens the Parcel and sends each enclosed segment as > an > > > independent IP packet. The final destination is then free to apply GRO > to the > > > incoming IP packets even if it does not understand Parcels. > > > > > > IP Parcels is about efficient shipping and handling just like the > major online > > > retailer service model I described during the talk. The goal is to > deliver the > > > fewest and largest possible parcels to the final destination rather > than > > > delivering lots of small IP packets. It is good for the network and > good for > > > the end systems both. If this were not true, then Amazon would send the > > > consumer 50 small boxes with 1 item each instead of 1 larger box with > all > > > 50 items inside. And, we all know what they would choose to do. > > > > > > > Do you have data that shows the benefits of IP Parcels in light of > > > > these requirements? > > > > > > I have data that shows that GSO/GRO is good for packaging sizes up to > 64KB > > > even if the enclosed segments will require IP fragmentation upon > transmission. > > > The data implies that even larger packaging sizes (up to a maximum of > 4MB) > > > would be better still. > > > > > > > Fred, > > > > You seem to be only looking at the problem from a per packet cost > > point of view. There is also per byte cost, particularly in the > > computation of the TCP/UDP checksum. The cost is hidden in modern > > implementations by checksum offload, and for segmentation offload we > > have methods to preserve the utility of checksum offload. IP parcels > > will have to also leverage checksum offload, because if the checksum > > is not offloaded then the cost of computing the payload checksum in > > CPU would dwarf any benefits we'd get by using segments larger than > > 64K. > > There is plenty of opportunity to apply hardware checksum offload since > the structure of a Parcel will be very standard. My experiments have been > with a protocol called LTP which is layered over UDP/IP as some other > upper layer protocols are. LTP includes a segment-by-segment checksum > that is used at its level in the absence of lower layer integrity checks, > so > for larger Parcels LTP would use that and turn off UDP checksums > altogether. You can't turn it off UDP checksums for IPv6 (except for narrow case of encapsulation). As far as I am aware, there are currently no hardware > checksum offload implementations available for calculating the > LTP checksums. > If it's a standard per packet Internet checksum then a lot of HW could do it. If it's something like CRC32 then probably not. LTP is a nice experiment, but I'm more interested as to the interaction between IP parcels and TCP or QUIC. > Speaking of standard, AFAICT GSO/GRO are doing something very > non-standard. GSO seems to be coding the IP ID field in the IPv4 > headers of packets with DF=1 which goes against RFC 6864. When > DF=1, GSO cannot simply claim the IP ID and code it as if there were > some sort of protocol. Or, if it does, there would be no way to > standardize it. > There was quite a bit of work and discussion on this in Linux. I believe the deviation from the standard was motivated by some deployed devices required the IPID be set on receive, and setting IPID with DF equals to 1 is thought to be innocuous. You may want to look at Alex Duyck's papers on UDP GSO, he wrote a lot of code in this area. Tom > Fred > > > > > Tom > > > > > Fred > > > > > > > Thanks, > > > > Tom > > > > > > > > > > > > > > > > > > > > > > > > Thanks - Fred > > > > > > > > > > _______________________________________________ > > > > > Int-area mailing list > > > > > Int-area@ietf.org > > > > > https://www.ietf.org/mailman/listinfo/int-area >
_______________________________________________ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
