Tom, > -----Original Message----- > From: Tom Herbert [mailto:t...@herbertland.com] > Sent: Tuesday, March 22, 2022 1:47 PM > To: Robinson, Herbie <herbie.robin...@stratus.com> > Cc: Templin (US), Fred L <fred.l.temp...@boeing.com>; int-area@ietf.org > Subject: Re: [Int-area] [EXTERNAL] Re: IP Parcels improves performance for > end systems > > EXT email: be mindful of links/attachments. > > > > On Tue, Mar 22, 2022 at 10:40 AM Robinson, Herbie > <herbie.robin...@stratus.com> wrote: > > > > > The nice thing about TSO/GSO/GRO is that they don't require any > > > changes to the protocol as just implementation techniques, also > > > they're one sided opitmizations meaning for instance that TSO can be > > > used at the sender without requiring GRO to be used at the receiver. > > > My understanding is that IP parcels requires new protocol that would > > > need to be implemented on both endpoints and possibly in some routers. > > > Do you have data that shows the benefits of IP Parcels in light of > > > these requirements? > > > > The various segmentation offload optimizations done by NICs have a number > > of drawbacks that the parcel scheme does not have: > > > > 1. They are still limited to 65K bytes. > > That's true, however we do start to get diminishing returns above 64K > (this is why I asked for real data, the existing offloads have already > solved the bulk of the problem AFAICT). The primary problem with these > optimizations is that they are opportunistic. e.g. we can only send > >64K on a TCP connection if that's available CWND. In the worst case, > e.g. TCP is in slow start, then we may only bei sending a few segments > so such optimizations don't help and can actually hurt if those > deploying these optimization under provision for the worst case that > the optimizations can't be used.
I can most definitely show that some ULPs that use UDP get much better single-segment (i.e., singleton IP Parcel) performance proportionally to the segment size. With NFS over UDP many decades ago, it was plainly evident that 8KB segments outperformed smaller segment sizes even though IP fragmentation was used. With LTP over UDP today, I can show that performance is directly related to segment size up to the maximum segment size of ~64KB - again, with heavy reliance on IP fragmentation. And, the performance for a singleton IP Parcel with segment size of ~64KB was much better than for an N-segment IP Parcel with smaller segment size (e.g., ~9KB, ~16KB, etc.). The performance curves suggest that with adequate hardware multi-segment IP Parcels with larger segment sizes would give the best performance, i.e., even if the Parcel exceeds 64KB by a little or even a lot. Fred > > > 2. The number of streams that can be optimized are significantly limited > > by memory in the NIC and usually by the size of the receive > queue. In other words, it words some of the time, but not all of the time. > > I don't see how IP parcels are any different in this regard, it would > similarly be reassembling segments into larger segments which requires > memory if the NIC is doing it. There might be an argument that this is > standard means to perform intelligent fragmentation, but again that > needs to be measured against the complexity and feasibility of > deploying a new on the wire protocol. > > > 3. The NIC is responsible for checksum computation and validation. If one > > is operating in a fault tolerant environment, one wants the > checksum computation/checking done in the fault tolerant domain (i.e., the > host, not the NIC). > > We still need checksum offload with IP parcels. For instance, if a > parcel contains 10 TCP segments we'd need the device to compute and > set 10 checksums. I'm not sure how this could work in IP parcels other > than using similar techniques for TSO. > > Tom > > > > > > > > > _______________________________________________ > > Int-area mailing list > > Int-area@ietf.org > > https://www.ietf.org/mailman/listinfo/int-area _______________________________________________ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
