Fred, I did some homework in the last few days, and I would describe your characterization of my comments as "under-informed" as being overly charitable :-( My apologies for wasting your time and everyone else's. This tutorial <https://www.sipanda.io/post/segmentation-offload-and-protocols-let-s-be-friends> from Tom Herbert, along socket API documentation for Windows <https://learn.microsoft.com/en-us/windows/win32/winsock/ipproto-udp-socket-options> and Linux <https://man7.org/linux/man-pages/man7/udp.7.html>, made it amply clear what is going on: GSO and GRO are specialized *implementation techniques*. In particular, the UDP variants, which are sometimes called USO and URO, allow an application to send or receive, in a single API call, a block of J datagrams, the first J-1 of which are all the same size L, with the Jth being no larger than the others. The sequence of packets that appear on the wire is not any different than what would have been sent had the application used J separate send API calls. USO and URO can be used independently by the two endpoints. When URO is used, the segment size L is attached to the packet's metadata when it is passed to the application so that upper layers can deduce the boundaries of the original UDP packets. The usage is sketched in Appendix D of draft-templin-6man-parcels2.
If I (finally) understand this all correctly, the IP Parcels specification provides a means to package the sequence of UDP datagrams that would be created by the USO API inside of a single IPv4 or IPv6 packet. If that IPv4 or IPv6 packet needs to be forwarded to a node that does not understand IP Parcels, it is converted into the same series of separately transmitted UDP datagrams that would be generated on the wire by USO today, except that a Parcel Parameters Option is appended to each of those UDP datagrams. The intent is that a receiving node that does not understand parcels or the Parcel Parameters Option would process those datagrams as it does today. URO should also work as it does today, provided that the presence of an options trailer does not interfere with URO as presently implemented (which as far as I know is an open question). With that being said, I do have some comments about the treatment of UDP options in IP Parcels, plus some more general ones: 1.) The plan of placing the UDP options common to all UDP datagrams in the parcel in a trailer will surely work; I understand that the two bytes immediately preceding the end of the parcel, as indicated by the Parcel Payload Length, contain the option field length, and it is not part of the options. But it is not necessary to do this. The UDP options could just as easily be placed before or after the parcel integrity block. The only reasons UDP options go in a trailer in regular UDP datagrams is because there is no other place to put them and still be backward compatible with implementations that are unaware of UDP options. IP Parcels amount to a new protocol, and the layout of a parcel is not constrained by backward compatibility. 2.) If the intent is that IP Parcels for UDP will use the same API as USO, there seems to be an implicit assumption that there will be an extension of that API that will allow for specification of a set of UDP options common to all datagrams in the parcel. As Tom Herbert noted in an earlier message, whether or not things will play out this way is an open question. Indeed, as far as I can tell, the current UDP options draft (now in AD review) defines only one option -- TIME -- that would actually be useful in this context, and that option is not one of the ones that are mandatory to implement. 3.) Perhaps I missed it, and I apologize if so, but I didn't see in draft-templin-6man-parcels2 what Next Header value would be placed in the IPv6 extension header immediately preceding the parcel payload area (typically the extension header would be the Hop-by-Hop options header containing the IPv6 Parcel Payload Option, but it could also be a Destination Options header if one followed the HbH Options header). That Next Header value should NOT be 6 (TCP) or 17 (UDP), because the IPv6 Payload Length is non-zero and the parcel payload is not laid out like a conventional TCP or UDP packet but is something else altogether, despite containing a notional TCP or UDP header. It is NOT sufficient to rely on the IPv6 Parcel Payload Option to let receivers know the difference, for a parcel-unaware receiver may well be configured to ignore and skip over any HbH Options Header or, if it is configured to process the HbH Options header, may not recognize the option and skip over it. Having a unique Next Header value (or Protocol Number) to identify a parcel payload would ensure that a parcel-unaware receiver would not misinterpret a parcel as something else. I would strongly recommend that the draft be modified to specify this and to include an appropriate request to the IANA. Granted, it would be necessary to add something to the Parcel Payload Option or to the parcel payload itself to identify the protocol it carries. Note that the RFC 2675 Jumbo Payload Option did not need this because the IPv6 Payload Length in that spec is zero, and receivers who do not implement that option would see an empty IPv6 payload and drop the packet. 4.) Note that comments (1) and (3) above also apply to Advanced Parcels. 5.) Given that the TCP or UDP Parcel Parameters Option is expendable -- that is, correctness will not be sacrificed if it is ignored or omitted -- I need to question its utility. Noting that it is different in every packet, Tom's tutorial <https://www.sipanda.io/post/segmentation-offload-and-protocols-let-s-be-friends> suggests that it would interfere with existing TCP GRO implementations. For UDP GRO the question is open whether or not the presence of a UDP options trailer (aka non-empty surplus area) of any kind (constant or not) would interfere with packet coalescing, but it's a possibility. Should either of these be true, we may be better off to omit the Parcel Parameters Option altogether. Tom Herbert raised the question of compatibility with checksum offload, and I shall deal with that in another message. Thanks and regards. Mike Heard On Fri, Nov 15, 2024 at 11:25 AM Templin (US), Fred L < fred.l.temp...@boeing.com> wrote: > Mike, your statement about not respecting datagram boundaries is incorrect > and under-informed – that is not what is happening. If you want documentary > evidence for GSO/GRO, it is found at www.kernel.org. > > > > *From:* C. M. Heard <he...@pobox.com> > *Sent:* Friday, November 15, 2024 10:48 AM > *To:* Templin (US), Fred L <fred.l.temp...@boeing.com> > *Cc:* Gorry Fairhurst <go...@erg.abdn.ac.uk>; Joe Touch < > to...@strayalpha.com>; Tom Herbert <t...@herbertland.com>; int-area < > int-area@ietf.org>; 6man <i...@ietf.org>; TSVWG <ts...@ietf.org> > *Subject:* [EXTERNAL] Re: [tsvwg] Re: UDP options [was IP Parcels and > Advanced Jumbos (AJs)] > > > > Fred, > > > > Please provide documentary evidence that GSO/GRO implementations send UDP > packets across the Internet without respecting the datagram boundaries set > by the original sender and without breaking the applications layered on top > of them. > > > > Note the word "UDP" in the above. > > > > Mike > > > > On Fri, Nov 15, 2024 at 10:36 AM Templin (US), Fred L < > fred.l.temp...@boeing.com> wrote: > > GSO/GRO for UDP are widely deployed and widely used in modern > Internetworking. There is no published RFC for them (yet), but popular > implementations invoke them all the time in actual practice. So, the > “reference” would be the source code of the popular implementations. > > > > Fred > > > > *From:* C. M. Heard <he...@pobox.com> > *Sent:* Friday, November 15, 2024 10:20 AM > *To:* Templin (US), Fred L <fred.l.temp...@boeing.com> > *Cc:* Gorry Fairhurst <go...@erg.abdn.ac.uk>; Joe Touch < > to...@strayalpha.com>; int-area <int-area@ietf.org>; 6man <i...@ietf.org>; > TSVWG <ts...@ietf.org> > *Subject:* Re: [tsvwg] Re: UDP options [was IP Parcels and Advanced > Jumbos (AJs)] > > > > Fred, > > > > Please provide a reference with evidence for that statement *as it > applies to UDP*. > > > > Mike > > > > > > On Fri, Nov 15, 2024 at 10:00 AM Templin (US), Fred L < > fred.l.temp...@boeing.com> wrote: > > Hi Mike, > > > > This is exactly what happens with Generic Segment Offload (GSO) and > Generic Receive Offload (GRO) in real networks today. After a multi-segment > GSO buffer is broken into individual IP packets during packetization, the > receiver applies GRO to restore the multi-segment buffer if possible; > otherwise, it delivers the individual IP packets to the upper layer. Each > individual IP packet is an atomic unit that will be understood by upper > layers even if it is not restored into the original multi-segment buffer > originally prepared by GSO. This is the way GSO/GRO work today, and IP > parcels does not change that. > > > > Fred > > > > *From:* C. M. Heard <he...@pobox.com> > *Sent:* Friday, November 15, 2024 9:36 AM > *To:* Templin (US), Fred L <fred.l.temp...@boeing.com> > *Cc:* Gorry Fairhurst <go...@erg.abdn.ac.uk>; Joe Touch < > to...@strayalpha.com>; int-area <int-area@ietf.org>; 6man <i...@ietf.org>; > TSVWG <ts...@ietf.org> > *Subject:* [EXTERNAL] Re: [tsvwg] Re: UDP options [was IP Parcels and > Advanced Jumbos (AJs)] > > > > EXT email: be mindful of links/attachments. > > > > > Fred, > > > > I very strongly disagree with your statement that it is not an error to > "it will instead deliver each of the individual IP packets to upper layers > without restoring the parce." > > > > That amounts to delivering PIECES of the UDP packet sent by the originator > to the upper layer. This would be exactly equivalent to allowing a receiver > that does not understand the UDP FRAG option to deliver the payload of each > individual fragment to the upper layer. The UDP options draft goes to > considerable effort to ensure that this does not happen. > > > > There is long-standing precedent that the boundaries of UDP datagrams are > preserved during transmission across the Internet. Many if not all > UDP-based protocols depend on that, explicitly or implicitly. I do not > understand why this point is even considered open for discussion. > > > > Mike > > > > On Fri, Nov 15, 2024 at 8:26 AM Templin (US), Fred L < > fred.l.temp...@boeing.com> wrote: > > Hi Mike, > > > > Thank you for looking and commenting. A UDP/IP parcel containing N > segments that undergoes packetization somewhere along the path will arrive > at the final destination as N individual IP packets, each containing a > Parcel Parameters UDP option. If the destination recognizes the option, it > will restore the N segment parcel within the kernel before delivering the > entire parcel as a single unit to upper layers. If the destination does not > recognize the option, it will instead deliver each of the individual IP > packets to upper layers without restoring the parcel which is not an error. > Very significantly, each of the individual IP packets can be dealt with as > standalone units once packetization has been applied; it is just that if > the receive-side does not apply restoration (i.e., GRO) the performance > optimization will be lost at that end. So, I don’t think there is any > problem to worry about. > > > > Fred Templin > > > > *From:* C. M. Heard <he...@pobox.com> > *Sent:* Thursday, November 14, 2024 4:55 PM > *To:* Templin (US), Fred L <fred.l.temp...@boeing.com> > *Cc:* Gorry Fairhurst <go...@erg.abdn.ac.uk>; Joe Touch < > to...@strayalpha.com>; int-area <int-area@ietf.org>; 6man <i...@ietf.org>; > TSVWG <ts...@ietf.org> > *Subject:* Re: [tsvwg] Re: UDP options [was IP Parcels and Advanced > Jumbos (AJs)] > > > > Fred and WG participants, > > > > I have finally freed up some cycles to read draft-templin-6man-parcels2-14 > <https://datatracker.ietf.org/doc/html/draft-templin-6man-parcels2-14>, > and I have found some issues with it that need to be addressed with respect > to its handling of UDP. > > > > The big one -- if I have correctly understood what I have read -- is that > it's possible for a single parcel of a parcellated UDP packet to be turned > into a stand-alone UDP packet (see Section 7.1 > <https://datatracker.ietf.org/doc/html/draft-templin-6man-parcels2#name-packetization-over-non-parc>) > and delivered to an end system as such (see Section 7.4 > <https://datatracker.ietf.org/doc/html/draft-templin-6man-parcels2#name-final-destination-restorati>). > That packet would contain a Parcel Parameters UDP Option to tell the > endpoint host that the packet is a parcel and not a complete UDP datagram, > but the option kind is taken from the SAFE option space (KIND = 127; see > draft-ietf-tsvwg-udp-options#section-10 > <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-udp-options#section-10>). > Legacy endpoints that do not understand UDP options will ignore that SAFE > option and will deliver the parcel as if it were a complete UDP datagram. > That, to my mind, is completely unacceptable. Unlike TCP, which is a > byte-stream protocol in which segment boundaries have no meaning for the > upper layer, UDP is a datagram protocol in which message boundaries are > meaningful to the upper layer. The protocol has a contract with the upper > layer to deliver a message as it was submitted or not at all. Delivering a > parcel in a manner that can be misinterpreted as a complete datagram > violates that contract. > > > > It is possible to repair this defect by making the Parcel Parameters > Option, or something with equivalent functionality, into an UNSAFE option. > My suggestion would be to define an UNSAFE version of the existing FRAG > option (see draft-ietf-tsvwg-udp-options#section-11.4 > <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-udp-options#section-11.4>) > -- let's call it UFRAG -- that would allow for packet sizes greater than > 65,535 bytes. The same option could be used to send singleton advanced > jumbo packets as atomic fragments. This would avoid any need to modify the > base UDP and UDP Options specifications. > > > > Additionally: during the review of draft-ietf-tsvwg-udp-options > <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-udp-options>, Joe > Touch correctly pointed out that RFC 2675 > <https://datatracker.ietf.org/doc/html/rfc2675> (and its predecessor RFC > 2147 <https://datatracker.ietf.org/doc/html/rfc2147>) failed to note that > it updated RFC 768 <https://datatracker.ietf.org/doc/html/rfc768>. > Similar concerns apply to TCP. If this draft foes forward, it should note > that it updates the UDP and TCP specifications, and it should get buy-in > from TSVWG and TCPM. > > > > Thanks and regards, > > > > Mike Heard > > > > > > > > > > > > On Sun, Sep 29, 2024 at 3:51 PM C. M. Heard <he...@pobox.com> wrote: > > Fred, > > > > I currently hold the editing pen for the changes to the UDP Options draft > that have been requested prior to the shepherd report, and my intention is > to remain silent about how, if at all, IP Parcels and Advanced Jumbos (AJs) > will support UDP Options. > > > > I'll provide comments on the IP Parcels and Advanced Jumbos work at a > later date, when I have spare intellectual cycles to fully comprehend the > contents of draft-templin-6man-parcels2. At this point I must confess that, > like Brian, I do not understand how a receiver will locate the options > trailer in the case of an IP Payload Length exceeding 65535. Like Joe, I > think it would be better to put the options just after the UDP header and > make a new UNSAFE option to delimit the position where the options end and > the user data begins.. But that discussion (and the corresponding update to > the UDP options draft) can occur when it is ripe; IMO that is not the case > at this time. > > > > Respectfully, > > > > Mike Heard > > > > On Sun, Sep 29, 2024 at 3:07 PM Templin (US), Fred L <Fred.L.Templin= > 40boeing....@dmarc.ietf.org> wrote: > > IP parcels and Advanced Jumbos (AJs) of all sizes ranging from 1 to 2^32 > are now eligible > > for using UDP options. This is just one way in which they offer a better > service than RFC2675 > > Jumbograms, but there are also many others. > > > > Joe, you can either note this in your draft or just leave it be and let my > draft do an > > “updates UDP options”. > > > > Thank you - Fred > > > > *From:* Brian Carpenter <brian.e.carpen...@gmail.com> > *Sent:* Saturday, September 28, 2024 2:20 AM > *To:* Gorry (erg) <go...@erg.abdn.ac.uk> > *Cc:* Joe Touch <to...@strayalpha.com>; Templin (US), Fred L < > fred.l.temp...@boeing.com>; Tim Chown <tim.ch...@jisc.ac.uk>; Internet > Area <Int-area@ietf.org>; IPv6 List <i...@ietf.org>; tsvwg IETF list < > ts...@ietf.org> > *Subject:* [EXTERNAL] Re: [tsvwg] Re: UDP options [was IP Parcels and > Advanced Jumbos (AJs)] > > > > EXT email: be mindful of links/attachments. > > > > > That works for me.. > > > > (via tiny screen & keyboard) > Regards, > Brian Carpenter > > > > On Sat, 28 Sept 2024, 19:08 Gorry (erg), <go...@erg.abdn.ac.uk> wrote: > > See below > > > On 28 Sep 2024, at 04:05, Brian E Carpenter <brian.e.carpen...@gmail.com> > wrote: > > > > Joe, > > On 28-Sep-24 03:13, to...@strayalpha.com wrote: > >>>> On Sep 27, 2024, at 7:58 AM, Templin (US), Fred L <Fred.L.Templin= > 40boeing....@dmarc.ietf.org> wrote: > >>> > >>>> Indeed. But if sendmsg() and recvmsg() can and do generate RFC2675 > packets, it means that any discussion of obsoleting RFC2675 should be > >>>> off the table. > >>> > >>> No one that I know of has suggested obsoleting RFC2675 - my documents > do not say "obsoletes" (nor even "updates”). > >> That approach to UDP jumbo grams is incompatible with UDP options. > >> And yes, there was a proposal to move that RFC to historic: > >> Jones, T., G. Fairhurst, "Change Status of RFC 2675 to Historic," > draft-jones-6man-historic-rfc2675, May 2019. > >> We COULD have a new option with a longer length, but that’s not in our > baseline draft. > > > > Wouldn't that be tricky, because the options follow the whole payload as > I understand it? So a JumboUDPgram has to be received in full, however big > it is, before the option saying that it's a jumbo can be received and > interpreted. > > > > Where the udp-options draft says: > > > >>> The technique has been proposed for deprecation [Jo19]. > > > > I think you'd better change it to something like: > > > > The technique is known to be in active use in special situations, so > cannot reasonably be deprecated. However, users of this technique cannot > simultaneously use UDP options. > > > > Brian > > > I do not think the I-D needs to say anything about the deployment status > of jumbograms, that another topic. > > I suggest if people wish, we just say that users of this technique can or > cannot simultaneously use UDP options. > > Gorry > > > > > >
_______________________________________________ Int-area mailing list -- int-area@ietf.org To unsubscribe send an email to int-area-le...@ietf.org
