Bob, Thanks for the very detailed and insightful comments! It will take a while to got through all of it. I do want to comment first on the extensibility method of GUE since that is the most salient feature relative to other proposals.
> 2.8/ Extensibility of the flags and optional fields scheme: doesn't work > S3.3: > This is meant to be "the primary mechanism of extensibility in GUE". > However, for extensibility to work, GUE needs to distinguish between: > * options: the base set of flags+options defined from the start and required > in all GUE code > * "extensions" (my term): future extensions to the flags and options. > I think one theme in your review is that you do not believe GUE applies conventional wisdom or follows the text-book approach of defining open-ended arbitrary length TLVs or header chains to make an extensible protocol. The design decisions for GUE in this regard are not accidental. We don't see that open ended extensibility-- TLVs or header chains-- are a good match for the low level critical data path where GUE will be deployed. Complexity, history with existing mechanisms, DOS concerns, HW support, security, requirements of extensibility, and the ramifications and limitations of using UDP factor into this rationale. Below are the points that motivated the design of extensibility in GUE: 1) Efficiency is important. As has been pointed out already on this list a lot of processing is required for TLVs (or equivalently a header chain). Types need to be validated, lengths need to be verified, ordering is combinatoric, we need deal with duplicate TLVs, etc. This sort of processing is difficult to do efficiently in the critical data path. Additionally, there is on the wire overhead needed to describe each TLV. Flag-fields in GUE avoid these problems. 2) Extensibility must be limited. Consider that the HBH extension header can have a length up to 256*8 bytes so that an attacker can fill the space with 255 eight byte TLVs for some undefined type with high order bits 00 (ignore if unknown) or even just use PadN. This is one reason why so few devices in the Internet properly support HBH options-- they're an obvious opportunity for DOS. Our conclusion is that we need to limit extensibility in the data path. We're looking for some limit of N bytes such that 0 < N < MTU. In GUE we choose N=128 because a) 40 bytes of TCP options were too small b) nice power of 2 c) this is ~10% of a 1500 byte packet. There is a provision in GUE to increase this in the future if warranted. 3) Extensibility is not something we want, it's something we need. In a perfect world we would be able to define precisely the minimal protocol that would satisfy all our requirements for the next fifty years. But that is not reality, we need the ability to adapt protocols in unforeseen ways to changing environment and security threats. Our conclusion is that use of extensibility should be constrained. It should not be a vehicle for vendors to introduce a spew of proprietary non-interoperable features nor a mechanism to bandaid over the lack of deployability of other protocol (e.g. IP options). Hence we don't need the ability to define a million different extensions just a handful will be needed. But it's absolutely critical that when we do go to deploy a new protocol extensions that we are able to do it-- the worse case scenario is that at some point we try to change a protocol in the data center only to find that we can't because some component in the path can't deal with change. 4) The introduction of new extensions will be a rare event. We would expect at most less than one new extensions introduced each year. This rate is extrapolated from the rate at which we see TCP options being added. Given this low rate, there are long periods when the data center will be in a steady state-- encapsulated packets always contain the exact same set of options. We want to optimize for this state by minimizing overhead and and processing complexity, as opposed to optimizing for the case of adding extensions. When we introduce a new extension at scale, regardless of what format it takes, it is always be a lot of work in a short-term. 5) Peer capabilities can be discovered or negotiated. The alternative to self describing objects is to describe capabilities of a peer upfront. Capabilities would include the things like supported extensions, but even higher layer items like protocol version or even supported encapsulation protocols (we cannot guarantee that one protocol will work in all cases). Sharing capabilities occurs between two peer endpoints can be done through configuration, negotiation, or a control plane. The latter is particularly interesting in nvo3 because there must be a control plane that propagates mapping information, adding a list of receive capabilities should not be difficult. 6) History does not bode well for open extended extensibility mechanisms. The extensibility mechanisms of IP, IPv4 options and IPv6 extension headers, are not widely deployed either in the Internet nor in datacenters. To be blunt, extensibility mechanisms of IP have been a disappointment. It is easy to blame middlebox hardware vendors for failing to support these, but we do see that TCP options have been successful. The major differences are are the TCP options are required for protocol operation (don't suffer from "use or lose it" syndrome) and the options used for a connection are negotiated. GRE (on which GUE is based) is an example of low level protocol with VLH that has successfully been deployed for a long time. 7) Middleboxes cannot correctly parse UDP payloads. UDP is an end to end protocol. Per RFC7605 UDP port numbers do not have specific meaning the network. This means that any intermediate device that parses a UDP payload based on port numbers may be wrong. Furthermore, if the device were to modify the payload that is misinterpreted this introduces the possibility of silent data corruption. At large scale we cannot guarantee exclusive use of an port number for an application, hence it is inevitable that middleboxes mucking with UDP payloads is going to be a problem for us. Given this and #8 below, the rules for dealing with GUE flags are simple: 1) All GUE fields are immutable in the network. This can be enforced by an integrity check over the header. 2) We allow the intermediate nodes will parse the GUE packets. This is an acknowledgment that intermediate devices will parse anything in the packet that is in plain text even if we tell them not to. 3) Intermediate devices MUST NOT drop GUE packets because they contain unknown flags. 4) Receivers MUST drop any packets with unknown flags. 8) A receiver ignoring unknown fields in a protocol it processes is a bad idea. Accepting unknown flags or fields at a receiver is a security risk. They could be a DOS attack, a means for an attacker to covertly send arbitrary data through the network, or a means of probing the network for security vulnerabilities. Also, unknown flags in GUE are "MUST drop" because processing these flags is necessary to correctly process the packet (note that none of the fundamental GUE extensions cannot simply be ignored when present). It might be argued that "MUST drop" violates the liberal receive policy of the robustness principle. I think the opposite it true. The robustness principle does not mean that we need to accept protocol that is incorrect, and if we ignore bits in protocol header then we can't determine if the protocol is correct. Consider that someone is sending a TLV with an illegal length. If a receiver does not understand the type then it may accept the packet, however another receiver implements the type might drop the packet because the TLV length is illegal. This examples illustrates ambiguity that is introduced as to what a correct packet is when protocol fields are allowed to be ignored. Our conclusion is that we don't want hosts wasting resources sending bits that have no effect, and we don't want receivers taking a security risk by accepting things they don't understand. 9) The use of UDP for encapsulation protocols increases risk. UDP is a transport protocol in the sense applications are free to open UDP sockets and send data over UDP in whatever form they choose. There are very few restrictions build into OSes to restrict the use of UDP, in particular any unprivileged application can send UDP packets to any port and destination address of their choosing. So it is trivial to forge packets for UDP encapsulations (compared to a non-UDP encapsulation like GRE). Our conclusion is that security is paramount. This is why both header security as well as DTLS are included in the fundamental extensions for GUE. Thanks, Tom _______________________________________________ nvo3 mailing list [email protected] https://www.ietf.org/mailman/listinfo/nvo3
