On 12/20/2021 10:03 AM, Templin (US), Fred L wrote:
Tom, sorry I will try to use my words more carefully; I am using GSO/GRO also for a UDP-based transport protocol – not QUIC but something similar. I like GSO/GRO very much; I am glad the service is available and I want to see it continue. But, my understanding of the services is that they leverage the IP ID field in whole IPv4 packets that are not eligible for fragmentation and those are limitations I am seeking to improve on.
Lots of QUIC implementation use UDP GSO/GRO. They do that to achieve good performance. Without GSO, we have observed that almost 2/3rd of the CPU time was spent in socket calls, mostly sendmsg on servers. Using GSO to send batches of packets results in big gains. For example, the same single-core implementation that achieves 500 Mbps without GSO gets to 4-5 Gbps while sending batches of packets of about 64K using GSO.
Note however that this is *UDP* GSO. The application prepares a batch of UDP packets, all sharing the same source and destination address and ports, and all but the last having the same length. The GSO call provides these header parameters, the common payload length, and a set of UDP payloads, concatenated in a large buffer. Each payload is a fully formed QUIC packet, encrypted and sealed using AEAD. There is no expectation that these packets would be somehow re-segmented or re-assembled in the network -- if the network did that, decryption would fail. There is also no expectation that the batch of packets will be delivered as a single batch to the receiver. It might happen by chance, but mostly it does not because GRO and GSO are not synchronized. On the other hand, the handling of the small packets by the QUIC stack is rarely considered a problem.
I want to enable a facility similar to GSO/GRO that works for both IPv4 and IPv6 packets and allows for lower layers to fragment if necessary. And, I want to use a well-behaved 32-bit IPv6 ID instead of the 16-bit IPv4 one where the use is not well defined when DF=1.
Consider encryption. QUIC certainly expects that packets sent will be received unchanged, or they would fail decryption. Also consider packet loss. In the GSO example, the sender might send a batch of 42 packets. If any of them is lost, it will be resent individually. QUIC could of course send a 64KB packet if the network allowed. But if a 64KB parcel is segmented in 42 segments, any packet loss will cause the loss of the whole parcel, and that would cause a significant drop in performance compared to GSO. Which is indeed the core of the "segmentation considered harmful" argument.
-- Christian Huitema _______________________________________________ Int-area mailing list [email protected] https://www.ietf.org/mailman/listinfo/int-area
