On 07/09/18 03:32, Eric Dumazet wrote: > Adding this complexity and icache pressure needs more experimental results. > What about RPC workloads (eg 100 concurrent netperf -t TCP_RR -- -r > 8000,8000 ) > > Thanks. Some more results. Note that the TCP_STREAM figures given in the cover letter were '-m 1450'; when I run that with '-m 8000' I hit line rate on my 10G NIC on both the old and new code. Also, these tests are still all with IRQs bound to a single core on the RX side. A further note: the Code Under Test is running on the netserver side (RX side for TCP_STREAM tests); the netperf side is running stock RHEL7u3 (kernel 3.10.0-514.el7.x86_64). This potentially matters more for the TCP_RR test as both sides have to receive data.
TCP_STREAM, 8000 bytes, GRO enabled (4 streams) old: 9.415 Gbit/s new: 9.417 Gbit/s (Welch p = 0.087, n₁ = n₂ = 3) There was however a noticeable reduction in *TX* CPU usage, of about 15%. I don't know why that should be (changes in ack timing, perhaps?) TCP_STREAM, 8000 bytes, GRO disabled (4 streams) old: 5.200 Gbit/s new: 5.839 Gbit/s (12.3% faster) (Welch p < 0.001, n₁ = n₂ = 6) TCP_RR, 8000 bytes, GRO enabled (100 streams) (FoM is one-way latency, 0.5 / tps) old: 855.833 us new: 862.033 us (0.7% slower) (Welch p = 0.040, n₁ = n₂ = 6) TCP_RR, 8000 bytes, GRO disabled (100 streams) old: 962.733 us new: 871.417 us (9.5% faster) (Welch p < 0.001, n₁ = n₂ = 6) Conclusion: with GRO on we pay a small but real RR penalty. With GRO off (thus also with traffic that can't be coalesced) we get a noticeable speed boost from being able to use netif_receive_skb_list_internal(). -Ed
