> Now a lot of my experience here comes from Linux many moons ago. > But I found a lot of hardware that wouldn't play nicely unless > offloading was disabled, with the most aggravating bugs. There's > also minor nuisances, like checksums not matching in tcpdump under > some circumstances. And some NICs would seem to "go bad" at some
Well, if you tcpdump packets outwards going which are going to get their checksums done by the hw, then tcpdump will correctly state that the checksum is bad, because it has not been done yet by the time it picks it up. > having offloading features disabled. I also think that there can > be some security issues where a packet might get by that the NIC > splits up (I think segmentation offloading can do this) and it gets > broken into another packet that would normally not be permitted. I think most of these offloadings are about having a single or few interrupts being done for a large(r) amounts of packets, especially those that belong to the same tcp stream and so on, so as far as I understand, the same checks for src ip,port and dst ip,port are still made, but regardless of if the offloading splits or combines ip packets, they all have to be valid in order to pass further into the tcp stack, its just a matter of being able to coalesce a stream of 10 small packets with 120 bytes payload each into one with 1200 instead, so that the tcp stack gets less work if some other part already did the work to combine them. Then the stack does the correct appropriate checks on that single packet, and if found bad, it will be dropped as usual. > That said, maybe some chipsets work great. And maybe some of the > features, especially say outbound, are quite reliable. I'm just a > skeptic after being bitten by it. Maybe some of it was Linux related? Could also have been "early days of offloading" which brought along bugs. Adaptec scsi raid cards also had weird bugs even though adaptec was a very known name for regular scsi chips. > I'm sure someone here has had a more positive experience with it, > or can attest that the performance gains are worthwhile, or that > it can be utilized only on hardware known to be reliable with it. This is why it gets in after a release, so we have a chance to test it on our platforms and hw. It seems like it was very selectively enabled and on chips that someone has run tests on, rather than "any network card and driver where the glossy broschure says it will work on". > Most of my experience involves gigabit hardware and performance > requirements, so I can see how at say 40gbit/sec, offloading is 40x > more valuable. Yeah, some part of the offloadings started at gig speeds, since when those first arrived, the CPU could not be expected to do all the work to fill a gig link like they could for 10 and 100Mbit/s, as cards now come in 40,50,100,400Gbit/s speeds, you are forced to have offloading or it would never get near linkspeed ever. This is a Mellanox ConnectX-6 Dx dual port 100GE card, listing its offloaders: Features for enp67s0f0np0: rx-checksumming: on tx-checksumming: on tx-checksum-ipv4: off [fixed] tx-checksum-ip-generic: on tx-checksum-ipv6: off [fixed] tx-checksum-fcoe-crc: off [fixed] tx-checksum-sctp: off [fixed] scatter-gather: on tx-scatter-gather: on tx-scatter-gather-fraglist: off [fixed] tcp-segmentation-offload: on tx-tcp-segmentation: on tx-tcp-ecn-segmentation: off [fixed] tx-tcp-mangleid-segmentation: off tx-tcp6-segmentation: on generic-segmentation-offload: on generic-receive-offload: on large-receive-offload: off rx-vlan-offload: on tx-vlan-offload: on ntuple-filters: off receive-hashing: on highdma: on [fixed] rx-vlan-filter: on vlan-challenged: off [fixed] tx-lockless: off [fixed] netns-local: off [fixed] tx-gso-robust: off [fixed] tx-fcoe-segmentation: off [fixed] tx-gre-segmentation: on tx-gre-csum-segmentation: on tx-ipxip4-segmentation: on tx-ipxip6-segmentation: on tx-udp_tnl-segmentation: on tx-udp_tnl-csum-segmentation: on tx-gso-partial: on tx-tunnel-remcsum-segmentation: off [fixed] tx-sctp-segmentation: off [fixed] tx-esp-segmentation: off [fixed] tx-udp-segmentation: on tx-gso-list: off [fixed] fcoe-mtu: off [fixed] tx-nocache-copy: off loopback: off [fixed] rx-fcs: off rx-all: on tx-vlan-stag-hw-insert: on rx-vlan-stag-hw-parse: off [fixed] rx-vlan-stag-filter: on [fixed] l2-fwd-offload: off [fixed] hw-tc-offload: on esp-hw-offload: off [fixed] esp-tx-csum-hw-offload: off [fixed] rx-udp_tunnel-port-offload: on tls-hw-tx-offload: off [fixed] tls-hw-rx-offload: off [fixed] rx-gro-hw: off [fixed] tls-hw-record: off [fixed] rx-gro-list: off macsec-hw-offload: on rx-udp-gro-forwarding: off hsr-tag-ins-offload: off [fixed] hsr-tag-rm-offload: off [fixed] hsr-fwd-offload: off [fixed] hsr-dup-offload: off [fixed] -- May the most significant bit of your life be positive.
