Hi Florin, You can ignore the DPDK checksum flag as it won’t be passed to next graph nodes(I thought this is a bug and I’ve pushed a patch to fix it, see https://gerrit.fd.io/r/c/vpp/+/26480. With my patch the L4 checksum flags that come from DPDK rte_mbuf.flags can be passed to VPP vlib_buf.flags so they can be used in other graph nodes). In this case it was the software did the TCP checksum calculation. So I think this behavior(checking the checksum flag or doing software validate in ip4-local node) is expected. I misunderstand your first email, now I am clear with the 2nd question. Thanks for your clarification.
Best Regards, Sun, Chenmin From: vpp-dev@lists.fd.io <vpp-dev@lists.fd.io> On Behalf Of Florin Coras Sent: Tuesday, April 14, 2020 10:24 PM To: Sun, Chenmin <chenmin....@intel.com> Cc: vpp-dev <vpp-dev@lists.fd.io> Subject: Re: [vpp-dev] VPP checksum behavior problems Hi Chenmin, If packets are marked as having a bad csum, or marked as needing a csum and that turns out to be wrong after ip-local computes it, ip-local should drop them. From the trace lower, l4 csum is reported as bad by dpdk. So why is this unexpected? Regards, Florin On Apr 14, 2020, at 6:14 AM, Sun, Chenmin <chenmin....@intel.com<mailto:chenmin....@intel.com>> wrote: Hi Florin, But what I saw in current master branch is, the bad checksum packets are all dropped in ip4-local node. See log below: 00:06:42:511858: dpdk-input 1/2 rx queue 0 buffer 0xfe57e2: current data 0, length 124, buffer-pool 0, ref-count 1, totlen-nifb 0, trace handle 0x1000000 ext-hdr-valid PKT MBUF: port 1, nb_segs 1, pkt_len 124 buf_len 2176, data_len 124, ol_flags 0x8a, data_off 128, phys_addr 0xbf95f900 packet_type 0x191 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0 rss 0x5bf78172 fdir.hi 0x0 fdir.lo 0x5bf78172 Packet Offload Flags PKT_RX_RSS_HASH (0x0002) RX packet with RSS hash result PKT_RX_L4_CKSUM_BAD (0x0008) L4 cksum of RX pkt. is not OK PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid Packet Types RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet RTE_PTYPE_L3_IPV4_EXT_UNKNOWN (0x0090) IPv4 packet with or without extension headers RTE_PTYPE_L4_TCP (0x0100) TCP packet IP4: 00:00:01:00:24:00 -> 00:00:00:00:01:01 TCP: 66.66.66.6 -> 66.66.66.66 tos 0x00, ttl 64, length 110, checksum 0x71be dscp CS0 ecn NON_ECN fragment id 0x0000 TCP: 111 -> 222 seq. 0x00000201 ack 0x00000200 flags 0x00, tcp header: 20 bytes window 0, checksum 0xf6b5 00:06:42:512116: ethernet-input frame: flags 0x3, hw-if-index 2, sw-if-index 2 IP4: 00:00:01:00:24:00 -> 00:00:00:00:01:01 00:06:42:512143: ip4-input-no-checksum TCP: 66.66.66.6 -> 66.66.66.66 tos 0x00, ttl 64, length 110, checksum 0x71be dscp CS0 ecn NON_ECN fragment id 0x0000 TCP: 111 -> 222 seq. 0x00000201 ack 0x00000200 flags 0x00, tcp header: 20 bytes window 0, checksum 0xf6b5 00:06:42:512175: ip4-lookup fib 0 dpo-idx 8 flow hash: 0x00000000 TCP: 66.66.66.6 -> 66.66.66.66 tos 0x00, ttl 64, length 110, checksum 0x71be dscp CS0 ecn NON_ECN fragment id 0x0000 TCP: 111 -> 222 seq. 0x00000201 ack 0x00000200 flags 0x00, tcp header: 20 bytes window 0, checksum 0xf6b5 00:06:42:512217: ip4-local TCP: 66.66.66.6 -> 66.66.66.66 tos 0x00, ttl 64, length 110, checksum 0x71be dscp CS0 ecn NON_ECN fragment id 0x0000 TCP: 111 -> 222 seq. 0x00000201 ack 0x00000200 flags 0x00, tcp header: 20 bytes window 0, checksum 0xf6b5 00:06:42:512280: ip4-drop TCP: 66.66.66.6 -> 66.66.66.66 tos 0x00, ttl 64, length 110, checksum 0x71be dscp CS0 ecn NON_ECN fragment id 0x0000 TCP: 111 -> 222 seq. 0x00000201 ack 0x00000200 flags 0x00, tcp header: 20 bytes window 0, checksum 0xf6b5 00:06:42:512308: error-drop rx:1/2 00:06:42:512323: drop ip4-local: bad tcp checksum Best Regards, Sun, Chenmin From: Florin Coras <fcoras.li...@gmail.com<mailto:fcoras.li...@gmail.com>> Sent: Tuesday, April 14, 2020 12:04 AM To: Sun, Chenmin <chenmin....@intel.com<mailto:chenmin....@intel.com>> Cc: vpp-dev <vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>> Subject: Re: [vpp-dev] VPP checksum behavior problems Hi Chenmin, Regarding your second question, some of the device drivers could mark buffers for tx csum offload which is honored only if data is about to be delivered to the network (or if some other action performed to the buffer requires csums to be computed). Otherwise, if packets end up consumed locally, e.g., tunnel or host stack termination, the data is assumed to be correct and no checksum is computed. So, for instance, if udp/tcp packets received from a tap interface are terminated locally, we don’t try to compute/validate checksum, since there’s nothing to validate as the checksum has not yet been filled in. Regards, Florin On Apr 13, 2020, at 12:52 AM, Sun, Chenmin <chenmin....@intel.com<mailto:chenmin....@intel.com>> wrote: [Edited Message Follows] Hi experts, I am confusing about the L4 checksum behavior in VPP. I noticed the code was there for a very long time(more than 3 years) so I open this topic to see if anybody knows the history and why. * The first problem is in DPDK plugin and was introduced in commit d81566ff92: Disable for-us udp/tcp checksum validation by default in Current DPDK plugin, the code clears flags VNET_BUFFER_F_L4_CHECKSUM_COMPUTED and VNET_BUFFER_F_L4_CHECKSUM_CORRECT when "enable-tcp-udp-checksum" option is enabled. While these two flags are used to identify if the L4 checksum is checked and correct, respectively. So I think this is a wrong behavior - are we doing the opposite? The current DPDK plugin assumes all the packets from DPDK are correct but in fact we should check per packet and set the corresponding bits. I've pushed a patch to fix this issue and Dave invited experts to help to review - Thanks Dave :-) * Another problem in the IP4 graph node. The following mechanism was first introduced in commit 96be8e88109b3e1 and got refactored in commit 1b25552eb. #define ip4_local_csum_is_offloaded(_b) \ _b->flags & VNET_BUFFER_F_OFFLOAD_TCP_CKSUM \ || _b->flags & VNET_BUFFER_F_OFFLOAD_UDP_CKSUM #define ip4_local_need_csum_check(is_tcp_udp, _b) \ (is_tcp_udp && !(_b->flags & VNET_BUFFER_F_L4_CHECKSUM_COMPUTED \ || ip4_local_csum_is_offloaded (_b))) #define ip4_local_csum_is_valid(_b) \ (_b->flags & VNET_BUFFER_F_L4_CHECKSUM_CORRECT \ || (ip4_local_csum_is_offloaded (_b))) != 0 In my understanding, the Marcos ip4_local_need_csum_check and ip4_local_csum_is_valid are used to check if the L4 checksum is computed and checked before ip4-local node. While the Macro ip4_local_csum_is_offloaded is to check if the L4 checksum can be offloaded to NIC on the TX side. I am not very clear why do we need to check the tx offload capability in the RX path - maybe we are expecting the NIC can help to re-calculate the checksum before sending out? Think about if an APP(local APP running upon TCP/UDP stack or a tunnel like VxLAN/GTPU/etc...) is running on L4, it would get invalid checksum packets(Considering the DPDK plugin marked all the packets are COMPUTED and CORRECT as I described in the first problem). see in: commit 96be8e88109b3e166b76f58e552dbe438d73bb73 Author: Jakub Grajciar <jakub.grajc...@pantheon.tech<mailto:jakub.grajc...@pantheon.tech>> Date: Mon Oct 30 14:56:17 2017 +0100 vnet: ip4/6_local->don't drop packet if marked for TCP/UDP offload cksum calculation
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#16074): https://lists.fd.io/g/vpp-dev/message/16074 Mute This Topic: https://lists.fd.io/mt/72982275/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
