I believe there is a bug with TCP traffic encrypted in Transport Mode over NAT (ie. UDP-encap ESP). I sent a message to the netfilter-devel list and someone suggested that I post to netdev.
Here are my messages. Thanks. -------- Original Message -------- Subject: Understanding Netfilter and IPSec Transport Mode over NAT Date: Tue, 09 Feb 2006 13:44:39 -0500 From: Chinh Nguyen <[EMAIL PROTECTED]> To: [EMAIL PROTECTED] Chinh Nguyen wrote: >> Hi, >> The first question is more academic. How does a per-socket bypass policy >> equals >> "accept transport mode ESP"? >> >> The second question is more pragmatic. Why doesn't this work for TCP? With >> encrypted TCP traffic, the skb->sp is NULL, therefore esp_post_input is not >> called. Why? However, the decrypted TCP packet itself seems to be sent up the >> stack since netstat reveals that bad TCP segments are being received. I discovered that the "bug" is in the function tcp_v4_rcv for kernel 2.6.16-rc1. After the ESP packet is decapped and decrypted in xfrm4_rcv_encap_finish, the unencrypted packet is pushed back through ip_local_deliver. For a UDP packet, it goes (back) to function udp_queue_rcv_skb. The first thing this function does is called xfrm4_policy_check. As noted previously, in xfrm4_policy_check, if the skb->sp != NULL, the esp_post_input function is called. The post input function sets skb->ip_summed to CHECKSUM_UNNECESSASRY if we are in transport mode. Therefore, further down in udp_queue_rcv_skb, we skip the checksum check and the packet is passed up the stack. However, for a decrypted TCP packet, the packet goes to tcp_v4_rcv. This function does the checksum check right away if skb->ip_summed != CHECKSUM_UNNECESSARY while xfrm4_policy_check is called a little later in the function. Therefore, the esp post input has not yet set the ip_summed to unnecessary. The decrypted packet fails the checksum and is discarded. To confirm this, I added another call to xfrm4_policy_check before the checksum check in tcp_v4_rcv (to call esp post input). Once patched, my systems were able to initiate TCP connections using Transport Mode/NAT. printk(KERN_INFO "tcp_v4_rcv: tcp patch\n"); if (!xfrm4_policy_check(NULL, XFRM_POLICY_IN, skb)) goto discard_it; nf_reset(skb); printk(KERN_INFO "tcp_v4_rcv: skb = %08x, skb->ip_summed = %d\n", skb, skb->ip_summed); /* An explanation is required here, I think. * Packet length and doff are validated by header prediction, * provided case of th->doff==0 is eliminated. * So, we defer the checks. */ if ((skb->ip_summed != CHECKSUM_UNNECESSARY && tcp_v4_checksum_init(skb))) goto bad_packet; printk(KERN_INFO "tcp_v4_rcv: sum ok\n"); I don't consider this a real patch because it's inefficient to have 2 calls to xfrm4_policy_check in tcp_v4_rcv. Is this the right list to bring up this issue or should I go to some other list (e.g., linux-kernel, etc)? Regards, Chinh -------- Original Message -------- Subject: Understanding Netfilter and IPSec Transport Mode over NAT Date: Tue, 07 Feb 2006 14:40:39 -0500 From: Chinh Nguyen <[EMAIL PROTECTED]> To: [EMAIL PROTECTED] Hi, I realize that this is a devel mailing-list. My apologies. I am having problem with Racoon and Transport Mode over NAT (for TCP traffic) for kernel 2.6.16-rc1. I'm trying to understand how Netfilter works and I'm stumped. The standard Racoon which pushes an in/out bypass per-socket policy accepts encrypted UDP traffic. After decap, the associated skb has a security path. As such, the post_input function (ie, esp4_post_input) sets the skb->ipsummed to ignore checksum. The result is that UDP traffic is accepted. Why doesn't this work for TCP? With encrypted TCP traffic, the skb->sp is NULL, therefore esp_post_input is not called. Why? However, the decrypted TCP packet itself seems to be sent up the stack since netstat reveals that bad TCP segments are being received. Chinh - To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html