I have a configuration on my box running OpenBSD 5.6 with errata that is supposed to tunnel IPv4 over IPv6. However, due to bugged RFC 2473 implementations shared across ALL BSD systems, pf's af-to comes into play when I decide to navigate through the abhorrent path MTU discovery issue found in gif. As a result, please do not blame me for reinventing the wheel, since RFC 2473 tunnels implemented in gif ALWAYS outputs at the MTU of 1280 REGARDLESS of the mtu option in ifconfig/hostname.if.
The configuration requires two af-to lines in pf.conf, one from inet6 to inet4, so as to listen for incoming protocol 4 from IPv6 and then translate the packet to IPv4, and the other from inet4 to inet6, which listens for inbound protocol 4 IPv4 packets and translate the packet to IPv6. The REAL problem is on the latter part. Before the pf.conf lines, let me outline my configuration here. I have global IPv6 access through gif0, which is an RFC 4213 tunnel. And a copy of hostname.gif0 is as follows: ----------BEGIN hostname.gif0---------- tunnel 146.185.21.99 216.66.88.98 inet6 2001:470:1f1c:10b::2 64 mtu 1480 inet 192.168.0.98 255.255.255.255 NONE group egress !/sbin/route -n -q add -mpath -inet6 default -host -inet6 2001:470:1f1c:10b::1 ----------END hostname.gif0---------- My local end of the quasi-RFC2473 tunnel sits on vether0, with associating hostname.vether0 as follows: ----------BEGIN hostname.vether0---------- inet 192.168.0.97 255.255.255.255 NONE up ----------END hostname.vether0---------- And now we are onto the quasi-RFC2472 tunnel, with associating hostname.gif2 as follows: ----------BEGIN hostname.gif2---------- tunnel 192.168.0.97 192.168.0.98 inet 172.31.255.253 255.255.255.252 NONE mtu 1280 up !/sbin/route -n -q add -host -inet 172.31.255.254 -host -inet 172.31.255.254 ----------END hostname.gif2---------- And the two lines below in my pf.conf is supposed to hook everything up: ----------BEGIN pf.conf---------- pass in quick inet proto 4 from 192.168.0.97 to 192.168.0.98 \ af-to inet6 from 2001:470:1f1c:10b::2 to 2001:470:c:6fa::2 pass in quick inet6 proto 4 from 2001:470:c:6fa::2 to 2001:470:1f1c:10b::2 \ af-to inet from 192.168.0.98 to 192.168.0.97 ----------END pf.conf---------- And for your cynicists, there's the sentence in 'man pf.conf', "The optional second argument is the host or subnet the original addresses are translated into for the destination." This is also supported by tcpdump results. The OpenBSD box peers with a Linux box, whose address family translation in this case is performed by socat with the following lines: ----------BEGIN socat-46.sh---------- /usr/local/bin/socat \ IP4-RECVFROM:4,bind=192.168.0.97,fork \ IP6-SENDTO:[2001:470:1f1c:10b::2]:4,bind=[2001:470:c:6fa::2],reuseaddr ----------END socat-46.sh---------- ----------BEGIN socat-64.sh---------- /usr/local/bin/socat \ IP6-RECVFROM:4,bind=[2001:470:c:6fa::2],fork \ IP4-SENDTO:192.168.0.98:4,bind=192.168.0.97,reuseaddr ----------END socat-46.sh---------- If you wanna have a further look for the setup on the Linux box, the equivalent Vyatta setup lines is as follows ----------BEGIN edit interface tunnel tun4---------- set address 172.31.255.254/30 set disable-link-detect set encapsulation ipip set local-ip 192.168.0.98 set mtu 1280 set multicast disable set remote-ip 192.168.0.97 ----------END edit interface tunnel tun4---------- ----------BEGIN edit interface ethernet eth1---------- set vif 0 address 192.168.0.97/32 set vif 1 address 192.168.0.98/32 ----------ENDedit interface ethernet eth1---------- Again, the IPv6 access on the Linux box is provided by an RFC 4213 tunnel, whose Vyatta-equivalent setup lines is as follows: ----------BEGIN edit interface tunnel tun2---------- set address 2001:470:c:6fa::2/64 set disable-link-detect set encapsulation sit set local-ip 0.0.0.0 set mtu 1480 set multicast disable set remote-ip 66.220.18.42 set ttl 255 ----------END edit interface tunnel tun2---------- Now, we run on the Linux box "ping -I 172.31.255.254 172.31.255.253", and on the OpenBSD box "tcpdump -i gif2". The following is a sample output on the OpenBSD box: ----------BEGIN tcpdump -i gif2---------- # tcpdump -i gif2 tcpdump: listening on gif2, link-type LOOP 02:50:02.673576 172.31.255.254 > 172.31.255.253: icmp: echo request (DF) 02:50:02.673978 172.31.255.253 > 172.31.255.254: icmp: echo reply (DF) 02:50:03.664722 172.31.255.254 > 172.31.255.253: icmp: echo request (DF) 02:50:03.665036 172.31.255.253 > 172.31.255.254: icmp: echo reply (DF) 02:50:04.664823 172.31.255.254 > 172.31.255.253: icmp: echo request (DF) 02:50:04.665148 172.31.255.253 > 172.31.255.254: icmp: echo reply (DF) 02:50:05.664215 172.31.255.254 > 172.31.255.253: icmp: echo request (DF) 02:50:05.664539 172.31.255.253 > 172.31.255.254: icmp: echo reply (DF) 02:50:06.664375 172.31.255.254 > 172.31.255.253: icmp: echo request (DF) 02:50:06.664721 172.31.255.253 > 172.31.255.254: icmp: echo reply (DF) 02:50:07.664206 172.31.255.254 > 172.31.255.253: icmp: echo request (DF) 02:50:07.664546 172.31.255.253 > 172.31.255.254: icmp: echo reply (DF) ^C 12 packets received by filter 0 packets dropped by kernel ----------END tcpdump -i gif2---------- No ICMP replies are ever received on the Linux box. And "tcpdump -i gif0 'proto 4'" on the OpenBSD box can confirm this: ----------BEGIN tcpdump -i gif0 'proto 4'---------- # tcpdump -i gif0 'proto 4' tcpdump: listening on gif0, link-type LOOP 02:52:53.663779 172.31.255.254 > 172.31.255.253: icmp: echo request (DF) (encap) 02:52:54.663889 172.31.255.254 > 172.31.255.253: icmp: echo request (DF) (encap) 02:52:55.663693 172.31.255.254 > 172.31.255.253: icmp: echo request (DF) (encap) 02:52:56.663644 172.31.255.254 > 172.31.255.253: icmp: echo request (DF) (encap) 02:52:57.663761 172.31.255.254 > 172.31.255.253: icmp: echo request (DF) (encap) 02:52:58.663805 172.31.255.254 > 172.31.255.253: icmp: echo request (DF) (encap) 02:52:59.664058 172.31.255.254 > 172.31.255.253: icmp: echo request (DF) (encap) 02:53:00.663951 172.31.255.254 > 172.31.255.253: icmp: echo request (DF) (encap) 02:53:01.664333 172.31.255.254 > 172.31.255.253: icmp: echo request (DF) (encap) 02:53:02.663995 172.31.255.254 > 172.31.255.253: icmp: echo request (DF) (encap) 02:53:03.663991 172.31.255.254 > 172.31.255.253: icmp: echo request (DF) (encap) 02:53:04.663993 172.31.255.254 > 172.31.255.253: icmp: echo request (DF) (encap) ^C 41 packets received by filter 0 packets dropped by kernel ----------END tcpdump -i gif0 'proto 4'---------- Allow to me sum up what happened in the tcpdump results here. The Linux box generate an ICMP echo request from 172.31.255.254 to 172.31.255.253, wraps it in an IP-in-IP packet from 192.168.0.98 to 192.168.0.97, and sends it to 192.168.0.97. socat takes the protocol 4 IPv4 packet at 192.168.0.98, decapsulates the packet, and re-encapsulates it as an protocol 4 IPv6 packet from 2001:470:c:6fa::2 to 2001:470:1f1c:10b::2. And then the OpenBSD box gets the packet and address-family-translates it to an protocol 4 IPv4 packet from 192.168.0.98 to 192.168.0.97, which is received on vether0. The packet is then decapsulated and reveals the original ICMP echo request. And then gif2 generates a reply, and all ends up in a blackhole. Note that for the ICMP replies to generate, the line "!/sbin/route -n -q add -host -inet 172.31.255.254 -host -inet 172.31.255.254" in hostname.gif2 is mandatory. And I have several questions here: 1) how do I get the ICMP echo reply properly encapsulated? 2) if ever the ICMP echo reply is properly encapsulated, why did pf fail to translate the protocol 4 IPv4 packet back to the equivalent IPv6 packet? 3) is this a pf or gif bug, or else? 4) how am I supposed to get the setup to work here? 5) are there any suggestion on tunneling IPv4 over IPv6 that circumvents the dreaded fixed-MTU-at-1280 issue, first between two OpenBSD hosts, and a second between an OpenBSD and a Linux host? Thank you. Best, Jing Zhou
