On Sat, Mar 17, 2018 at 05:21:53PM -0700, Max Parmer wrote: > I've been having a good time running some VMD guests on 6.2 and assigning them > external IPs which are binat'd to them by the VM host. Recently I learned my > hosting provider delegates a /64 to it's dedicated boxes and thought this > might > be an interesting scenario to improve, and possibly simplify, by routing IPv6 > directly to my guests. > > To start, I ensured that IPv6 was properly functional on the host: > > # cat /etc/hostname.em0 > > inet6 autoconf > > inet6 2607:53xx:6x:7a3b:: 64 eui64 > > inet xxx.xxx.211.59 255.255.255.0 > > inet alias xxx.xxx.219.108 255.255.255.0 > > inet alias xxx.xx.248.240 255.255.255.0 > > inet alias xxx.xx.248.241 255.255.255.0 > > inet alias xxx.xx.248.242 255.255.255.0 > > inet alias xxx.xx.248.243 255.255.255.0 > > # cat /etc/mygate > > xxx.xxx.211.254 > > fe80::205:73ff:fea0:1%em0 > > # ifconfig em0 > > em0: flags=208843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,AUTOCONF6> mtu 1500 > > lladdr 0c:c4:7a:45:37:34 > > index 1 priority 0 llprio 3 > > groups: egress > > media: Ethernet autoselect (1000baseT full-duplex,master,rxpause) > > status: active > > inet6 fe80::ec4:7aff:fe45:3734%em0 prefixlen 64 scopeid 0x1 > > inet6 2607:53xx:6x:7a3b:ec4:7aff:fe45:3734 prefixlen 64 > > inet xxx.xxx.211.59 netmask 0xffffff00 broadcast xxx.xxx.211.255 > > inet xxx.xxx.219.108 netmask 0xffffff00 broadcast xxx.xxx.219.255 > > inet xxx.xx.248.240 netmask 0xffffff00 broadcast xxx.xx.248.255 > > inet xxx.xx.248.241 netmask 0xffffff00 broadcast xxx.xx.248.255 > > inet xxx.xx.248.242 netmask 0xffffff00 broadcast xxx.xx.248.255 > > inet xxx.xx.248.243 netmask 0xffffff00 broadcast xxx.xx.248.255 > > # ifconfig vether0 > > vether0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500 > > lladdr 00:00:d3:00:d0:0d > > index 5 priority 0 llprio 3 > > groups: vether > > media: Ethernet autoselect > > status: active > > inet6 fe80::200:d3ff:fe00:d00d%vether0 prefixlen 64 scopeid 0x5 > > inet 10.0.23.1 netmask 0xffffff00 broadcast 10.0.23.255 > > # ifconfig bridge0 > > bridge0: flags=41<UP,RUNNING> > > description: switch1-local > > index 6 llprio 3 > > groups: bridge > > priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp > > designated: id 00:00:00:00:00:00 priority 0 > > vether0 flags=3<LEARNING,DISCOVER> > > port 5 ifpriority 0 ifcost 0 > > tap0 flags=3<LEARNING,DISCOVER> > > port 8 ifpriority 0 ifcost 0 > > Addresses (max cache: 100, timeout: 240): > > 00:00:d3:00:12:00 tap0 1 flags=0<> > > # slaacctl show interface em0 > > em0: > > index: 1 running: yes privacy: yes > > lladdr: 0c:c4:7a:45:37:34 > > inet6: fe80::ec4:7aff:fe45:3734%em0 > > Router Advertisement from fe80::205:73ff:fea0:1%em0 > > received: 2018-03-17 20:01:39; 143s ago > > Cur Hop Limit: 64, M: 0, O: 0, Router Lifetime: 1800s > > Default Router Preference: Medium > > Reachable Time: 0ms, Retrans Timer: 0ms > > prefix: 2607:53xx:6x:7aff:ff:ff:ff:ff/56 > > On-link: 1, Autonomous address-configuration: 1 > > vltime: 2592000, pltime: 604800 > > Default router proposals > > id: 1, state: CONFIGURED > > router: fe80::205:73ff:fea0:1%em0 > > router lifetime: 1800 > > Preference: Medium > > updated: 2018-03-17 20:01:39; 143s ago, timeout: 1642s > > # route -nv show -inet6 > > Routing tables > > > > Internet6: > > Destination Gateway Flags > > Refs Use Mtu Prio Iface Label > > default fe80::205:73ff:fea0:1%em0 UGS > > 0 4 - 56 em0 "slaacd" > > ::/96 ::1 UGRS > > 0 0 32768 8 lo0 > > ::/104 ::1 UGRS > > 0 0 32768 8 lo0 > > ::1 ::1 UHhl > > 14 28 32768 1 lo0 > > ::127.0.0.0/104 ::1 UGRS > > 0 0 32768 8 lo0 > > ::224.0.0.0/100 ::1 UGRS > > 0 0 32768 8 lo0 > > ::255.0.0.0/104 ::1 UGRS > > 0 0 32768 8 lo0 > > ::ffff:0.0.0.0/96 ::1 UGRS > > 0 0 32768 8 lo0 > > 2002::/24 ::1 UGRS > > 0 0 32768 8 lo0 > > 2002:7f00::/24 ::1 UGRS > > 0 0 32768 8 lo0 > > 2002:e000::/20 ::1 UGRS > > 0 0 32768 8 lo0 > > 2002:ff00::/24 ::1 UGRS > > 0 0 32768 8 lo0 > > 2607:53xx:6x:7a3b::/64 2607:53xx:6x:7a3b:ec4:7aff:fe45:3734 UCn > > 0 0 - 4 em0 > > 2607:53xx:6x:7a3b:ec4:7aff:fe45:3734 0c:c4:7a:45:37:34 UHLl > > 0 173 - 1 em0 > > fe80::/10 ::1 UGRS > > 0 2 32768 8 lo0 > > fec0::/10 ::1 UGRS > > 0 0 32768 8 lo0 > > fe80::%em0/64 fe80::ec4:7aff:fe45:3734%em0 UCn > > 3 5 - 4 em0 > > fe80::205:73ff:fea0:1%em0 00:05:73:a0:00:01 UHLch > > 1 95 - 3 em0 > > fe80::2ff:ffff:feff:fffd%em0 00:ff:ff:ff:ff:fd UHLc > > 0 1155 - 3 em0 > > fe80::2ff:ffff:feff:fffe%em0 00:ff:ff:ff:ff:fe UHLc > > 0 945 - 3 em0 > > fe80::ec4:7aff:fe45:3734%em0 0c:c4:7a:45:37:34 UHLl > > 0 373 - 1 em0 > > fe80::1%lo0 fe80::1%lo0 UHl > > 0 0 32768 1 lo0 > > fe80::%vether0/64 fe80::200:d3ff:fe00:d00d%vether0 UCn > > 1 1 - 4 vether0 > > fe80::200:d3ff:fe00:1200%vether0 00:00:d3:00:12:00 UHLc > > 1 11 - 3 vether0 > > fe80::200:d3ff:fe00:d00d%vether0 00:00:d3:00:d0:0d UHLl > > 0 38 - 1 vether0 > > ff01::/16 ::1 UGRS > > 0 2 32768 8 lo0 > > ff01::%em0/32 fe80::ec4:7aff:fe45:3734%em0 Um > > 0 1 - 4 em0 > > ff01::%lo0/32 ::1 Um > > 0 1 32768 4 lo0 > > ff01::%vether0/32 fe80::200:d3ff:fe00:d00d%vether0 Um > > 0 0 - 4 vether0 > > ff02::/16 ::1 UGRS > > 0 2 32768 8 lo0 > > ff02::%em0/32 fe80::ec4:7aff:fe45:3734%em0 Um > > 0 1 - 4 em0 > > ff02::%lo0/32 ::1 Um > > 0 1 32768 4 lo0 > > ff02::%vether0/32 fe80::200:d3ff:fe00:d00d%vether0 Um > > 0 0 - 4 vether0 > > # ping6 -c3 google.com > > PING google.com (2607:f8b0:4004:809::200e): 56 data bytes > > 64 bytes from 2607:f8b0:4004:809::200e: icmp_seq=0 hlim=56 time=13.756 ms > > 64 bytes from 2607:f8b0:4004:809::200e: icmp_seq=1 hlim=56 time=13.748 ms > > 64 bytes from 2607:f8b0:4004:809::200e: icmp_seq=2 hlim=56 time=13.744 ms > > > > --- google.com ping statistics --- > > 3 packets transmitted, 3 packets received, 0.0% packet loss > > round-trip min/avg/max/std-dev = 13.744/13.749/13.756/0.005 ms > > # ftp -6 -Mo - https://wtfismyip.com/text > > Trying 2607:53xx:6x:7f8a::... > > Requesting https://wtfismyip.com/text > > 2607:53xx:6x:7a3b:ec4:7aff:fe45:3734 > > 37 bytes received in 0.00 seconds (145.11 KB/s) > > Next up, IPv6 forwarding on the host and interfaces for the guests: > > # cat /etc/sysctl.conf > > net.inet.ip.forwarding=1 > > net.inet6.ip6.forwarding=1 > > # cat /etc/hostname.vether0 > > lladdr 00:00:D3:00:D0:0D > > inet6 eui64 > > inet 10.0.23.1 255.255.255.0 NONE > > # cat /etc/hostname.bridge0 > > add vether0 > > up > > # egrep 'rtadvd|vmd' /etc/rc.conf.local > > rtadvd_flags=vether0 > > vmd_flags= > > # stat /etc/rtadvd.conf > > stat: /etc/rtadvd.conf: No such file or directory > > # cat /etc/vm.conf > > switch "local" { > > interface bridge0 > > add vether0 > > } > > vm "chat" { > > disable > > memory 1G > > owner maxp > > disk "/home/images/chat.img" > > interface tap { > > locked lladdr "00:00:D3:00:12:00" > > switch "local" > > } > > } > > # pfctl -d > > pfctl: pf not enabled > > I've got PF disabled for the moment, just to keep things simple. > > Then, within the VM... > > # cat /etc/hostname.vio0 > > dhcp > > inet6 autoconf > > inet6 alias 2607:53xx:6x:7a3b:: 64 eui64 > > # slaacctl show interface vio0 > > vio0: > > index: 1 running: yes privacy: yes > > lladdr: 00:00:d3:00:12:00 > > inet6: fe80::200:d3ff:fe00:1200%vio0 > > Router Advertisement from fe80::200:d3ff:fe00:d00d%vio0 > > received: 2018-03-17 22:05:48; 83s ago > > Cur Hop Limit: 64, M: 0, O: 0, Router Lifetime: 1800s > > Default Router Preference: Medium > > Reachable Time: 0ms, Retrans Timer: 0ms > > Default router proposals > > id: 1, state: CONFIGURED > > router: fe80::200:d3ff:fe00:d00d%vio0 > > router lifetime: 1800 > > Preference: Medium > > updated: 2018-03-17 22:05:48; 83s ago, timeout: 1702s > > # ifconfig vio0 > > vio0: > > flags=208b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST,AUTOCONF6> > > mtu 1500 > > lladdr 00:00:d3:00:12:00 > > index 1 priority 0 llprio 3 > > groups: egress > > media: Ethernet autoselect > > status: active > > inet 10.0.23.30 netmask 0xffffff00 broadcast 10.0.23.255 > > inet6 fe80::200:d3ff:fe00:1200%vio0 prefixlen 64 scopeid 0x1 > > inet6 2607:53xx:6x:7a3b:200:d3ff:fe00:1200 prefixlen 64 > > # route -nv show -inet6 > > Routing tables > > > > Internet6: > > Destination Gateway Flags > > Refs Use Mtu Prio Iface Label > > default fe80::200:d3ff:fe00:d00d%vio0 UGS > > 0 0 - 56 vio0 "slaacd" > > ::/96 ::1 UGRS > > 0 0 32768 8 lo0 > > ::/104 ::1 UGRS > > 0 0 32768 8 lo0 > > ::1 ::1 UHhl > > 14 28 32768 1 lo0 > > ::127.0.0.0/104 ::1 UGRS > > 0 0 32768 8 lo0 > > ::224.0.0.0/100 ::1 UGRS > > 0 0 32768 8 lo0 > > ::255.0.0.0/104 ::1 UGRS > > 0 0 32768 8 lo0 > > ::ffff:0.0.0.0/96 ::1 UGRS > > 0 0 32768 8 lo0 > > 2002::/24 ::1 UGRS > > 0 0 32768 8 lo0 > > 2002:7f00::/24 ::1 UGRS > > 0 0 32768 8 lo0 > > 2002:e000::/20 ::1 UGRS > > 0 0 32768 8 lo0 > > 2002:ff00::/24 ::1 UGRS > > 0 0 32768 8 lo0 > > 2607:53xx:6x:7a3b::/64 2607:53xx:6x:7a3b:200:d3ff:fe00:1200 UCn > > 0 0 - 4 vio0 > > 2607:53xx:6x:7a3b:200:d3ff:fe00:1200 00:00:d3:00:12:00 UHLl > > 0 0 - 1 vio0 > > fe80::/10 ::1 UGRS > > 0 1 32768 8 lo0 > > fec0::/10 ::1 UGRS > > 0 0 32768 8 lo0 > > fe80::%vio0/64 fe80::200:d3ff:fe00:1200%vio0 UCn > > 1 1 - 4 vio0 > > fe80::200:d3ff:fe00:1200%vio0 00:00:d3:00:12:00 UHLl > > 0 3 - 1 vio0 > > fe80::200:d3ff:fe00:d00d%vio0 00:00:d3:00:d0:0d UHLch > > 1 43 - 3 vio0 > > fe80::1%lo0 fe80::1%lo0 UHl > > 0 0 32768 1 lo0 > > ff01::/16 ::1 UGRS > > 0 1 32768 8 lo0 > > ff01::%vio0/32 fe80::200:d3ff:fe00:1200%vio0 Um > > 0 1 - 4 vio0 > > ff01::%lo0/32 ::1 Um > > 0 1 32768 4 lo0 > > ff02::/16 ::1 UGRS > > 0 1 32768 8 lo0 > > ff02::%vio0/32 fe80::200:d3ff:fe00:1200%vio0 Um > > 0 1 - 4 vio0 > > ff02::%lo0/32 ::1 Um > > 0 1 32768 4 lo0 > > # ifconfig vio0 > > vio0: > > flags=208b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST,AUTOCONF6> > > mtu 1500 > > lladdr 00:00:d3:00:12:00 > > index 1 priority 0 llprio 3 > > groups: egress > > media: Ethernet autoselect > > status: active > > inet 10.0.23.30 netmask 0xffffff00 broadcast 10.0.23.255 > > inet6 fe80::200:d3ff:fe00:1200%vio0 prefixlen 64 scopeid 0x1 > > inet6 2607:53xx:6x:7a3b:200:d3ff:fe00:1200 prefixlen 64 > > # ping6 fe80::200:d3ff:fe00:d00d%vio0 > > PING fe80::200:d3ff:fe00:d00d%vio0 (fe80::200:d3ff:fe00:d00d%vio0): 56 data > > bytes > > 64 bytes from fe80::200:d3ff:fe00:d00d%vio0: icmp_seq=0 hlim=64 time=0.344 > > ms > > 64 bytes from fe80::200:d3ff:fe00:d00d%vio0: icmp_seq=1 hlim=64 time=0.208 > > ms > > 64 bytes from fe80::200:d3ff:fe00:d00d%vio0: icmp_seq=2 hlim=64 time=0.207 > > ms > > ^C > > --- fe80::200:d3ff:fe00:d00d%vio0 ping statistics --- > > 3 packets transmitted, 3 packets received, 0.0% packet loss > > round-trip min/avg/max/std-dev = 0.207/0.253/0.344/0.064 ms > > # ping6 2607:f8b0:400a:800::200e > > PING 2607:f8b0:400a:800::200e (2607:f8b0:400a:800::200e): 56 data bytes > > ^C > > --- 2607:f8b0:400a:800::200e ping statistics --- > > 3 packets transmitted, 0 packets received, 100.0% packet loss > > # ndp -an > > Neighbor Linklayer Address Netif Expire S > > Flags > > 2607:53xx:6x:7a3b:200:d3ff:fe00:1200 00:00:d3:00:12:00 vio0 permanent R l > > fe80::200:d3ff:fe00:1200%vio0 00:00:d3:00:12:00 vio0 permanent R l > > fe80::200:d3ff:fe00:d00d%vio0 00:00:d3:00:d0:0d vio0 23h59m57s S R > > Back on the host: > > # ping6 2607:53xx:6x:7a3b:200:d3ff:fe00:1200%vether0 > > ping6: no address associated with name > > # ping6 fe80::200:d3ff:fe00:1200%vether0 > > PING fe80::200:d3ff:fe00:1200%vether0 (fe80::200:d3ff:fe00:1200%vether0): > > 56 data bytes > > 64 bytes from fe80::200:d3ff:fe00:1200%vether0: icmp_seq=0 hlim=64 > > time=0.265 ms > > 64 bytes from fe80::200:d3ff:fe00:1200%vether0: icmp_seq=1 hlim=64 > > time=0.237 ms > > 64 bytes from fe80::200:d3ff:fe00:1200%vether0: icmp_seq=2 hlim=64 > > time=0.218 ms > > 64 bytes from fe80::200:d3ff:fe00:1200%vether0: icmp_seq=3 hlim=64 > > time=0.220 ms > > ^C > > --- fe80::200:d3ff:fe00:1200%vether0 ping statistics --- > > 4 packets transmitted, 4 packets received, 0.0% packet loss > > round-trip min/avg/max/std-dev = 0.218/0.235/0.265/0.019 ms > > # ndp -a > > Neighbor Linklayer Address Netif Expire S > > Flags > > 2607:53xx:6x:7a3b:ec4:7aff:fe45:3734 0c:c4:7a:45:37:34 em0 permanent R l > > fe80::205:73ff:fea0:1%em0 00:05:73:a0:00:01 em0 18s R R > > fe80::2ff:ffff:feff:fffd%em0 00:ff:ff:ff:ff:fd em0 23h59m31s S R > > fe80::2ff:ffff:feff:fffe%em0 00:ff:ff:ff:ff:fe em0 23h58m49s S R > > fe80::ec4:7aff:fe45:3734%em0 0c:c4:7a:45:37:34 em0 permanent R l > > fe80::200:d3ff:fe00:1200%vether0 00:00:d3:00:12:00 vether0 23h59m34s S > > fe80::200:d3ff:fe00:d00d%vether0 00:00:d3:00:d0:0d vether0 permanent R > > lR > > Watching on em0 on the host I can see the v6 traffic from the guest leaving > and I can > see NDP traffic looking for who has 1200, but the reply never flows out: > > 1521332053.797396 00:ff:ff:ff:ff:fd 0c:c4:7a:45:37:34 86dd 86: > > fe80::2ff:ffff:feff:fffd > 2607:53xx:6x:7a3b:ec4:7aff:fe45:3734: icmp6: > > neighbor sol: who has 2607:53xx:6x:7a3b:ec4:7aff:fe45:3734 [class 0xe0] > > 1521332053.797413 0c:c4:7a:45:37:34 00:ff:ff:ff:ff:fd 86dd 78: > > fe80::ec4:7aff:fe45:3734 > fe80::2ff:ffff:feff:fffd: icmp6: neighbor adv: > > tgt is 2607:53xx:6x:7a3b:ec4:7aff:fe45:3734 > > 1521332054.211392 00:ff:ff:ff:ff:fd 33:33:ff:00:12:00 86dd 86: > > fe80::2ff:ffff:feff:fffd > ff02::1:ff00:1200: icmp6: neighbor sol: who has > > 2607:53xx:6x:7a3b:200:d3ff:fe00:1200 [class 0xe0] > > 1521332055.055953 00:05:73:a0:00:01 33:33:00:00:00:66 86dd 114: > > fe80::2ff:ffff:feff:fffd.2029 > ff02::66.2029: udp 52 [class 0xc0] > > 1521332055.393267 0c:c4:7a:45:37:34 00:05:73:a0:00:01 86dd 118: > > 2607:53xx:6x:7a3b:200:d3ff:fe00:1200 > 2607:f8b0:400a:800::200e: icmp6: > > echo request > > I also don't see the NDP traffic on vio0 on the guest. > > I'm probably missing something elementary, being a bit new to IPv6 routers. > Any > insights would be hugely helpful. >
Correction: I *do* see NDP traffic on vio0 on the guest after all, I didn't watch it long enough. -- 0x7D964D3361142ACF
