On Mon, Jan 29, 2018 at 10:11:08PM -0700, Scott Seekamp wrote:
> On 29.01.2018 17:28, Mike Larkin wrote:
>
> > On Mon, Jan 29, 2018 at 03:07:49PM -0700, Scott Seekamp wrote:
> >
> > > I'm attempting to set up an OpenBSD virtualized environment and
> > > running into
> > > issues.
> > >
> > > OpenBSD 6.2 AMD64 hypervisor
> > >
> > > vm.conf:
> > >
> > > vm "vm1" {
> > > disable
> > > memory 1g
> > > disk "/vmm/vm1.img"
> > > interface { switch "uplink" }
> > > }
> > >
> > > vm "vm2" {
> > > disable
> > > memory 1g
> > > disk "/vmm/vm2.img"
> > > interface { switch "uplink" }
> > > }
> > >
> > > vm "vm3" {
> > > disable
> > > memory 1g
> > > disk "/vmm/vm3.img"
> > > interface { switch "uplink" }
> > > }
> > >
> > > switch "uplink" {
> > > interface switch0
> > > add vlan50
> > > }
> > >
> > > I let vmd bring up the switch and once up it shows:
> > >
> > > switch0: flags=41<UP,RUNNING>
> > > description: switch1-uplink
> > > index 108 llprio 3
> > > groups: switch
> > > datapath 0x6393eae0ca8447fb maxflow 10000 maxgroup 1000
> > > vlan50 flags=0<>
> > > port 10 ifpriority 0 ifcost 0
> > > tap3 flags=0<>
> > > port 112 ifpriority 0 ifcost 0
> > > vether0 flags=1000<LOCAL>
> > > port 4294967294 ifpriority 0 ifcost 0
> > > tap1 flags=0<>
> > > port 115 ifpriority 0 ifcost 0
> > > tap2 flags=0<>
> > > port 116 ifpriority 0 ifcost 0
> > >
> > > vether0 defined as:
> > >
> > > vether0: flags=41<UP,RUNNING> mtu 1500
> > > lladdr fe:e1:ba:df:57:3b
> > > index 113 priority 0 llprio 3
> > > groups: vether egress
> > > media: Ethernet autoselect
> > > status: active
> > > inet 172.50.7.254 netmask 0xfffff800
> > >
> > > vlan50 is a vlan on top of a trunk (2 em's LACP to a switch):
> > >
> > > vlan50: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST>
> > > mtu 1500
> > > lladdr 00:25:90:0c:78:42
> > > description: Lab
> > > index 10 priority 0 llprio 3
> > > encap: vnetid 50 parent trunk0
> > > groups: vlan
> > > media: Ethernet autoselect
> > > status: active
> > >
> > > I have 3 vm's as defined above:
> > >
> > > ID PID VCPUS MAXMEM CURMEM TTY OWNER NAME
> > > 2 11840 1 1.0G 450M ttyp6 root vm1
> > > 1 14496 1 1.0G 560M ttyp2 root vm2
> > > 3 48053 1 1.0G 450M ttyp7 root vm3
> > >
> > > The vm's are configured with sequential IP's in the vlan50 network
> > > such as:
> > >
> > > vio0:
> > > flags=8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST>
> > > mtu 1500
> > > lladdr fe:e1:bb:d1:58:82
> > > index 1 priority 0 llprio 3
> > > groups: egress
> > > media: Ethernet autoselect
> > > status: active
> > > inet 172.50.0.6 netmask 0xfffff800 broadcast 172.50.7.255
> > >
> > > The problem:
> > >
> > > I can ping from vm -> vm
> > > I can ping from vm -> vether0 IP
> > > I can ping from vether0 to vm
> > >
> > > I cannot communicate out of the VM to the rest of the environment
> > > reliably.
> > >
> > > tcpdump on the tap interface shows:
> > >
> > > tcpdump -i tap2
> > > tcpdump: listening on tap2, link-type EN10MB
> > > 14:58:54.507948 172.50.0.6 > 172.50.0.1: icmp: echo request
> > > 14:58:56.527958 172.50.0.6 > 172.50.0.1: icmp: echo request
> > >
> > > tcpdump on the vlan interface shows the traffic gets out and tries
> > > to come
> > > back:
> > >
> > > tcpdump -i vlan50
> > > tcpdump: listening on vlan50, link-type EN10MB
> > > 15:02:23.657480 172.50.0.6 > 172.50.0.1: icmp: echo request
> > > 15:02:23.657656 172.50.0.1 > 172.50.0.6: icmp: echo reply
> > > 15:02:25.667671 172.50.0.6 > 172.50.0.1: icmp: echo request
> > > 15:02:25.667864 172.50.0.1 > 172.50.0.6: icmp: echo reply
> > >
> > > I'm using the out of the box pf rules on both the hypervisor box and
> > > vm's.
> > >
> > > I can see the macs of the vm's in the switch table:
> > >
> > > switchctl show sum
> > > Switch Port Type Name Info
> > > 1 switch /dev/switch0
> > > 1 10 mac 00:0d:b9:42:d0:fc age
> > > 135s
> > > 1 115 mac fe:e1:bb:d1:1c:f5 age
> > > 135s
> > > 1 116 mac fe:e1:bb:d1:58:82 age
> > > 201s
> > > 1 112 mac fe:e1:bb:d1:6d:f4 age
> > > 201s
> > >
> > > What am I missing? What would block the incoming traffic from
> > > getting back
> > > to the vm from the host level?
> > >
> > > Thanks
> > > Scott
> >
> > Can you try adding pass rules for the tapX interfaces to /etc/pf.conf
> > (don't
> > forget to reload the rules). And maybe the vether too. (Mine contains a
> > sequence of "pass on tap0" "pass on tap1" , etc, lines).
> >
> > I was having similar issues until I did that; I could never figure out
> > what
> > rule was blocking it but that seemed to fix it.
> >
> > Also, not sure if you need net.inet.ip.forwarding=1 in your sysctl
> > config
> > in this environment?
> >
> > -ml
> >
>
> Mike you are my hero - I can't believe I didn't try that. Looks like I'm in
> business thank you!
>
> Scott
>
Glad to hear it. Thanks for reporting this, at least it's in the list archives
in case someone else sees similar problems.
-ml
