Hi Len,
Hi Remove the Ip addresses from the agg0 interfaces
put the Ip addresses on the vlan interfaces only
ie
mg /etc/hostname.vlanxxx
up vnetid xxx
inet 10.10.xx.1/24
if you need to route between the vlans make sure you enable forwarding in
the kernel with sysctl
when you get it working make sure to post to the Misc List :)
Hope this helps,
On Thu, 12 Nov 2020 at 00:18, len zaifman <leona...@sympatico.ca> wrote:
> I am setting up a new system as a firewall using OpenBSD 6.8 current
> -uname -a
> OpenBSD fw1.lfz.net 6.8 GENERIC.MP#175 amd64.
>
> I have 3 vlans 70,77,79 on the firewall using two em devices, em0 and
> em1, in an aggregation to serve these vlans.
>
>
> There is a Unifi switch which has 2 ports (where em0,em1 are attached)
> set up to pass tagged vlans 70,77,79. The switch ip is 10.10.70.3.
>
> I have a linux host setup on vans 70,77,79 and at address 77 -
> 10.10.70.77, 10.10.77.77,10.10.79.77.
>
>
> So far i cannot communicate over the vlans. Before I vlanned these
> subnets : ie only vlan 1 everywhere - communication worked fine.
>
> So i do not believe there is a physical issue. The issues arose with the
> introduction of the vlans. Is there a configuration issue that anyone
> can spot?
>
>
> Thank you for any help you can give.
>
> Evidence:
>
> ping on the firewall works locally
>
> for n in 0 7 9 ; do ping -c 2 10.10.7${n}.1 ; done
> PING 10.10.70.1 (10.10.70.1): 56 data bytes
> 64 bytes from 10.10.70.1: icmp_seq=0 ttl=255 time=0.037 ms
> 64 bytes from 10.10.70.1: icmp_seq=1 ttl=255 time=0.025 ms
>
> --- 10.10.70.1 ping statistics ---
> 2 packets transmitted, 2 packets received, 0.0% packet loss
> round-trip min/avg/max/std-dev = 0.025/0.031/0.037/0.006 ms
> PING 10.10.77.1 (10.10.77.1): 56 data bytes
> 64 bytes from 10.10.77.1: icmp_seq=0 ttl=255 time=0.038 ms
> 64 bytes from 10.10.77.1: icmp_seq=1 ttl=255 time=0.025 ms
>
> --- 10.10.77.1 ping statistics ---
> 2 packets transmitted, 2 packets received, 0.0% packet loss
> round-trip min/avg/max/std-dev = 0.025/0.031/0.038/0.006 ms
> PING 10.10.79.1 (10.10.79.1): 56 data bytes
> 64 bytes from 10.10.79.1: icmp_seq=0 ttl=255 time=0.038 ms
> 64 bytes from 10.10.79.1: icmp_seq=1 ttl=255 time=0.025 ms
>
> --- 10.10.79.1 ping statistics ---
> 2 packets transmitted, 2 packets received, 0.0% packet loss
> round-trip min/avg/max/std-dev = 0.025/0.032/0.038/0.007 ms
>
>
> ping to the switch does not work
>
> ping -c 2 10.10.70.3
> PING 10.10.70.3 (10.10.70.3): 56 data bytes
>
> --- 10.10.70.3 ping statistics ---
> 2 packets transmitted, 0 packets received, 100.0% packet loss
>
> ping to the linux host does not work.
>
> ping -c 2 10.10.70.3
> PING 10.10.70.3 (10.10.70.3): 56 data bytes
>
> --- 10.10.70.3 ping statistics ---
> 2 packets transmitted, 0 packets received, 100.0% packet loss
> [13:47:04] leonardz@fw1 etc>>for n in 0 7 9 ; do ping -c 2
> 10.10.7${n}.77 ; done
> PING 10.10.70.77 (10.10.70.77): 56 data bytes
>
> --- 10.10.70.77 ping statistics ---
> 2 packets transmitted, 0 packets received, 100.0% packet loss
> PING 10.10.77.77 (10.10.77.77): 56 data bytes
>
> --- 10.10.77.77 ping statistics ---
> 2 packets transmitted, 0 packets received, 100.0% packet loss
> PING 10.10.79.77 (10.10.79.77): 56 data bytes
>
> --- 10.10.79.77 ping statistics ---
> 2 packets transmitted, 0 packets received, 100.0% packet loss
>
> I did the tests both with pfctl -e (enabled) and pfctl -d (disabled). It
> made no difference
>
>
> The setup is described below
>
> Here is the setup:
>
> ===== hostname.aggr0
> debug
> trunkport em0
> trunkport em1
> up
> inet 10.10.70.1/24
> alias 10.10.77.1/24
> alias 10.10.79.1/24
>
>
> ===== hostname.em0
> up
>
> ===== hostname.em1
> up
>
>
> ===== hostname.vlan70
> parent aggr0 vnetid 70
> 10.10.70.0/24
>
> ===== hostname.vlan77
> parent aggr0 vnetid 77
> 10.10.77.0/24
>
> ===== hostname.vlan79
> parent aggr0 vnetid 79
> 10.10.79.0/24
>
>
> Ifconfig -A shows the vlans are setup
>
> ===== aggr0
> aggr0: flags=8847<UP,BROADCAST,DEBUG,RUNNING,SIMPLEX,MULTICAST> mtu 1500
> lladdr fe:e1:ba:d0:f4:8c
> index 6 priority 0 llprio 7
> trunk: trunkproto lacp
> trunk id: [(8000,fe:e1:ba:d0:f4:8c,0006,0000,0000),
> (8000,e0:63:da:8e:78:d7,03E8,0000,0000)]
> em0 lacp actor system pri 0x8000 mac fe:e1:ba:d0:f4:8c, key
> 0x6, port pri 0x8000 number 0x1
> em0 lacp actor state
> activity,aggregation,sync,collecting,distributing
> em0 lacp partner system pri 0x8000 mac e0:63:da:8e:78:d7, key
> 0x3e8, port pri 0x1 number 0x9
> em0 lacp partner state
> activity,aggregation,sync,collecting,distributing
> em0 port active,collecting,distributing
> em1 lacp actor system pri 0x8000 mac fe:e1:ba:d0:f4:8c, key
> 0x6, port pri 0x8000 number 0x2
> em1 lacp actor state
> activity,aggregation,sync,collecting,distributing
> em1 lacp partner system pri 0x8000 mac e0:63:da:8e:78:d7, key
> 0x3e8, port pri 0x1 number 0xa
> em1 lacp partner state
> activity,aggregation,sync,collecting,distributing
> em1 port active,collecting,distributing
> groups: aggr
> media: Ethernet autoselect
> status: active
> inet 10.10.70.1 netmask 0xffffff00 broadcast 10.10.70.255
> inet 10.10.77.1 netmask 0xffffff00 broadcast 10.10.77.255
> inet 10.10.79.1 netmask 0xffffff00 broadcast 10.10.79.255
>
> ===== em0
> em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
> lladdr fe:e1:ba:d0:f4:8c
> index 1 priority 0 llprio 3
> trunk: trunkdev aggr0
> media: Ethernet autoselect (1000baseT full-duplex)
> status: active
>
> ===== em1
> em1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
> lladdr fe:e1:ba:d0:f4:8c
> index 2 priority 0 llprio 3
> trunk: trunkdev aggr0
> media: Ethernet autoselect (1000baseT full-duplex)
> status: active
> pfctl -sr
> block return all
> pass all flags S/SA
> block return in on ! lo0 proto tcp from any to any port 6000:6010
> block return out log proto tcp all user = 55
> block return out log proto udp all user = 55
> pass out log on aggr0 inet proto icmp from 10.10.70.0/24 to any label
> "pings"
> pass out log on aggr0 inet proto icmp from 10.10.77.0/24 to any label
> "pings"
> pass out log on aggr0 inet proto icmp from 10.10.79.0/24 to any label
> "pings"
> pass in on vlan70 all flags S/SA label "vlan70" tag vlan70
> pass out on vlan70 all flags S/SA label "vlan70o" tag vlan70o
> ===== vlan70
> vlan70: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
> lladdr fe:e1:ba:d0:f4:8c
> index 7 priority 0 llprio 3
> encap: vnetid 70 parent aggr0 txprio packet rxprio outer
> groups: vlan
> media: Ethernet autoselect
> status: active
> inet 10.10.70.0 netmask 0xffffff00 broadcast 10.10.70.255
>
> ===== vlan77
> vlan77: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
> lladdr fe:e1:ba:d0:f4:8c
> index 8 priority 0 llprio 3
> encap: vnetid 77 parent aggr0 txprio packet rxprio outer
> groups: vlan
> media: Ethernet autoselect
> status: active
> inet 10.10.77.0 netmask 0xffffff00 broadcast 10.10.77.255
>
> ===== vlan79
> vlan79: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
> lladdr fe:e1:ba:d0:f4:8c
> index 9 priority 0 llprio 3
> encap: vnetid 79 parent aggr0 txprio packet rxprio outer
> groups: vlan
> media: Ethernet autoselect
> status: active
> inet 10.10.79.0 netmask 0xffffff00 broadcast 10.10.79.255
>
>
> Routes
>
>
> netstat -f inet -rn
> Routing tables
>
> Internet:
> Destination Gateway Flags Refs Use Mtu Prio
> Iface
> default 192.168.7.1 UGS 5 4045 - 8 re0
> 224/4 127.0.0.1 URS 0 116 32768 8 lo0
> 10.10.70/24 10.10.70.1 UCPn 1 7387 - 4 aggr0
> 10.10.70/24 10.10.70.0 UCPn 0 0 - 4 vlan70
> 10.10.70.0 fe:e1:ba:d0:f4:8c UHLl 0 0 - 1 vlan70
> 10.10.70.1 fe:e1:ba:d0:f4:8c UHLl 0 26 - 1 aggr0
> 10.10.70.3 e0:63:da:8e:78:d7 UHLc 0 7158 - 3 aggr0
> 10.10.70.255 10.10.70.1 UHPb 0 0 - 1 aggr0
> 10.10.70.255 10.10.70.0 UHPb 0 0 - 1 vlan70
> 10.10.77/24 10.10.77.1 UCPn 0 1 - 4 aggr0
> 10.10.77/24 10.10.77.0 UCPn 0 0 - 4 vlan77
> 10.10.77.0 fe:e1:ba:d0:f4:8c UHLl 0 0 - 1 vlan77
> 10.10.77.1 fe:e1:ba:d0:f4:8c UHLl 0 31 - 1 aggr0
> 10.10.77.255 10.10.77.1 UHPb 0 0 - 1 aggr0
> 10.10.77.255 10.10.77.0 UHPb 0 0 - 1 vlan77
> 10.10.79/24 10.10.79.1 UCPn 0 1 - 4 aggr0
> 10.10.79/24 10.10.79.0 UCPn 0 0 - 4 vlan79
> 10.10.79.0 fe:e1:ba:d0:f4:8c UHLl 0 0 - 1 vlan79
> 10.10.79.1 fe:e1:ba:d0:f4:8c UHLl 0 36 - 1 aggr0
> 10.10.79.255 10.10.79.1 UHPb 0 0 - 1 aggr0
> 10.10.79.255 10.10.79.0 UHPb 0 0 - 1 vlan79
> 127/8 127.0.0.1 UGRS 0 0 32768 8 lo0
> 127.0.0.1 127.0.0.1 UHhl 1 17 32768 1 lo0
> 192.168.7/24 192.168.7.4 UCn 1 0 - 4 re0
> 192.168.7.1 00:1b:21:18:88:72 UHLch 5 14796 - 3 re0
> 192.168.7.4 8c:ec:4b:7a:04:dc UHLl 0 184 - 1 re0
> 192.168.7.255 192.168.7.4 UHb 0 0 - 1 re0
>
>
> the pf rules when pf enabled
>
> pfctl -sr
> block return all
> pass all flags S/SA
> block return in on ! lo0 proto tcp from any to any port 6000:6010
> block return out log proto tcp all user = 55
> block return out log proto udp all user = 55
> pass out log on aggr0 inet proto icmp from 10.10.70.0/24 to any label
> "pings"
> pass out log on aggr0 inet proto icmp from 10.10.77.0/24 to any label
> "pings"
> pass out log on aggr0 inet proto icmp from 10.10.79.0/24 to any label
> "pings"
> pass in on vlan70 all flags S/SA label "vlan70" tag vlan70
> pass out on vlan70 all flags S/SA label "vlan70o" tag vlan70o
>
> sysctl for ip forwarding is set
>
> net.inet.ip.forwarding=1
>
>
>
--
Kindest regards,
Tom Smyth.
