I think this is more about BGP EVPN than CloudStack but would appreciate
anyone that could help. So basically, I've tried the Advanced Networking
with VLAN isolation for my POC and now want to migrate to VXLAN.
I would say that I've little to no knowledge about VXLAN in particular and
BGP EVPN. Why don't I use multicast instead is because our spine leaf
switches have been already configured with basic EVPN (though not tested
yet).
We have 2 spine switches (Mellanox SN2700) and 2 leaf switches (Mellanox
SN2410) running BGP unnumbered for underlay between them. Underlay between
the hypervisor that is running FRR and the leaf switches also configured
with BGP unnumbered.
All the switches and hypervisor are assigned with 4-byte private ASN
individually. Here is the FRR config on the hypervisor, all basic config:
```
ip forwarding
ipv6 forwarding
interface ens3f0np0
no ipv6 nd suppress-ra
exit
interface ens3f1np1
no ipv6 nd suppress-ra
exit
router bgp 4200100005
bgp router-id 10.XXX.118.1
no bgp ebgp-requires-policy
neighbor uplink peer-group
neighbor uplink remote-as external
neighbor ens3f0np0 interface peer-group uplink
neighbor ens3f1np1 interface peer-group uplink
address-family ipv4 unicast
network 10.XXX.118.1/32
exit-address-family
address-family ipv6 unicast
network 2407:XXXX:0:1::1/128
neighbor uplink activate
neighbor uplink soft-reconfiguration inbound
exit-address-family
address-family l2vpn evpn
neighbor uplink activate
neighbor uplink attribute-unchanged next-hop
advertise-all-vni
advertise-svi-ip
exit-address-family
```
Now I want to configure cloudbr0 bridge as the management interface for ACS.
I've done it like so:
```
nmcli connection add type bridge con-name cloudbr0 ifname cloudbr0 \
ipv4.method manual ipv4.addresses 10.XXX.113.11/24 ipv4.gateway 10.XXX.113.1
\
ipv4.dns 1.1.1.1,8.8.8.8 \
ipv6.method manual ipv6.addresses 2407:XXXX:200:c002::11/64 ipv6.gateway
2407:XXXX:200:c002::1 \
ipv6.dns 2606:4700:4700::1111,2001:4860:4860::8888 \
bridge.stp no ethernet.mtu 9216
nmcli connection add type vxlan slave-type bridge con-name vxlan10027 ifname
vxlan10027 \
id 10027 destination-port 4789 local 2407:XXXX:0:1::1 vxlan.learning no \
master cloudbr0 ethernet.mtu 9216 dev lo
nmcli connection up cloudbr0
nmcli connection up vxlan10027
```
I can see that the EVPN route with MAC address of cloudbr0 can be seen on
both leaf switches. However, I can't ping from the hypervisor to its gateway
(.1) which is a firewall running somewhere that's connected to a switchport
tagged with VLAN 27.
