On 11/21/18 6:30 AM, Alexis Bauvin wrote: > Le 20 nov. 2018 à 22:45, David Ahern <d...@cumulusnetworks.com> a écrit : >> >> On 11/20/18 7:23 AM, Alexis Bauvin wrote: >>> We are trying to isolate the VXLAN traffic from different VMs with VRF as >>> shown >>> in the schemas below: >>> >>> +-------------------------+ +----------------------------+ >>> | +----------+ | | +------------+ | >>> | | | | | | | | >>> | | tap-red | | | | tap-blue | | >>> | | | | | | | | >>> | +----+-----+ | | +-----+------+ | >>> | | | | | | >>> | | | | | | >>> | +----+---+ | | +----+----+ | >>> | | | | | | | | >>> | | br-red | | | | br-blue | | >>> | | | | | | | | >>> | +----+---+ | | +----+----+ | >>> | | | | | | >>> | | | | | | >>> | | | | | | >>> | +----+--------+ | | +--------------+ | >>> | | | | | | | | >>> | | vxlan-red | | | | vxlan-blue | | >>> | | | | | | | | >>> | +------+------+ | | +-------+------+ | >>> | | | | | | >>> | | VRF | | | VRF | >>> | | red | | | blue | >>> +-------------------------+ +----------------------------+ >> >> Roopa and I were discussing this setup and are puzzled by the VRF >> association here. Does br-red and br-blue have an address? The commands >> below do not show it and from our perspective seems odd for this >> scenario. If it does not have an address, then there is no reason for >> the VRF labeling. > > Yes, br-red and br-blue have an address. Addresses (both MAC and IP) > are anycast addresses, to make an anycast gateway for the VMs behind > the taps. This serves for a classical evpn symmetric distributed > routing, both between vxlans and towards external networks. I am using > FRR as a control plane to exchange bgp-evpn information between > hypervisors. > > The schematic could have been more complete, including more bridges > and more taps in the vrf, with an added bridge+vxlan for the l3vni. > I did not include them as I wanted to focus on the underlay itself > and not on what vxlan was used for. > > Here is a more complete example: > +---------+ > | | > | vrf1000 | > | | > +--+-+-+--+ > | | | > +---------------------+ | +---------------+ > | | | > +------+------+ +------+------+ +----+---+ > | | | | | | > | br0 | | br1 | | br1000 | > | 10.0.0.1/24 | | 10.0.1.1/24 | | | > | | | | +----+---+ > +----+-+-+----+ +-----+-+-----+ | > | | | | | | > +-------+ | +-------+ +--+ +---+ | > | | | | | | > +--+---+ +---+--+ +----+---+ +---+--+ +---+----+ +-----+-----+ > | | | | | | | | | | | | > | tap0 | | tap1 | | vxlan0 | | tap2 | | vxlan1 | | vxlan1000 | > | | | | | | | | | | | | > +------+ +------+ +--------+ +------+ +--------+ +-----------+ > . . . > . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . > . > . +---------+ +--------------+ > . | | | | > . | vrf1001 | | vrf-underlay | > . | | | | > . +---+-+---+ +-------+------+ > . | | | > . +---------+ +----------+ | > . | | | > . +------+------+ +----+---+ | > . | | | | | > . | br2 | | br1001 | | > . | 10.0.2.1/24 | | | | > . | | +----+---+ | > . +----+-+-+----+ | | > . | | | | | > . +-----+ | +-------+ | | > . | | | | +-------+-------+ > . +---+--+ +--+---+ +---+----+ +-----+-----+ | | > . | | | | | | | | | eth0.2030 | > . | tap3 | | tap4 | | vxlan2 | | vxlan1001 | | 172.16.0.2/24 | > . | | | | | | | | | | > . +------+ +------+ +--------+ +-----------+ +---------------+ > . . . ^ . > . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . > (vxlan lower device is eth0.2030) v > +----------------+ > | | > | eth0 | > | 192.168.1.2/24 | > | | > +----------------+ > > In this example, the underlay is rather simple. A more complex case > could be with several uplinks in the underlay vrf, and a dummy/loopback > with a /32 ip, used as a vxlan lower device and to bind routing daemons > (e.g. bgpd). > > +--------------+ > | | > | vrf-underlay | > | | > +-----+-+-+----+ > | | | > +---------------+ | +---------------+ > | | | > +-------+-------+ +-------+-------+ +-------+--------+ > | | | | | | > | eth1 | | eth2 | | dummy0 | > | 172.16.0.2/31 | | 172.16.0.4/31 | | 192.168.0.1/32 | > | | | | | | > +---------------+ +---------------+ +----------------+ > . > . . . . . . . . . . . . . . . . . . . . . . . > (vxlan lower device is dummy0)
Thanks for sending a more complete example. I forwarded it to some FRR folks as well. > > As for default vrf, it would contain for example management traffic. We > don’t want to mix the underlay routing tables to other traffics. Management VRF? :-) > >> Also, it would be good to have a unit test this case. Can you create a >> shell script that creates the setup and runs a few tests verifying >> connectivity? You can use network namespaces and veth pairs in place of >> the VM with a tap device. From there the functionality is the same. >> Tests can be initial VRF association for the vxlan lower device, >> changing the VRF to another device, and then changing again back to >> default VRF - checking proper connectivity for each. > > Sure! I’ve just finished writing it, but I am not sure of the best way > so send it. I guess I will have to add it to a v4 of the patches, in > tools/testing/selftests/net? yes, that would be good. Thank you for writing it.
