On Thu, Dec 1, 2022 at 1:13 PM Gavin McKee via discuss
<ovs-discuss@openvswitch.org> wrote:
>
> Numan,
>
> I have a similar problem to Roberto, (not exact but I believe he will run
> into this issue also) in which I need a G-ARP to generate a type 2 mac-ip
> route in EVPN. When I add the NAT with the binding to the logical switch
> port and external MAC I see a G-ARP immediately , the switch learns this
> G-ARP and generates the correct ip route. The problem is garp is NOT sent
> periodically so if the ARP cache times out you always need traffic from the
> overlay to keep this alive. Should OVN be sending a periodic garp and if so
> on what interval?
I think garps are sent initially for a VIF with a burst of 5 or so packets.
If we want to send periodic garps, we need to enhance the
ovn-controller. Perhaps we can add an option to say if we want
continuous garps or an initial burst.
Please feel free to propose a patch if you think you can tackle this.
Otherwise can you please create a github issue so that it's not out of
radar.
Thanks
Numan
>
> I'm running Ovn 22.03.0
>
> root@usc01a-lab-ovn04:/home/gmckee# ovn-nbctl list nat
> _uuid : 59489984-ac1e-4f4b-8dc8-cace49a64360
> allowed_ext_ips : []
> exempted_ext_ips : []
> external_ids : {}
> external_ip : "216.147.118.8"
> external_mac : "0a:99:00:32:15:07"
> external_port_range : ""
> logical_ip : "10.200.1.12"
> logical_port : c1-sw1-p2
> options : {stateless="false"}
> type : dnat_and_snat
>
>
> root@usc01a-lab-ovn04:/home/gmckee# ovn-nbctl list Logical_Switch_Port
> localswitch-lr0
> _uuid : 010984e7-102c-4917-a6c9-59f6d554ffea
> addresses : [router]
> dhcpv4_options : []
> dhcpv6_options : []
> dynamic_addresses : []
> enabled : []
> external_ids : {}
> ha_chassis_group : []
> name : localswitch-lr0
> options : {nat-addresses=router, router-port=c1-lr0-localswitch}
> parent_name : []
> port_security : []
> tag : []
> tag_request : []
> type : router
> up : true
>
> On Thu, 1 Dec 2022 at 09:18, Numan Siddique via discuss
> <ovs-discuss@openvswitch.org> wrote:
>>
>> On Thu, Dec 1, 2022 at 10:15 AM Roberto Bartzen Acosta via discuss
>> <ovs-discuss@openvswitch.org> wrote:
>> >
>> > Hey folks,
>> >
>> > I would like some help to understand how DVR works in ovn/ovs. Currently,
>> > DVR support for FIP addresses works perfectly. However, I would like to
>> > understand if it is possible to extend this behavior to IPv6 addresses.
>> > When I talk about IPv6 addresses, I'm referring to the GUA addresses that
>> > are allocated to VMs (e.g., considering an openstack deployment).
>> >
>> > I understand that there are other requirements for GUA addresses to be
>> > routed to external networks by OVN (here comes the work of the
>> > neutron-dynamic-routing project + FRR on hypervisors/openvswitch). For DVR
>> > to be enabled properly, the provider networks must be stretched over the
>> > Underlay Network and each Compute Node would have the bridge for external
>> > traffic. In an L3 Leaf-Spine Underlay network one option for this
>> > requirement to be reached is for the Underlay Network be able to stretch
>> > an L2 domain(VLAN) via VXLAN as dataplane and BGP EVPN as Control Plane.
>> > In this solution, the Leaf switches would need to work as HW VTEP Gateway
>> > to initiate and terminate the VXLANs tunnels and use BGP EVPN to learn and
>> > advertise the MAC Addresses from the Compute Node’s provider network.
>> >
>> > The common reference architecture is detailed in [picture 1] and the
>> > design for the DVR+FRR solution (IPv4/IPv6) is detailed in [picture 2].
>> >
>> > What's the problem? well, inbound traffic to a GUA address goes through
>> > the chassis where the router's external port resides.
>> >
>> >
>> > Looking at the DVR implementation for IPv4, I see that the solution is
>> > heavily based on the idea of NAT. From initial support in [1], some fixes
>> > in [2] and [3], and scalability improvements in [4].
>> >
>> >
>> > With some debugs in the openflow pipeline, I see the entries for snat and
>> > dnat (FIP = 200.201.0.226) with reference to is_chassis_resident (with
>> > cr-lrp).
>> >
>> > _uuid : d37d412b-28ed-4d55-8f97-533571ffb3c1
>> > actions : "ct_dnat_in_czone(192.168.0.120);"
>> > controller_meter : []
>> > external_ids : {source="northd.c:12693", stage-hint="5e41244f",
>> > stage-name=lr_in_dnat}
>> > logical_datapath : c3ea6752-5b60-40ec-9734-dcfecbb59f68
>> > logical_dp_group : []
>> > match : "ip && ip4.dst == 200.201.0.226 && inport ==
>> > \"lrp-bcf42b8c-5ca5-44d1-8065-6f36892a1473\" &&
>> > is_chassis_resident(\"cr-lrp-bcf42b8c-5ca5-44d1-8065-6f36892a1473\")"
>> > pipeline : ingress
>> > priority : 100
>> > table_id : 6
>> > tags : {}
>> > hash : 0
>> >
>> >
>> > Does distributed router support for IPv6 need some kind of NAT rule to
>> > work? And about NAT, I saw that there is a test case on system-ovn.at [5]
>> > that validates NAT operation for a distributed router with IPv6 N/S
>> > traffic, is that right? maybe for local addresses it might make some
>> > sense, but for N/S traffic - GUA address (does this test make sense?).
>>
>> In the case of DVR for FIPS, when a NAT entry is added for the FIP in
>> the logical router, CMS has to set external_mac and logical_port
>> fields.
>> When these are set, ovn-controller claiming the VM of the FIP will
>> reply for the ARP requests for the FIP. The same for IPv6 FIPs. Also
>> the ovn-controller sends periodic GARPs.
>>
>> I think for the use case you mentioned, same has to happen. i.e For
>> a Neighbor Solicitation request for the GUA of the VM, the reply
>> should be sent from the same compute node hosting the VM.
>>
>> Can you please try adding a NAT entry in the logical router like below ?
>>
>> # ovn-nbctl --help | grep lr-nat-ad
>> lr-nat-add ROUTER TYPE EXTERNAL_IP LOGICAL_IP [LOGICAL_PORT EXTERNAL_MAC]
>>
>> # ovn-nbctl lr-nat-add <logical_router> dnat_and_snat <VM GUA> <VM
>> GUA> <VM logical port> <SOME CHOSEN RANDOM MAC>
>>
>> I've not tried this. If you notice the EXTERNAL_IP and LOGICAL_IP are
>> the same (ie VM GUA). If you add an entry like this,
>> ovn should add logical flows to respond to IPv6 NS requests for the VM GUA.
>>
>> If this doesn't work, then we need to think of a solution in OVN.
>>
>> Thanks
>> Numan
>>
>> >
>> > However, it is clear that when the external port of the router is in
>> > another chassis, the incoming traffic passes through the external port of
>> > the router [tcpdump 1]. The traffic is redirected to the chassis resident
>> > via geneve [dpcl 1] and goes through the resident chassis for the external
>> > destination GUA directly via vxlan[tcpdump 2] without going back through
>> > geneve to the router.
>> >
>> >
>> > Any ideas about how to program the ovn so that the chassis with the GUA
>> > address can receive and send traffic directly? (without redirection to the
>> > centralized router).
>> >
>> > Regards,
>> > Roberto
>> >
>> > [picture 1] -
>> > https://drive.google.com/file/d/1oaGmKbFGHqMwBxKVxsT4I-7rFt2pXSJW/view?usp=sharing
>> > [picture 2] -
>> > https://drive.google.com/file/d/1E-MRe9WJubPz5ZP836bNPFPRMuPCt4_s/view?usp=sharing
>> >
>> >
>> > [1]
>> > https://github.com/ovn-org/ovn/commit/ceacd9d49316d16b9273151bc1ecae9a2b2beeb8
>> > [2]
>> > https://github.com/ovn-org/ovn/commit/551e3d989557bd2249d5bbe0978b44b775c5e619
>> > [3]
>> > https://github.com/ovn-org/ovn/commit/8244c6b6bd8802a018e4ec3d3665510ebb16a9c7
>> > [4]
>> > https://github.com/ovn-org/ovn/commit/2dc7869436de32205f60128172196b3a207ab265
>> > [5] https://github.com/ovn-org/ovn/blob/main/tests/system-ovn.at#L3628
>> >
>> >
>> > [tcpdump 1] - router external port chassis
>> >
>> > root@compute1:~# tcpdump -nni br-301 not port 22 -ne
>> > tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
>> > listening on br-301, link-type EN10MB (Ethernet), snapshot length 262144
>> > bytes
>> > 14:54:23.442467 fa:16:3e:7b:d4:cf > a2:8f:16:4c:6e:e9, ethertype IPv4
>> > (0x0800), length 74: 200.201.0.110.57524 > 10.0.3.1.53: Flags [S], seq
>> > 2063839790, win 64492, options [mss 1402,sackOK,TS val 3936281288 ecr
>> > 0,nop,wscale 6], length 0
>> > 14:54:24.142076 a2:8f:16:4c:6e:e9 > fa:16:3e:9d:67:1f, ethertype IPv6
>> > (0x86dd), length 118: 2001:db8:aaa::2 > 2001:db8:1234::140: ICMP6, echo
>> > request, id 9014, seq 5, length 64
>> > 14:54:24.473037 fa:16:3e:7b:d4:cf > a2:8f:16:4c:6e:e9, ethertype IPv4
>> > (0x0800), length 74: 200.201.0.110.57524 > 10.0.3.1.53: Flags [S], seq
>> > 2063839790, win 64492, options [mss 1402,sackOK,TS val 3936282318 ecr
>> > 0,nop,wscale 6], length 0
>> > 14:54:25.143193 a2:8f:16:4c:6e:e9 > fa:16:3e:9d:67:1f, ethertype IPv6
>> > (0x86dd), length 118: 2001:db8:aaa::2 > 2001:db8:1234::140: ICMP6, echo
>> > request, id 9014, seq 6, length 64
>> > 14:54:25.166861 a2:8f:16:4c:6e:e9 > fa:16:3e:9d:67:1f, ethertype IPv6
>> > (0x86dd), length 86: fe80::a08f:16ff:fe4c:6ee9 > 2001:db8:4321:42::316:
>> > ICMP6, neighbor solicitation, who has 2001:db8:4321:42::316, length 32
>> > 14:54:25.168384 fa:16:3e:9d:67:1f > a2:8f:16:4c:6e:e9, ethertype IPv6
>> > (0x86dd), length 86: 2001:db8:4321:42::316 > fe80::a08f:16ff:fe4c:6ee9:
>> > ICMP6, neighbor advertisement, tgt is 2001:db8:4321:42::316, length 32
>> > 14:54:26.145280 a2:8f:16:4c:6e:e9 > fa:16:3e:9d:67:1f, ethertype IPv6
>> > (0x86dd), length 118: 2001:db8:aaa::2 > 2001:db8:1234::140: ICMP6, echo
>> > request, id 9014, seq 7, length 64
>> > 14:54:26.489182 fa:16:3e:7b:d4:cf > a2:8f:16:4c:6e:e9, ethertype IPv4
>> > (0x0800), length 74: 200.201.0.110.57524 > 10.0.3.1.53: Flags [S], seq
>> > 2063839790, win 64492, options [mss 1402,sackOK,TS val 3936284334 ecr
>> > 0,nop,wscale 6], length 0
>> > 14:54:27.146833 a2:8f:16:4c:6e:e9 > fa:16:3e:9d:67:1f, ethertype IPv6
>> > (0x86dd), length 118: 2001:db8:aaa::2 > 2001:db8:1234::140: ICMP6, echo
>> > request, id 9014, seq 8, length 64
>> > ^C
>> > 9 packets captured
>> > 9 packets received by filter
>> > 0 packets dropped by kernel
>> >
>> >
>> >
>> > [dpcl 1] - router external port chassis
>> >
>> > root@compute1:~# ovs-dpctl dump-flows
>> > recirc_id(0),tunnel(tun_id=0x0,src=192.168.200.30,dst=192.168.200.10,flags(-df+csum+key)),in_port(1),eth(),eth_type(0x0800),ipv4(proto=17,frag=no),udp(dst=3784),
>> > packets:183319, bytes:12099054, used:0.228s,
>> > actions:userspace(pid=4294967295,slow_path(bfd))
>> > recirc_id(0),in_port(5),eth(src=fa:16:3e:cc:79:0a,dst=fa:16:3e:2d:98:92),eth_type(0x0800),ipv4(src=192.168.0.199,dst=0.0.0.0/128.0.0.0,proto=6,frag=no),
>> > packets:50, bytes:3700, used:2.976s, flags:S,
>> > actions:ct(zone=7),recirc(0x4dc)
>> > recirc_id(0x4fa),in_port(5),eth(src=fa:16:3e:7b:d4:cf),eth_type(0x0800),ipv4(src=192.168.0.199,frag=no),
>> > packets:64306, bytes:4759173, used:2.976s, flags:S,
>> > actions:ct(commit,zone=1,nat(src=200.201.0.110)),recirc(0x4e2)
>> > recirc_id(0x629),in_port(4),ct_state(+new-est-rel-rpl-inv+trk),ct_label(0/0x1),eth(src=a2:8f:16:4c:6e:e9,dst=fa:16:3e:9d:67:1f),eth_type(0x86dd),ipv6(src=2000::/ffc0::,dst=2001:db8:1234::140,proto=58,tclass=0/0x3,hlimit=63,frag=no),
>> > packets:19, bytes:2242, used:0.284s,
>> > actions:ct_clear,set(tunnel(tun_id=0x2,dst=192.168.200.11,ttl=64,tp_dst=6081,geneve({class=0x102,type=0x80,len=4,0x60008}),flags(df|csum|key))),set(eth(src=fa:16:3e:26:0b:f7,dst=fa:16:3e:c4:15:40)),set(ipv6(hlimit=62)),1
>> > recirc_id(0x4e2),in_port(5),ct_state(+new-est-rel-rpl-inv+trk),ct_label(0/0x1),eth(src=fa:16:3e:7b:d4:cf,dst=a2:8f:16:4c:6e:e9),eth_type(0x0800),ipv4(dst=0.0.0.0/128.0.0.0,frag=no),
>> > packets:64306, bytes:4759173, used:2.976s, flags:S, actions:ct_clear,4
>> > recirc_id(0x4dc),in_port(5),ct_state(+new-est-rel-rpl-inv+trk),ct_label(0/0x1),eth(src=fa:16:3e:cc:79:0a,dst=fa:16:3e:2d:98:92),eth_type(0x0800),ipv4(src=192.168.0.199,dst=8.0.0.0/248.0.0.0,proto=6,ttl=64,frag=no),
>> > packets:50, bytes:3700, used:2.976s, flags:S,
>> > actions:ct(commit,zone=7,label=0/0x1,nat(src)),set(eth(src=fa:16:3e:7b:d4:cf,dst=a2:8f:16:4c:6e:e9)),set(ipv4(ttl=63)),ct(zone=1,nat),recirc(0x4fa)
>> > recirc_id(0),tunnel(tun_id=0x0,src=192.168.200.11,dst=192.168.200.10,flags(-df+csum+key)),in_port(1),eth(),eth_type(0x0800),ipv4(proto=17,frag=no),udp(dst=3784),
>> > packets:183257, bytes:12094962, used:0.404s,
>> > actions:userspace(pid=4294967295,slow_path(bfd))
>> > recirc_id(0),in_port(4),eth(src=a2:8f:16:4c:6e:e9,dst=fa:16:3e:9d:67:1f),eth_type(0x86dd),ipv6(src=2001:db8:aaa::/ffff:ffff:ffff:ffff::,dst=2001:db8:1234::140,proto=58,hlimit=63,frag=no),
>> > packets:19, bytes:2242, used:0.284s, actions:ct(zone=1,nat),recirc(0x628)
>> > recirc_id(0x628),in_port(4),eth(),eth_type(0x86dd),ipv6(dst=2001:db8:1234::140,frag=no),
>> > packets:19, bytes:2242, used:0.284s,
>> > actions:ct(commit,zone=1,nat(dst=2001:db8:1234::140)),recirc(0x629)
>> >
>> >
>> > [tcpdump 2] - GUA address chassis
>> >
>> > root@compute2:~# tcpdump -nni br-301 not port 22 -ne
>> > tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
>> > listening on br-301, link-type EN10MB (Ethernet), snapshot length 262144
>> > bytes
>> > 14:54:30.146532 fa:16:3e:0f:d9:1a > a2:8f:16:4c:6e:e9, ethertype IPv6
>> > (0x86dd), length 118: 2001:db8:1234::140 > 2001:db8:aaa::2: ICMP6, echo
>> > reply, id 9014, seq 11, length 64
>> > 14:54:31.149147 fa:16:3e:0f:d9:1a > a2:8f:16:4c:6e:e9, ethertype IPv6
>> > (0x86dd), length 118: 2001:db8:1234::140 > 2001:db8:aaa::2: ICMP6, echo
>> > reply, id 9014, seq 12, length 64
>> > ^C
>> > 2 packets captured
>> > 2 packets received by filter
>> > 0 packets dropped by kernel
>> >
>> >
>> >
>> >
>> >
>> > ‘Esta mensagem é direcionada apenas para os endereços constantes no
>> > cabeçalho inicial. Se você não está listado nos endereços constantes no
>> > cabeçalho, pedimos-lhe que desconsidere completamente o conteúdo dessa
>> > mensagem e cuja cópia, encaminhamento e/ou execução das ações citadas
>> > estão imediatamente anuladas e proibidas’.
>> >
>> > ‘Apesar do Magazine Luiza tomar todas as precauções razoáveis para
>> > assegurar que nenhum vírus esteja presente nesse e-mail, a empresa não
>> > poderá aceitar a responsabilidade por quaisquer perdas ou danos causados
>> > por esse e-mail ou por seus anexos’.
>> >
>> > _______________________________________________
>> > discuss mailing list
>> > disc...@openvswitch.org
>> > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
>> _______________________________________________
>> discuss mailing list
>> disc...@openvswitch.org
>> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
>
> _______________________________________________
> discuss mailing list
> disc...@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
_______________________________________________
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
