Hi,
With the backports for multiple DGP applied and using --gateway-port, the
traffic between AZs is not natted and routed as expected and the traffic to
Internet works fine and it is natted.
But when I do not use --gateway-port, the traffic to the Internet works
fine but the traffic against a remote AZ that should be routed does not
work. So this following statement may not be correct, as having multiple
DGPs the traffic that supposed to be natted is working without use
--gateway-port:
"--gateway-port option allows specifying the distributed
gateway port of router where the NAT rule needs to be
applied. GATEWAY_PORT should reference a
Logical_Router_Port row that is a distributed gateway port
of router. When router has multiple distributed gateway
ports and the gateway port for this NAT can’t be inferred
from the external_ip, it is an error to not specify the
GATEWAY_PORT."
When checking the traffic between AZs, in the local AZ where --gateway-port
is not being in use, the traffic from a remote AZ is dropped:
recirc_id(0),tunnel(tun_id=0xff0002,src=192.168.40.50,dst=192.168.40.221,geneve({class=0x102,type=0x80,len=4,0x30002/0x7fffffff}),flags(-df+csum+key)),in_port(2),ct_label(0/0x2),eth(src=aa:aa:aa:aa:aa:10,ds
t=aa:aa:aa:aa:aa:20),eth_type(0x0800),ipv4(src=
8.0.0.0/248.0.0.0,dst=20.0.1.2,ttl=63,frag=no), packets:2, bytes:196,
used:0.345s,
actions:set(eth(src=00:de:ad:fe:00:01,dst=00:de:ad:01:00:01)),set(ipv4(ttl=6
2)),3
recirc_id(0),in_port(3),ct_label(0/0x2),eth(src=00:de:ad:01:00:01,dst=00:de:ad:fe:00:01),eth_type(0x0800),ipv4(src=
20.0.1.2/255.255.255.254,dst=10.0.1.0/255.255.255.0,ttl=64,frag=no),
packets:2, bytes:196,
used:0.345s, actions:drop
But for a local AZ where --gateway-port is being in use, the traffic from a
remote AZ works fine:
recirc_id(0),in_port(3),ct_label(0/0x2),eth(src=00:de:ad:01:00:01,dst=00:de:ad:fe:00:01),eth_type(0x0800),ipv4(src=
30.0.1.2/255.255.255.254,dst=10.0.1.0/255.255.255.0,tos=0/0x3,ttl=64,frag=no),
packets:2, b
ytes:196, used:0.596s,
actions:set(tunnel(tun_id=0xff0002,dst=192.168.40.50,ttl=64,tp_dst=6081,geneve({class=0x102,type=0x80,len=4,0x10003}),flags(df|csum|key))),set(eth(src=aa:aa:aa:aa:aa:30,dst=aa:aa:aa:a
a:aa:10)),set(ipv4(ttl=63)),2
recirc_id(0),tunnel(tun_id=0xff0002,src=192.168.40.50,dst=192.168.40.247,geneve({class=0x102,type=0x80,len=4,0x30001/0x7fffffff}),flags(-df+csum+key)),in_port(2),ct_label(0/0x2),eth(src=aa:aa:aa:aa:aa:10,ds
t=aa:aa:aa:aa:aa:30),eth_type(0x0800),ipv4(src=
8.0.0.0/248.0.0.0,dst=30.0.1.2,ttl=63,frag=no), packets:2, bytes:196,
used:0.596s,
actions:set(eth(src=00:de:ad:fe:00:01,dst=00:de:ad:01:00:01)),set(ipv4(ttl=6
2)),3
It seems I have to use --gateway-port when using OVN Interconnect even
knowing both are not related. So, it seems there is a bug.
Tiago Pires
On Wed, Mar 15, 2023 at 7:48 PM Han Zhou <hz...@ovn.org> wrote:
>
>
> On Wed, Mar 15, 2023 at 1:00 PM Tiago Pires <tiago.pi...@luizalabs.com>
> wrote:
> >
> > Hi Vladislav,
> >
> > It seems the gateway_port option was added on 22.09 according with this
> commit:
> https://github.com/ovn-org/ovn/commit/4f93381d7d38aa21f56fb3ff4ec00490fca12614
> .
> > It is what I need in order to make my use case to work, let me try it.
>
> Thanks for reporting this issue. It would be good to try the gateway_port
> option, but it also seems to be a bug somewhere if it behaves as what you
> described in the example, because:
>
> "When a logical router has multiple distributed gateway ports and this
> column is not set for a NAT
> rule, then the rule will be applied at the distributed
> gateway port which is in the same network as
> the external_ip of the NAT rule "
>
> We need to check more on this.
>
> Regards,
> Han
>
> >
> > Thank you
> >
> > Tiago Pires
> >
> >
> >
> > On Wed, Mar 15, 2023 at 2:10 PM Vladislav Odintsov <odiv...@gmail.com>
> wrote:
> >>
> >> I’m sorry, of course I meant gateway_port instead of logical_port:
> >>
> >> gateway_port: optional weak reference to Logical_Router_Port
> >> A distributed gateway port in the Logical_Router_Port
> table where the NAT rule needs to be applied.
> >>
> >> When multiple distributed gateway ports are configured on
> a Logical_Router, applying a NAT rule at
> >> each of the distributed gateway ports might not be
> desired. Consider the case where a logical router
> >> has 2 distributed gateway port, one with networks
> 50.0.0.10/24 and the other with networks
> >> 60.0.0.10/24. If the logical router has a NAT rule of
> type snat, logical_ip 10.1.1.0/24 and exter‐
> >> nal_ip 50.1.1.20/24, the rule needs to be selectively
> applied on matching packets entering/leaving
> >> through the distributed gateway port with networks
> 50.0.0.10/24.
> >>
> >> When a logical router has multiple distributed
> gateway ports and this column is not set for a NAT
> >> rule, then the rule will be applied at the distributed
> gateway port which is in the same network as
> >> the external_ip of the NAT rule, if such a router port
> exists. If logical router has a single dis‐
> >> tributed gateway port and this column is not set for a
> NAT rule, the rule will be applied at the dis‐
> >> tributed gateway port even if the router port is not
> in the same network as the external_ip of the
> >> NAT rule.
> >>
> >> On 15 Mar 2023, at 20:05, Vladislav Odintsov via discuss <
> ovs-discuss@openvswitch.org> wrote:
> >>
> >> Hi,
> >>
> >> since you’ve configured multiple LRPs with GW chassis, you must supply
> logical_port for NAT rule. Did you configure it?
> >> You should see appropriate message in ovn-northd logfile.
> >>
> >> logical_port: optional string
> >> The name of the logical port where the logical_ip resides.
> >>
> >> This is only used on distributed routers. This must be
> specified in order for the NAT rule to be pro‐
> >> cessed in a distributed manner on all chassis. If this
> is not specified for a NAT rule on a distrib‐
> >> uted router, then this NAT rule will be processed in a
> centralized manner on the gateway port
> >> instance on the gateway chassis.
> >>
> >> On 15 Mar 2023, at 19:22, Tiago Pires via discuss <
> ovs-discuss@openvswitch.org> wrote:
> >>
> >> Hi,
> >>
> >> In an OVN Interconnection environment (OVN 22.03) with a few AZs, I
> noticed that when the OVN router has a SNAT enabled or DNAT_AND_SNAT,
> >> the traffic between the AZs is nated.
> >> When checking the OVN router's logical flows, it is possible to see the
> LSP that is connected into the transit switch with NAT enabled:
> >>
> >> Scenario:
> >>
> >> OVN Global database:
> >> # ovn-ic-sbctl show
> >> availability-zone az1
> >> gateway ovn-central-1
> >> hostname: ovn-central-1
> >> type: geneve
> >> ip: 192.168.40.50
> >> port ts1-r1-az1
> >> transit switch: ts1
> >> address: ["aa:aa:aa:aa:aa:10 169.254.100.10/24"]
> >> availability-zone az2
> >> gateway ovn-central-2
> >> hostname: ovn-central-2
> >> type: geneve
> >> ip: 192.168.40.221
> >> port ts1-r1-az2
> >> transit switch: ts1
> >> address: ["aa:aa:aa:aa:aa:20 169.254.100.20/24"]
> >> availability-zone az3
> >> gateway ovn-central-3
> >> hostname: ovn-central-3
> >> type: geneve
> >> ip: 192.168.40.247
> >> port ts1-r1-az3
> >> transit switch: ts1
> >> address: ["aa:aa:aa:aa:aa:30 169.254.100.30/24"]
> >>
> >> OVN Central (az1)
> >>
> >> # ovn-nbctl show r1
> >> router 3e80e81a-58b5-41b1-9600-5bfc917c4ace (r1)
> >> port r1-ts1-az1
> >> mac: "aa:aa:aa:aa:aa:10"
> >> networks: ["169.254.100.10/24"]
> >> gateway chassis: [ovn-central-1]
> >> port r1_s1
> >> mac: "00:de:ad:fe:0:1"
> >> networks: ["10.0.1.1/24"]
> >> port r1_public
> >> mac: "00:de:ad:ff:0:1"
> >> networks: ["200.10.0.1/24"]
> >> gateway chassis: [ovn-central-1]
> >> nat df2b79d3-1334-4af3-8f61-5a46490f8a9c
> >> external ip: "200.10.0.101"
> >> logical ip: "10.0.1.2"
> >> type: "dnat_and_snat"
> >>
> >> OVN Logical Flows:
> >> table=3 (lr_out_snat ), priority=161 , match=(ip && ip4.src ==
> 10.0.1.2 && outport == "r1-ts1-az1" &&
> is_chassis_resident("cr-r1-ts1-az1")),
> action=(ct_snat_in_czone(200.10.0.101);)
> >>
> >> The datapath flows into OVS shows that the traffic is being nated and
> sent to the remote chassi gateway in AZ2:
> >>
> >>
> recirc_id(0x14),in_port(3),eth(src=aa:aa:aa:aa:aa:10,dst=aa:aa:aa:aa:aa:20),eth_type(0x0800),ipv4(dst=
> 200.16.0.0/255.240.0.0,tos=0/0x3,frag=no), packets:3, bytes:294,
> used:0.888s,
> actions:ct_clear,set(tunnel(tun_id=0xff0002,dst=192.168.40.221,ttl=64,tp_dst=6081,geneve({class=0x102,type=0x80,len=4,0x10002}),flags(df|csum|key))),2
> >>
> recirc_id(0x13),in_port(3),eth(),eth_type(0x0800),ipv4(src=10.0.1.2,frag=no),
> packets:3, bytes:294, used:0.888s,
> actions:ct(commit,zone=2,nat(src=200.10.0.101)),recirc(0x14)
> >>
> recirc_id(0),in_port(3),eth(src=00:de:ad:01:00:01,dst=00:de:ad:fe:00:01),eth_type(0x0800),ipv4(src=10.0.1.2,dst=
> 200.20.0.0/255.255.255.0,ttl=64,frag=no), packets:3, bytes:294,
> used:0.888s, actions:set(e
> >>
> th(src=aa:aa:aa:aa:aa:10,dst=aa:aa:aa:aa:aa:20)),set(ipv4(ttl=63)),ct(zone=2,nat),recirc(0x13)
> >>
> >> Is this behavior expected by design or is it a bug? In my use case, I
> would like for the traffic between AZs to be routed instead of nated.
> >>
> >> Tiago Pires
> >>
> >>
> >> ‘Esta mensagem é direcionada apenas para os endereços constantes no
> cabeçalho inicial. Se você não está listado nos endereços constantes no
> cabeçalho, pedimos-lhe que desconsidere completamente o conteúdo dessa
> mensagem e cuja cópia, encaminhamento e/ou execução das ações citadas estão
> imediatamente anuladas e proibidas’.
> >>
> >> ‘Apesar do Magazine Luiza tomar todas as precauções razoáveis para
> assegurar que nenhum vírus esteja presente nesse e-mail, a empresa não
> poderá aceitar a responsabilidade por quaisquer perdas ou danos causados
> por esse e-mail ou por seus anexos’.
> >>
> >> _______________________________________________
> >> discuss mailing list
> >> disc...@openvswitch.org
> >> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
> >>
> >>
> >>
> >> Regards,
> >> Vladislav Odintsov
> >>
> >> _______________________________________________
> >> discuss mailing list
> >> disc...@openvswitch.org
> >> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
> >>
> >>
> >>
> >> Regards,
> >> Vladislav Odintsov
> >>
> >
> >
> > ‘Esta mensagem é direcionada apenas para os endereços constantes no
> cabeçalho inicial. Se você não está listado nos endereços constantes no
> cabeçalho, pedimos-lhe que desconsidere completamente o conteúdo dessa
> mensagem e cuja cópia, encaminhamento e/ou execução das ações citadas estão
> imediatamente anuladas e proibidas’.
> >
> > ‘Apesar do Magazine Luiza tomar todas as precauções razoáveis para
> assegurar que nenhum vírus esteja presente nesse e-mail, a empresa não
> poderá aceitar a responsabilidade por quaisquer perdas ou danos causados
> por esse e-mail ou por seus anexos’.
>
--
_‘Esta mensagem é direcionada apenas para os endereços constantes no
cabeçalho inicial. Se você não está listado nos endereços constantes no
cabeçalho, pedimos-lhe que desconsidere completamente o conteúdo dessa
mensagem e cuja cópia, encaminhamento e/ou execução das ações citadas estão
imediatamente anuladas e proibidas’._
* **‘Apesar do Magazine Luiza tomar
todas as precauções razoáveis para assegurar que nenhum vírus esteja
presente nesse e-mail, a empresa não poderá aceitar a responsabilidade por
quaisquer perdas ou danos causados por esse e-mail ou por seus anexos’.*
_______________________________________________
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
