Hi Krzysztof, Thanks for the DBs and all the details.
I gave it a try on my local setup, using your DBs. The behavior below is identical on v20.06.2, branch-20.12 and current master. # ovn-nbctl --columns addresses,port_security list logical_switch_port 81d23182-37ac-4d3d-815e-4c25d26fe154 addresses : ["fa:16:3e:ee:19:b2 10.0.0.11"] port_security : ["fa:16:3e:ee:19:b2 10.0.0.11 172.16.0.0/16"] # ovn-nbctl --columns addresses,port_security list logical_switch_port 8649293f-8ca9-48d0-a0cd-1c2587e48889 addresses : ["fa:16:3e:bc:20:10 10.0.0.10"] port_security : ["fa:16:3e:bc:20:10 10.0.0.10 172.16.0.0/16"] # ovn-nbctl acl-list pg_ed081ef3_754a_492f_80b2_fb73cd2dceed from-lport 1002 (inport == @pg_ed081ef3_754a_492f_80b2_fb73cd2dceed && ip4) allow-related from-lport 1002 (inport == @pg_ed081ef3_754a_492f_80b2_fb73cd2dceed && ip6) allow-related to-lport 1002 (outport == @pg_ed081ef3_754a_492f_80b2_fb73cd2dceed && ip4 && ip4.src == $pg_ed081ef3_754a_492f_80b2_fb73cd2dceed_ip4) allow-related to-lport 1002 (outport == @pg_ed081ef3_754a_492f_80b2_fb73cd2dceed && ip4 && ip4.src == $pg_ed081ef3_754a_492f_80b2_fb73cd2dceed_ip4 && tcp) allow-related to-lport 1002 (outport == @pg_ed081ef3_754a_492f_80b2_fb73cd2dceed && ip4 && ip4.src == 0.0.0.0/0 && icmp4) allow-related to-lport 1002 (outport == @pg_ed081ef3_754a_492f_80b2_fb73cd2dceed && ip4 && ip4.src == 0.0.0.0/0 && tcp && tcp.dst == 22) allow-related to-lport 1002 (outport == @pg_ed081ef3_754a_492f_80b2_fb73cd2dceed && ip4 && ip4.src == 0.0.0.0/0 && udp && udp.dst == 12345) allow-related to-lport 1002 (outport == @pg_ed081ef3_754a_492f_80b2_fb73cd2dceed && ip6 && ip6.src == $pg_ed081ef3_754a_492f_80b2_fb73cd2dceed_ip6) allow-related # ovn-sbctl list address_set pg_ed081ef3_754a_492f_80b2_fb73cd2dceed_ip4 _uuid : e0769c02-d3a6-48e4-ab50-124785c501fd addresses : ["10.0.0.10", "10.0.0.11", "172.16.0.104", "172.16.0.147", "172.17.0.136", "172.17.0.149", "172.17.0.161", "172.17.0.6", "172.17.0.7", "172.17.0.94"] name : pg_ed081ef3_754a_492f_80b2_fb73cd2dceed_ip4 For TCP traffic 172.16.0.11 -> 172.16.0.10 we don't hit any of the allow-related ACLs. 172.16.0.11 is not set as a port address on port 81d23182-37ac-4d3d-815e-4c25d26fe154 so it will not be included in the auto-generated address_set pg_ed081ef3_754a_492f_80b2_fb73cd2dceed_ip4. On the other hand, for TCP traffic 10.0.0.11 -> 172.16.0.10 we hit the 4th allow-related ACL: to-lport 1002 (outport == @pg_ed081ef3_754a_492f_80b2_fb73cd2dceed && ip4 && ip4.src == $pg_ed081ef3_754a_492f_80b2_fb73cd2dceed_ip4 && tcp) allow-related ICMP traffic matches the 5th allow-related ACL which doesn't check source IPs: to-lport 1002 (outport == @pg_ed081ef3_754a_492f_80b2_fb73cd2dceed && ip4 && ip4.src == 0.0.0.0/0 && icmp4) allow-related So, unless I'm missing something, OVN is doing what it's supposed to do based on the current configuration. To avoid adding a new ACL, one option would be to add the 172.16.0.10 and 172.16.0.11 IPs to the corresponding logical_switch_ports "addresses" column, i.e., something like: # ovn-nbctl lsp-set-addresses 81d23182-37ac-4d3d-815e-4c25d26fe154 "fa:16:3e:ee:19:b2 10.0.0.11 172.16.0.11" # ovn-nbctl lsp-set-addresses 8649293f-8ca9-48d0-a0cd-1c2587e48889 "fa:16:3e:bc:20:10 10.0.0.10 172.16.0.10" # ovn-nbctl --columns addresses,port_security list logical_switch_port 81d23182-37ac-4d3d-815e-4c25d26fe154 addresses : ["fa:16:3e:ee:19:b2 10.0.0.11 172.16.0.11"] port_security : ["fa:16:3e:ee:19:b2 10.0.0.11 172.16.0.0/16"] # ovn-nbctl --columns addresses,port_security list logical_switch_port 8649293f-8ca9-48d0-a0cd-1c2587e48889 addresses : ["fa:16:3e:bc:20:10 10.0.0.10 172.16.0.10"] port_security : ["fa:16:3e:bc:20:10 10.0.0.10 172.16.0.0/16"] Now TCP traffic between 172.16.0.11 and 172.16.0.10 is allowed: $ ip netns exec vm2 nc -z -v -s 172.16.0.11 172.16.0.10 8000 Ncat: Version 7.50 ( https://nmap.org/ncat ) Ncat: Connected to 172.16.0.10:8000. Ncat: 0 bytes sent, 0 bytes received in 0.02 seconds. Thanks, Dumitru On 12/14/20 4:43 PM, Krzysztof Klimonda wrote: > Hi Numan, > > https://signal.klimonda.com/ovnnb.db-broken.txt - this is the "initial" state > where TCP is not being established (but ping works) > https://signal.klimonda.com/ovnnb.db-working.txt - this is after I create a > separate IP-based rule to allow TCP traffic > > In both examples, security group in question is > ed081ef3-754a-492f-80b2-fb73cd2dceed which is mapped to > pg_ed081ef3_754a_492f_80b2_fb73cd2dceed port group. > > In the second ovnnb.db (ovnnb.db-working.txt), there is an extra ACL > fb464efc-f63b-494b-b59b-6c2860dcecba added from CLI via: > > `openstack security group rule create --ingress --protocol tcp --remote-ip > 172.16.0.0/24 default` > > -- Krzysztof Klimonda kklimo...@syntaxhighlighted.com On Mon, Dec 14, > 2020, at 14:14, Numan Siddique wrote: >> On Mon, Dec 14, 2020 at 4:01 PM Krzysztof Klimonda >> <kklimo...@syntaxhighlighted.com> wrote: >>> Hi, >>> >>> After upgrading to OVN 20.12.0 snapshot d8bc0377c I've noticed a problem in >>> communication between VMs that use allowed address pairs and remote group >>> id in security groups. I believe it has worked properly with OVN 20.06.2 >>> release (although I have no way of verifying it right now). >>> >> Thanks for reporting the issue. >> >> Is it possible for you to share the OVN NB DB somewhere ? >> >> It would be easier to reproduce the issue with the DB. >> >> Thanks >> Numan >> >>> Given the following scenario: >>> >>> - 2 VMs with IP addresses: vm-a with IP addresses 10.0.0.10 and 172.16.0.10 >>> and vm-b with IP addresses 10.0.0.11 and 172.16.0.11 where 10.0.0.0/8 >>> addresses are set on ports, and 172.16.0.0/16 addresses are set in >>> allowed-address for on ports >>> - There is single security group attached to both ports allowing for >>> ingress tcp traffic coming from the same security group (remote-group) >>> - There is a TCP service listening on 10.0.0.10 on port 8000 >>> >>> When I try accessing service from vm-b using 10.0.0.10 address, ovn >>> forwards traffic properly. However, when I try accessing same service via >>> 172.16.0.10 traffic is dropped. >>> >>> When I trace packets between VMs using ovn-trace, for first scenario the >>> last step is: >>> >>> ----8<----8<---- >>> ct_next(ct_state=est|trk /* default (use --ct to customize) */) >>> --------------------------------------------------------------- >>> 4. ls_out_acl_hint (ovn-northd.c:5292): !ct.new && ct.est && !ct.rpl && >>> ct_label.blocked == 0, priority 4, uuid ab5a233e >>> reg0[8] = 1; >>> reg0[10] = 1; >>> next; >>> 5. ls_out_acl (ovn-northd.c:5498): reg0[8] == 1 && (outport == >>> @pg_ed081ef3_754a_492f_80b2_fb73cd2dceed && ip4 && ip4.src == >>> $pg_ed081ef3_754a_492f_80b2_fb73cd2dceed_ip4 && tcp), priority 2002, uuid >>> d92706d4 >>> next; >>> 9. ls_out_port_sec_ip (ovn-northd.c:4525): outport == "864929" && eth.dst >>> == fa:16:3e:bc:20:10 && ip4.dst == {255.255.255.255, 224.0.0.0/4, >>> 10.0.0.10, 172.16.0.0/16}, priority 90, uuid ff3390b1 >>> next; >>> 10. ls_out_port_sec_l2 (ovn-northd.c:4929): outport == "864929" && eth.dst >>> == {fa:16:3e:bc:20:10}, priority 50, uuid af91c05c >>> output; >>> /* output to "864929", type "" */ >>> ----8<----8<---- >>> >>> However, when I use 172.16.0.0/24 addresses, the last step changes to: >>> >>> ----8<----8<---- >>> ct_next(ct_state=est|trk /* default (use --ct to customize) */) >>> --------------------------------------------------------------- >>> 4. ls_out_acl_hint (ovn-northd.c:5292): !ct.new && ct.est && !ct.rpl && >>> ct_label.blocked == 0, priority 4, uuid ab5a233e >>> reg0[8] = 1; >>> reg0[10] = 1; >>> next; >>> 5. ls_out_acl (ovn-northd.c:5553): reg0[10] == 1 && (outport == >>> @neutron_pg_drop && ip), priority 2001, uuid e36c0840 >>> ct_commit { ct_label.blocked = 1; }; >>> ----8<----8<---- >>> >>> Further notes: >>> >>> - ICMP traffic between 172.16.0.0/24 addresses is forwarded correctly, with >>> last step of ovn-trace being: >>> >>> ----8<----8<---- >>> ct_next(ct_state=est|trk /* default (use --ct to customize) */) >>> --------------------------------------------------------------- >>> 4. ls_out_acl_hint (ovn-northd.c:5292): !ct.new && ct.est && !ct.rpl && >>> ct_label.blocked == 0, priority 4, uuid ab5a233e >>> reg0[8] = 1; >>> reg0[10] = 1; >>> next; >>> 5. ls_out_acl (ovn-northd.c:5498): reg0[8] == 1 && (outport == >>> @pg_ed081ef3_754a_492f_80b2_fb73cd2dceed && ip4 && ip4.src == 0.0.0.0/0 && >>> icmp4), priority 2002, uuid cd1705d8 >>> next; >>> 9. ls_out_port_sec_ip (ovn-northd.c:4525): outport == "864929" && eth.dst >>> == fa:16:3e:bc:20:10 && ip4.dst == {255.255.255.255, 224.0.0.0/4, >>> 10.0.0.10, 172.16.0.0/16}, priority 90, uuid ff3390b1 >>> next; >>> 10. ls_out_port_sec_l2 (ovn-northd.c:4929): outport == "864929" && eth.dst >>> == {fa:16:3e:bc:20:10}, priority 50, uuid af91c05c >>> output; >>> /* output to "864929", type "" */ >>> ----8<----8<---- >>> >>> - If I replace security group rule, changing remote group to remote ip, >>> traffic is forwarded correctly and last step in ovn-trace is: >>> >>> ----8<----8<---- >>> ct_next(ct_state=est|trk /* default (use --ct to customize) */) >>> --------------------------------------------------------------- >>> 4. ls_out_acl_hint (ovn-northd.c:5292): !ct.new && ct.est && !ct.rpl && >>> ct_label.blocked == 0, priority 4, uuid ab5a233e >>> reg0[8] = 1; >>> reg0[10] = 1; >>> next; >>> 5. ls_out_acl (ovn-northd.c:5498): reg0[8] == 1 && (outport == >>> @pg_ed081ef3_754a_492f_80b2_fb73cd2dceed && ip4 && ip4.src == 172.16.0.0/24 >>> && tcp), priority 2002, uuid a0871ca2 >>> next; >>> 9. ls_out_port_sec_ip (ovn-northd.c:4525): outport == "864929" && eth.dst >>> == fa:16:3e:bc:20:10 && ip4.dst == {255.255.255.255, 224.0.0.0/4, >>> 10.0.0.10, 172.16.0.0/16}, priority 90, uuid ff3390b1 >>> next; >>> 10. ls_out_port_sec_l2 (ovn-northd.c:4929): outport == "864929" && eth.dst >>> == {fa:16:3e:bc:20:10}, priority 50, uuid af91c05c >>> output; >>> /* output to "864929", type "" */ >>> ----8<----8<---- >>> >>> -- >>> Krzysztof Klimonda >>> kklimo...@syntaxhighlighted.com >>> _______________________________________________ >>> discuss mailing list >>> disc...@openvswitch.org >>> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss >>> > _______________________________________________ > discuss mailing list > disc...@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss > _______________________________________________ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
