> On Dec 15, 2020, at 6:20 AM, Dumitru Ceara <dce...@redhat.com> wrote: > > On 12/15/20 12:02 PM, Krzysztof Klimonda wrote: >> Hi Dumitru, >> >> Thanks for checking it out. >> >> On Tue, Dec 15, 2020, at 10:45, Dumitru Ceara wrote: >>> Hi Krzysztof, >>> >>> Thanks for the DBs and all the details. >>> >>> I gave it a try on my local setup, using your DBs. The behavior below >>> is identical on v20.06.2, branch-20.12 and current master. >> >> I have to admit I'm not sure why in your tests the behaviour is the same in >> 20.06.2 - unfortunately I no longer have access to the environment with that >> version, so I can't re-verify that on my end and I based it on the fact that >> this SG was created by magnum for kubernetes cluster deployed on top of >> openstack, and the cluster was being tested successfully. The earliest I can >> try recreating this with 20.06.2 release is next year, so I'm not sure if >> this will be of any help. >> > > I just rechecked with 20.06.2 and I'm 100% sure that with the DBs you > provided and the behavior is exactly the same as with newer branches > (including latest master). > >> [...] >>> >>> For TCP traffic 172.16.0.11 -> 172.16.0.10 we don't hit any of the >>> allow-related ACLs. 172.16.0.11 is not set as a port address on port >>> 81d23182-37ac-4d3d-815e-4c25d26fe154 so it will not be included in the >>> auto-generated address_set pg_ed081ef3_754a_492f_80b2_fb73cd2dceed_ip4. >>> >>> On the other hand, for TCP traffic 10.0.0.11 -> 172.16.0.10 we hit the >>> 4th allow-related ACL: >>> to-lport 1002 (outport == @pg_ed081ef3_754a_492f_80b2_fb73cd2dceed && >>> ip4 && ip4.src == $pg_ed081ef3_754a_492f_80b2_fb73cd2dceed_ip4 && tcp) >>> allow-related >>> >>> ICMP traffic matches the 5th allow-related ACL which doesn't check >>> source IPs: >>> to-lport 1002 (outport == @pg_ed081ef3_754a_492f_80b2_fb73cd2dceed && >>> ip4 && ip4.src == 0.0.0.0/0 && icmp4) allow-related >>> >>> So, unless I'm missing something, OVN is doing what it's supposed to do >>> based on the current configuration. To avoid adding a new ACL, one >>> option would be to add the 172.16.0.10 and 172.16.0.11 IPs to the >>> corresponding logical_switch_ports "addresses" column, i.e., something like: >> >> The problem with adding those addresses to LSP is that they are outside of >> OVN/neutron control - this is a kubernetes cluster deployed with magnum, >> where CNI plugin (calico) doesn't use any tunneling between k8s nodes, each >> pod has address in 172.16.0.0/16 subnet and uses addresses from >> 172.16.0.0/16 to communicate witch other pods. To accomodate that ports have >> 172.16.0.0/16 added to their allowed addresses. >> >> I'm assuming that, if OVN is doing what it's supposed to be doing based on >> the configuration[1], then there is a mismatch between neutron and OVN >> behaviour in regards to SG with allowed address pairs? I guess someone from >> neutron team would have to comment whether it's > > I agree, some input from neutron would be great. (cc-ing Daniel).
Hi folks. I'm caching up on this thread and have a follow up question/request. @Krzysztof: can you tell me a little more about how the ips 172.16.0.10 and 172.16.0.11 are added as secondary on the neutron ports? Are you using multiple fixed ip addresses? Do you have 2 neutron subnets configured on the same neutron network? The reason I ask is because a neutron network and its associated neutron subnet(s) are "aggregated" into a single logical switch in OVN. In fact, neutron subnets have no corresponding row in OVN. So I wonder if what you really want are 2 separate neutron ports for each vm, instead of a single port with overlapping subnets. Thanks, -- flaviof > >> >> [1] I'm not entirely convinced that's the case given that ICMP traffic is >> being forwarded - I see how it's doing what programmed flows are telling it >> to do, but that doesn't seem to be expected result. >> > > I'm confused, why wouldn't this be expected? > > The ICMP ACL explicitly allows traffic to any port in the port group if > it's ICMP and comes from any IP source: > > to-lport 1002 (outport == @pg_ed081ef3_754a_492f_80b2_fb73cd2dceed && > ip4 && ip4.src == 0.0.0.0/0 && icmp4) allow-related > > For TCP the match is more restrictive: > > The TCP ACL allows traffic to any port in the port group if the IP > source is part of the addresses defined on the logical ports that are > part of the port group: > > to-lport 1002 (outport == @pg_ed081ef3_754a_492f_80b2_fb73cd2dceed > &&ip4 && ip4.src == $pg_ed081ef3_754a_492f_80b2_fb73cd2dceed_ip4 && tcp) > allow-related > > But the traffic comes from 172.16.0.11 which is not an address on the > logical switch port. So traffic doesn't match the allow ACL and instead > matches the deny-all ACL. > > OVN cannot automatically deduce anything from the fact that > port_security includes 172.16.0.0/16 and without help from the CMS it > cannot decide to allow traffic from 172.16.0.11. > > Regards, > Dumitru > >> Best Regards, >> Chris >> >>> >>> Thanks, >>> Dumitru >>> >>> >>> On 12/14/20 4:43 PM, Krzysztof Klimonda wrote: >>>> Hi Numan, >>>> >>>> https://signal.klimonda.com/ovnnb.db-broken.txt - this is the "initial" >>>> state where TCP is not being established (but ping works) >>>> https://signal.klimonda.com/ovnnb.db-working.txt - this is after I create >>>> a separate IP-based rule to allow TCP traffic >>>> >>>> In both examples, security group in question is >>>> ed081ef3-754a-492f-80b2-fb73cd2dceed which is mapped to >>>> pg_ed081ef3_754a_492f_80b2_fb73cd2dceed port group. >>>> >>>> In the second ovnnb.db (ovnnb.db-working.txt), there is an extra ACL >>>> fb464efc-f63b-494b-b59b-6c2860dcecba added from CLI via: >>>> >>>> `openstack security group rule create --ingress --protocol tcp --remote-ip >>>> 172.16.0.0/24 default` >>>> >>>> -- Krzysztof Klimonda kklimo...@syntaxhighlighted.com On Mon, Dec 14, >>>> 2020, at 14:14, Numan Siddique wrote: >>>>> On Mon, Dec 14, 2020 at 4:01 PM Krzysztof Klimonda >>>>> <kklimo...@syntaxhighlighted.com> wrote: >>>>>> Hi, >>>>>> >>>>>> After upgrading to OVN 20.12.0 snapshot d8bc0377c I've noticed a problem >>>>>> in communication between VMs that use allowed address pairs and remote >>>>>> group id in security groups. I believe it has worked properly with OVN >>>>>> 20.06.2 release (although I have no way of verifying it right now). >>>>>> >>>>> Thanks for reporting the issue. >>>>> >>>>> Is it possible for you to share the OVN NB DB somewhere ? >>>>> >>>>> It would be easier to reproduce the issue with the DB. >>>>> >>>>> Thanks >>>>> Numan >>>>> >>>>>> Given the following scenario: >>>>>> >>>>>> - 2 VMs with IP addresses: vm-a with IP addresses 10.0.0.10 and >>>>>> 172.16.0.10 and vm-b with IP addresses 10.0.0.11 and 172.16.0.11 where >>>>>> 10.0.0.0/8 addresses are set on ports, and 172.16.0.0/16 addresses are >>>>>> set in allowed-address for on ports >>>>>> - There is single security group attached to both ports allowing for >>>>>> ingress tcp traffic coming from the same security group (remote-group) >>>>>> - There is a TCP service listening on 10.0.0.10 on port 8000 >>>>>> >>>>>> When I try accessing service from vm-b using 10.0.0.10 address, ovn >>>>>> forwards traffic properly. However, when I try accessing same service >>>>>> via 172.16.0.10 traffic is dropped. >>>>>> >>>>>> When I trace packets between VMs using ovn-trace, for first scenario the >>>>>> last step is: >>>>>> >>>>>> ----8<----8<---- >>>>>> ct_next(ct_state=est|trk /* default (use --ct to customize) */) >>>>>> --------------------------------------------------------------- >>>>>> 4. ls_out_acl_hint (ovn-northd.c:5292): !ct.new && ct.est && !ct.rpl && >>>>>> ct_label.blocked == 0, priority 4, uuid ab5a233e >>>>>> reg0[8] = 1; >>>>>> reg0[10] = 1; >>>>>> next; >>>>>> 5. ls_out_acl (ovn-northd.c:5498): reg0[8] == 1 && (outport == >>>>>> @pg_ed081ef3_754a_492f_80b2_fb73cd2dceed && ip4 && ip4.src == >>>>>> $pg_ed081ef3_754a_492f_80b2_fb73cd2dceed_ip4 && tcp), priority 2002, >>>>>> uuid d92706d4 >>>>>> next; >>>>>> 9. ls_out_port_sec_ip (ovn-northd.c:4525): outport == "864929" && >>>>>> eth.dst == fa:16:3e:bc:20:10 && ip4.dst == {255.255.255.255, >>>>>> 224.0.0.0/4, 10.0.0.10, 172.16.0.0/16}, priority 90, uuid ff3390b1 >>>>>> next; >>>>>> 10. ls_out_port_sec_l2 (ovn-northd.c:4929): outport == "864929" && >>>>>> eth.dst == {fa:16:3e:bc:20:10}, priority 50, uuid af91c05c >>>>>> output; >>>>>> /* output to "864929", type "" */ >>>>>> ----8<----8<---- >>>>>> >>>>>> However, when I use 172.16.0.0/24 addresses, the last step changes to: >>>>>> >>>>>> ----8<----8<---- >>>>>> ct_next(ct_state=est|trk /* default (use --ct to customize) */) >>>>>> --------------------------------------------------------------- >>>>>> 4. ls_out_acl_hint (ovn-northd.c:5292): !ct.new && ct.est && !ct.rpl && >>>>>> ct_label.blocked == 0, priority 4, uuid ab5a233e >>>>>> reg0[8] = 1; >>>>>> reg0[10] = 1; >>>>>> next; >>>>>> 5. ls_out_acl (ovn-northd.c:5553): reg0[10] == 1 && (outport == >>>>>> @neutron_pg_drop && ip), priority 2001, uuid e36c0840 >>>>>> ct_commit { ct_label.blocked = 1; }; >>>>>> ----8<----8<---- >>>>>> >>>>>> Further notes: >>>>>> >>>>>> - ICMP traffic between 172.16.0.0/24 addresses is forwarded correctly, >>>>>> with last step of ovn-trace being: >>>>>> >>>>>> ----8<----8<---- >>>>>> ct_next(ct_state=est|trk /* default (use --ct to customize) */) >>>>>> --------------------------------------------------------------- >>>>>> 4. ls_out_acl_hint (ovn-northd.c:5292): !ct.new && ct.est && !ct.rpl && >>>>>> ct_label.blocked == 0, priority 4, uuid ab5a233e >>>>>> reg0[8] = 1; >>>>>> reg0[10] = 1; >>>>>> next; >>>>>> 5. ls_out_acl (ovn-northd.c:5498): reg0[8] == 1 && (outport == >>>>>> @pg_ed081ef3_754a_492f_80b2_fb73cd2dceed && ip4 && ip4.src == 0.0.0.0/0 >>>>>> && icmp4), priority 2002, uuid cd1705d8 >>>>>> next; >>>>>> 9. ls_out_port_sec_ip (ovn-northd.c:4525): outport == "864929" && >>>>>> eth.dst == fa:16:3e:bc:20:10 && ip4.dst == {255.255.255.255, >>>>>> 224.0.0.0/4, 10.0.0.10, 172.16.0.0/16}, priority 90, uuid ff3390b1 >>>>>> next; >>>>>> 10. ls_out_port_sec_l2 (ovn-northd.c:4929): outport == "864929" && >>>>>> eth.dst == {fa:16:3e:bc:20:10}, priority 50, uuid af91c05c >>>>>> output; >>>>>> /* output to "864929", type "" */ >>>>>> ----8<----8<---- >>>>>> >>>>>> - If I replace security group rule, changing remote group to remote ip, >>>>>> traffic is forwarded correctly and last step in ovn-trace is: >>>>>> >>>>>> ----8<----8<---- >>>>>> ct_next(ct_state=est|trk /* default (use --ct to customize) */) >>>>>> --------------------------------------------------------------- >>>>>> 4. ls_out_acl_hint (ovn-northd.c:5292): !ct.new && ct.est && !ct.rpl && >>>>>> ct_label.blocked == 0, priority 4, uuid ab5a233e >>>>>> reg0[8] = 1; >>>>>> reg0[10] = 1; >>>>>> next; >>>>>> 5. ls_out_acl (ovn-northd.c:5498): reg0[8] == 1 && (outport == >>>>>> @pg_ed081ef3_754a_492f_80b2_fb73cd2dceed && ip4 && ip4.src == >>>>>> 172.16.0.0/24 && tcp), priority 2002, uuid a0871ca2 >>>>>> next; >>>>>> 9. ls_out_port_sec_ip (ovn-northd.c:4525): outport == "864929" && >>>>>> eth.dst == fa:16:3e:bc:20:10 && ip4.dst == {255.255.255.255, >>>>>> 224.0.0.0/4, 10.0.0.10, 172.16.0.0/16}, priority 90, uuid ff3390b1 >>>>>> next; >>>>>> 10. ls_out_port_sec_l2 (ovn-northd.c:4929): outport == "864929" && >>>>>> eth.dst == {fa:16:3e:bc:20:10}, priority 50, uuid af91c05c >>>>>> output; >>>>>> /* output to "864929", type "" */ >>>>>> ----8<----8<---- >>>>>> >>>>>> -- >>>>>> Krzysztof Klimonda >>>>>> kklimo...@syntaxhighlighted.com >>>>>> _______________________________________________ >>>>>> discuss mailing list >>>>>> disc...@openvswitch.org >>>>>> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss >>>>>> >>>> _______________________________________________ >>>> discuss mailing list >>>> disc...@openvswitch.org >>>> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss >>>> >>> >>> >> > > _______________________________________________ > discuss mailing list > disc...@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
signature.asc
Description: Message signed with OpenPGP
_______________________________________________ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
