On Fri, Sep 18, 2020 at 12:58 AM Alexander Constantinescu < acons...@redhat.com> wrote:
> Hi Han > > Sorry for the late reply. > > Is this the current situation? >> > > Yes, it is. > > When you say there are too many default routes, what do you mean in the >> above example? How would the SOUTH_TO_NORTH_IP solve the problem? >> > > Each <east-west routeX> corresponds to a node in our cluster, like this: > > <east-west route1> ip4.src == <logical switch Y subnet> && ip4.dst == > <node 1 IPv4>, allow > <east-west route1> ip6.src == <logical switch Y subnet> && ip6.dst == > <node 1 IPv6>, allow > <east-west route1> ip4.src == <logical switch Y subnet> && ip4.dst == > <node 2 IPv4>, allow > ... > <east-west route1> ip6.src == <logical switch Y subnet> && ip6.dst == > <node X IPv6>, allow > > so on large clusters (say 1000 nodes) with IPv6 and IPv4 enabled we can > reach ~2000 logical router policies. By having the SOUTH_TO_NORTH_IP we can > completely remove all of them and have the "default route" logical router > policy specify: > > default route (lowest priority): ip4.src == <logical switch Y subnet> > ip4.dst == SOUTH_TO_NORTH_IP, nexthop = <gateway router for Y> > > In addition, if SOUTH_TO_NORTH_IP is a user defined IP, >> > > I didn't think it should be user defined, more so "system defined", like > 0.0.0.0/0 > > I am not sure how would it work, because ip4.dst is the dst IP from >> packet header >> > > I didn't intend for such an IP to be used solely as a destination IP, but > as source too, if the user requires it. > > Comparing it with SOUTH_TO_NORTH_IP would just result in mismatch, unless >> all south-to-north traffic really has this IP as destination (I guess >> that's not the case). >> > > Sure, I just wanted to assess the feasibility of such an IP from OVN's > point of view. Obviously the real destination IP would be different, but I > (without knowing the underlying works of OVN) thought there might be a > programmable way of saying: "this IP is unknown to my network topology, so > I could use identifier/alias grouping all such IPs under an umbrella > identifier such as X.X.X.X/X" > > I see. You don't really want the packet's dst IP to be replaced, so what you need is some value in metadata and then you could define a policy to match that metadata and specify the nexthop. This is supported by the policy route table. However, what really matters here is how to define "unknown IPs". We can't say an IP is unknown just because there is no port with the IP assigned. There can still be east-west traffic for such IP, e.g. for nested workload in a VM, etc. From OVN's point of view this is completely user defined, and the best way for the user to tell OVN this information is through the route policies (which means you don't really need that extra metadata to achieve this). I think it is the way how k8s-ovn is using OVN that causes this many routes required. If the IP allocation can be better managed so that a big range of IP is for east-west and otherwise north-south, then you probably end up with a much smaller number of policies. > > On Wed, Sep 16, 2020 at 11:09 PM Han Zhou <hz...@ovn.org> wrote: > >> >> >> On Wed, Sep 16, 2020 at 10:07 AM Alexander Constantinescu < >> acons...@redhat.com> wrote: >> >>> In this example it is equivalent to just "ip4.src == 10.244.2.5/32"'. >>>> >>> >>> Yes, I was just using it as an example (though, granted, noop example) >>> >>> Some background to help steer the discussion: >>> >>> Essentially the functionality here is to have south -> north traffic >>> from certain logical switch ports exit the cluster through a dedicated node >>> (an egress node if you will). To do this we currently have a set of default >>> logical router policies, intended to leave east <-> west traffic untouched, >>> and then logical router policies with a lower priority, which specify >>> reroute actions for this functionality to happen. However, on large >>> clusters, there's this concern that the default logical router policies >>> will become too many. Hence why the idea here would be to drop them >>> completely and have this "special IP" that we can use to filter on the >>> destination, south -> north, traffic . >>> >>> If you have a default route, anything "unknown" would just hit the >>>> default route, right? Why would you need another IP for this purpose? >>>> >>> >>> As to remove the default logical router policies, which can become a >>> lot, on big clusters - as described above. With only reroute policies of >>> type: "ip4.src == 10.244.2.5/32 && ip4.dst == SOUTH_TO_NORTH_IP" things >>> would become lighter. >>> >> >> Thanks for the background. So you have: >> <east-west route1> >> <east-west route2> >> ... >> default route (lowest priority): ip4.src == <logical switch X subnet>, >> nexthop = <gateway router for X> >> default route (lowest priority): ip4.src == <logical switch Y subnet>, >> nexthop = <gateway router for Y> >> >> Is this the current situation? >> When you say there are too many default routes, what do you mean in the >> above example? How would the SOUTH_TO_NORTH_IP solve the problem? >> >> In addition, if SOUTH_TO_NORTH_IP is a user defined IP, I am not sure how >> would it work, because ip4.dst is the dst IP from packet header. Comparing >> it with SOUTH_TO_NORTH_IP would just result in mismatch, unless all >> south-to-north traffic really has this IP as destination (I guess that's >> not the case). >> >> >>> In policies/ACL you will need to make sure the priorities are set >>>> properly to achieve the default-route behavior. >>>> >>> >>> Yes, so this is currently done, as described above. >>> >>> On Wed, Sep 16, 2020 at 6:35 PM Han Zhou <hz...@ovn.org> wrote: >>> >>>> >>>> >>>> On Wed, Sep 16, 2020 at 5:42 AM Alexander Constantinescu < >>>> acons...@redhat.com> wrote: >>>> > >>>> > Hi >>>> > >>>> > I was wondering if anybody is aware of an IP address signifying >>>> "external IP destinations"? >>>> > >>>> > Currently in OVN we can use the IP address 0.0.0.0/0 for match >>>> expressions in logical routing policies / ACLs when we want to specify a >>>> source or destination IP equating to the pseudo term: "all IP >>>> addresses",ex: 'match="ip4.src == 10.244.2.5/32 && ip4.dst ==0.0.0.0/0 >>>> "' >>>> > >>>> In this example it is equivalent to just "ip4.src == 10.244.2.5/32"'. >>>> >>>> > Essentially what I would need to do for an OVN-Kubernetes feature is >>>> specify such a match condition for south -> north traffic, i.e when the >>>> destination IP address is external to the cluster, and most likely >>>> "unknown" to OVN. Thus, when OVN does not know how to route it within the >>>> OVN network topology and has no choice except sending it out the default >>>> route. >>>> > >>>> > Do we have such an IP address in OVN/OVS? Would it be feasible to >>>> introduce, in case there is none? >>>> > >>>> We don't have such a special IP except 0.0.0.0/0. If you have a >>>> default route, anything "unknown" would just hit the default route, right? >>>> Why would you need another IP for this purpose? In >>>> logical_router_static_route the priority is based on prefix length. In >>>> policies/ACL you will need to make sure the priorities are set properly to >>>> achieve the default-route behavior. >>>> >>>> Thanks, >>>> Han >>>> >>>> > Thanks in advance! >>>> > >>>> > -- >>>> > >>>> > Best regards, >>>> > >>>> > >>>> > Alexander Constantinescu >>>> > >>>> > Software Engineer, Openshift SDN >>>> > >>>> > Red Hat >>>> > >>>> > acons...@redhat.com >>>> > >>>> > _______________________________________________ >>>> > discuss mailing list >>>> > disc...@openvswitch.org >>>> > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss >>>> >>> >>> >>> -- >>> >>> Best regards, >>> >>> >>> Alexander Constantinescu >>> >>> Software Engineer, Openshift SDN >>> >>> Red Hat <https://www.redhat.com/> >>> >>> acons...@redhat.com >>> <https://www.redhat.com/> >>> >> > > -- > > Best regards, > > > Alexander Constantinescu > > Software Engineer, Openshift SDN > > Red Hat <https://www.redhat.com/> > > acons...@redhat.com > <https://www.redhat.com/> >
_______________________________________________ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
