> On May 11, 2016, at 10:45 PM, Darrell Ball <dlu...@gmail.com> wrote:
>
>
>
>> On Wed, May 11, 2016 at 8:51 PM, Guru Shetty <guru....@gmail.com> wrote:
>>
>>
>>
>>
>> > On May 11, 2016, at 8:45 PM, Darrell Ball <dlu...@gmail.com> wrote:
>> >
>> >> On Wed, May 11, 2016 at 4:42 PM, Guru Shetty <g...@ovn.org> wrote:
>> >>
>> >>
>> >>>
>> >>> Some reasons why having a “transit LS” is “undesirable” is:
>> >>>
>> >>> 1) 1) It creates additional requirements at the CMS layer for setting
>> >>> up networks; i.e. additional programming is required at the OVN
>> >>> northbound
>> >>> interface for the special transit LSs, interactions with the logical
>> >>> router
>> >>> peers.
>> >>
>> >> Agreed that there is additional work needed for the CMS plugin. That work
>> >> is needed even if it is just peering as they need to convert one router in
>> >> to two in OVN (unless OVN automatically makes this split)
>> >
>> > The work to coordinate 2 logical routers and one special LS is more and
>> > also more complicated than
>> > to coordinate 2 logical routers.
>> >
>> >
>> >>
>> >>
>> >>>
>> >>> In cases where some NSX products do this, it is hidden from the user, as
>> >>> one would minimally expect.
>> >>>
>> >>> 2) 2) From OVN POV, it adds an additional OVN datapath to all
>> >>> processing to the packet path and programming/processing for that
>> >>> datapath.
>> >>>
>> >>> because you have
>> >>>
>> >>> R1<->Transit LS<->R2
>> >>>
>> >>> vs
>> >>>
>> >>> R1<->R2
>> >>
>> >> Agreed that there is an additional datapath.
>> >>
>> >>
>> >>>
>> >>> 3) 3) You have to coordinate the transit LS subnet to handle all
>> >>> addresses in this same subnet for all the logical routers and all their
>> >>> transit LS peers.
>> >>
>> >> I don't understand what you mean. If a user uses one gateway, a transit LS
>> >> only gets connected by 2 routers.
>> >> Other routers get their own transit LS.
>> >
>> >
>> > Each group of logical routers communicating has it own Transit LS.
>> >
>> > Take an example with one gateway router and 1000 distributed logical
>> > routers for 1000 tenants/hv,
>> > connecting 1000 HVs for now.
>> > Lets assume each distributed logical router only talks to the gateway
>> > router.
>>
>> That is a wrong assumption. Each tenant has his own gateway router (or more)
>
> Its less of an assumption but more of an example for illustrative purposes;
> but its good that you
> mention it.
I think one of the main discussion points was needing thousands of arp flows
and thousands of subnets, and it was on an incorrect logical topology, I am
glad that it is not an issue any more.
>
> The DR<->GR direct connection approach as well as the transit LS approach can
> re-use private
> IP pools across internal distributed logical routers, which amount to VRF
> contexts for tenants networks.
>
> The Transit LS approach does not scale due to the large number of distributed
> datapaths required and
> other high special flow requirements. It has more complex and higher
> subnetting requirements. In addition, there is greater complexity for
> northbound management.
Okay, now to summarize from my understanding:
* A transit LS uses one internal subnet to connect multiple GR with one DR
whereas direct multiple GR to one DR via peering uses multiple internal
subnets.
* A transit LS uses an additional logical datapath (split between 2 machines
via tunnel) per logical topology which is a disadvantage as it involves going
through an additional egress or ingress openflow pipeline in one host.
* A transit LS lets you split a DR and GR in such a way that the packet
entering physical gateway gets into egress pipeline of a switch and can be made
to renter the ingress pipeline of a router making it easier to apply stateful
policies as packets always enter ingress pipeline of a router in all directions
(NS, SN and east west)
* The general ability to connect multiple router to a switch (which this patch
is about) also lets you connect your physical interface of your physical
gateway connected to a physical topology to a LS in ovn which inturn is
connected to multiple GRs. Each GR will have floating ips and will respond to
ARPs for those floating IPs.
Overall, Though I see the possibility of implementing direct GR to DR
connections via peering, it feels right now that it will be additional work for
not a lot of added benefits.
>
>
>
>
>
>
>>
>>
>> > So thats 1000 Transit LSs.
>> > -> 1001 addresses per subnet for each of 1000 subnets (1 for each Transit
>> > LS) ?
>> >
>> >
>> >
>> >
>> >>
>> >>
>> >>>
>> >>> 4) 4) Seems like L3 load balancing, ECMP, would be more complicated at
>> >>> best.
>> >>>
>> >>> 5) 5) 1000s of additional arp resolve flows rules are needed in
>> >>> normal
>> >>> cases in addition to added burden of the special transit LS others flows.
>> >>
>> >> I don't understand why that would be the case.
>> >
>> >
>> > Each Transit LS creates an arp resolve flow for each peer router port.
>> > Lets say we have 1000 HVs, each HV has 1000 tenants.
>> > Lets says we have 1000 distributed logical routers, minimally 1 per tenant.
>> > For simplicity, lets assume the 1000 distributed logical routers are
>> > connected to a single lonely gateway (GR).
>> >
>> > How many Transit LS (i.e. extra) distributed datapaths ?
>> > 1000 ?
>> >
>> > How many Transit LS router type logical ports in total across all 1000
>> > Transit LSs would you
>> > expect with this 1000 HV network ?
>> > 1001 per Transit LS * 1000 Transit LSs = 1,001,000
>> > Each of these requires a special arp resolve flow, as well - 1,001,000.
>> >
>> > For distributed logical router to gateway connections via Transit LSs, we
>> > have
>> > 1001 address per subnet for each of 1000 subnets = 1,001,000 addresses
>> > but again we need 1001 addresses per subnet !
>> >
>> >
>> >
>> >>
>> >>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>>
>> >>>>
>> >>>>> This is usually only done if there is existing switching equipment than
>> >>>>> must be traversed
>> >>>>> But in the case, we are just dealing with software where we have total
>> >>>>> flexibility.
>> >>>>
>> >>>> From OpenStack perspective, each tenant gets a router and multiple
>> >>>> switches with different subnets. The idea is that the OpenStack network
>> >>>> plugin, can at best split this single tenant router into 2 with a
>> >>> switch in
>> >>>> between on a subnet that neutron does not use. Taking the same logic
>> >>>> forward for k8s, I can support multiple gateways.
>> >>>
>> >>> As far as I have heard from Openstack folks, each tenant can create
>> >>> multiple logical routers; they can even overlap subnets if they wanted.
>> >>> So,
>> >>> I am not sure the above assumption holds true.
>> >>
>> >> A tenant can create multiple logical routers. But a connected logical
>> >> topology can have one router connected to the gateway, atleast that is the
>> >> common use case.
>> >
>> > hmm, I thought there was some kind of limitation that required creating
>> > Transit LSs,
>> > but there seems no such limitation ? I guess we need to dig more on this
>> > one.
>> >
>> >
>> >
>> >>
>> >>>
>> >>>
>> >>>>
>> >>>> Connecting multiple routers to each other without a switch makes it a
>> >>> pain
>> >>>> to remember the interconnections and create multiple subnets (which may
>> >>>> simply not be available for a networking backend for internal use).
>> >>>
>> >>> Well, just looking at the 10.x.x.x private IP range, there are more than
>> >>> 16
>> >>> million IP addresses
>> >>>
>> >>> Point to point direct router links, Rx<->Ry subnets, can be /31. In the
>> >>> most extreme case, maybe 20,000 addresses would be used. I don’t think
>> >>> there should be a subnet depletion problem here, unless it’s a contrived
>> >>> misconfiguration.
>> >>
>> >> Who is going to manage all the interconnections? As far as I see a transit
>> >> LS is much more simpler.
>> >
>> >
>> > Lets compare.
>> >
>> > To be consistent with the previous example, I described earlier, lets again
>> > take the 1000 HV
>> > example, using 1000 distributed logical routers for 1000 tenants,
>> > connected to a single and lonely gateway.
>> >
>> > For distributed logical router to gateway connections only, we have
>> > for direct DR<->GR connections:
>> >
>> > 2000 subnet address per distributed logical router * 1000 distributed
>> > logical routers
>> > = 2,000,000 vs 1,001,000 for Transit LSs usage; same magnitude,
>> > but we only need 2 addresses per subnet compared to 1001 addresses per
>> > subnet for Transit LS
>> > usage.
>> >
>> > There is no subnet depletion problem with direct DR<->GR connections, but
>> > there may be
>> > with DR<->Transit LS<->GR.
>> >
>> > The question is who is going to manage 1000 subnets needing 1001 addresses
>> > each, using Transit LSs.
>> >
>> >
>> >
>> >>
>> >>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>>
>> >>>>
>> >>>>>
>> >>>>>
>> >>>>>>
>> >>>>>> With the above goal in mind, this commit gives the general
>> >>>>>> ability to connect multiple routers via a switch.
>> >>>>>>
>> >>>>>> Signed-off-by: Gurucharan Shetty <g...@ovn.org>
>> >>>>>> ---
>> >>>>>> This needs the following 2 commits under review to
>> >>>>>> first go in.
>> >>>>>> 1. ofproto-dpif: Rename "recurse" to "indentation".
>> >>>>>> 2. ofproto-dpif: Do not count resubmit to later tables against limit.
>> >>>>>> ---
>> >>>>>> ovn/controller/lflow.c | 19 ++--
>> >>>>>> ovn/northd/ovn-northd.c | 56 ++++++++++-
>> >>>>>> ovn/ovn-nb.xml | 7 --
>> >>>>>> tests/ovn.at | 236
>> >>>>>> +++++++++++++++++++++++++++++++++++++++++++++++
>> >>>>>> 4 files changed, 299 insertions(+), 19 deletions(-)
>> >>>>>>
>> >>>>>> diff --git a/ovn/controller/lflow.c b/ovn/controller/lflow.c
>> >>>>>> index 96b7c66..efc427d 100644
>> >>>>>> --- a/ovn/controller/lflow.c
>> >>>>>> +++ b/ovn/controller/lflow.c
>> >>>>>> @@ -215,15 +215,15 @@ add_logical_flows(struct controller_ctx *ctx,
>> >>>>>> const struct lport_index *lports,
>> >>>>>> if (is_switch(ldp)) {
>> >>>>>> /* For a logical switch datapath, local_datapaths tells
>> >>> us
>> >>>>>> if there
>> >>>>>> * are any local ports for this datapath. If not, we can
>> >>>>>> skip
>> >>>>>> - * processing logical flows if the flow belongs to egress
>> >>>>>> pipeline
>> >>>>>> - * or if that logical switch datapath is not patched to
>> >>> any
>> >>>>>> logical
>> >>>>>> - * router.
>> >>>>>> + * processing logical flows if that logical switch
>> >>> datapath
>> >>>>>> is not
>> >>>>>> + * patched to any logical router.
>> >>>>>> *
>> >>>>>> - * Otherwise, we still need the ingress pipeline because
>> >>>>>> even if
>> >>>>>> - * there are no local ports, we still may need to execute
>> >>>>>> the ingress
>> >>>>>> - * pipeline after a packet leaves a logical router.
>> >>> Further
>> >>>>>> - * optimization is possible, but not based on what we
>> >>> know
>> >>>>>> with
>> >>>>>> - * local_datapaths right now.
>> >>>>>> + * Otherwise, we still need both ingress and egress
>> >>> pipeline
>> >>>>>> + * because even if there are no local ports, we still may
>> >>>>>> need to
>> >>>>>> + * execute the ingress pipeline after a packet leaves a
>> >>>>>> logical
>> >>>>>> + * router and we need to do egress pipeline for a switch
>> >>>>>> that
>> >>>>>> + * is connected to only routers. Further optimization is
>> >>>>>> possible,
>> >>>>>> + * but not based on what we know with local_datapaths
>> >>> right
>> >>>>>> now.
>> >>>>>> *
>> >>>>>> * A better approach would be a kind of "flood fill"
>> >>>>>> algorithm:
>> >>>>>> *
>> >>>>>> @@ -242,9 +242,6 @@ add_logical_flows(struct controller_ctx *ctx,
>> >>> const
>> >>>>>> struct lport_index *lports,
>> >>>>>> */
>> >>>>>>
>> >>>>>> if (!get_local_datapath(local_datapaths,
>> >>> ldp->tunnel_key)) {
>> >>>>>> - if (!ingress) {
>> >>>>>> - continue;
>> >>>>>> - }
>> >>>>>
>> >>>>> This is removing a previous change that was done for some optimization
>> >>>>> for avoiding
>> >>>>> unnecessary flow creation.
>> >>>>> I am not sure about the process here, but maybe this should be tracked
>> >>> as
>> >>>>> a separate
>> >>>>> patch ?
>> >>>> The above change is needed for this patch to work. The optimization
>> >>> above
>> >>>> assumes that a switch always has atleast one real physical endpoint.
>> >>> With
>> >>>> this change, since a switch can only be connected to routers, we need to
>> >>>> remove the optimization. The optimization itself will need more careful
>> >>>> consideration and with more information provided to ovn-controller, it
>> >>> can
>> >>>> ideally be improved, but I would ideally not want l3 gateway work
>> >>> delayed
>> >>>> because of it.
>> >>> My point was that this code change removes a previous optimization. This
>> >>> leads to 2 questions – is it worth removing the optimization for the new
>> >>> code as is and is it worth adding back optimizations with the added
>> >>> complexity of the new code.
>> >>>
>> >>>
>> >>>
>> >>>>
>> >>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>> if (!get_patched_datapath(patched_datapaths,
>> >>>>>> ldp->tunnel_key)) {
>> >>>>>> continue;
>> >>>>>> diff --git a/ovn/northd/ovn-northd.c b/ovn/northd/ovn-northd.c
>> >>>>>> index 9e03606..3a29770 100644
>> >>>>>> --- a/ovn/northd/ovn-northd.c
>> >>>>>> +++ b/ovn/northd/ovn-northd.c
>> >>>>>> @@ -2110,7 +2110,13 @@ build_lrouter_flows(struct hmap *datapaths,
>> >>>>>> struct hmap *ports,
>> >>>>>> free(actions);
>> >>>>>> free(match);
>> >>>>>> }
>> >>>>>> - } else if (op->od->n_router_ports) {
>> >>>>>> + } else if (op->od->n_router_ports && strcmp(op->nbs->type,
>> >>>>>> "router")) {
>> >>>>>> + /* This is a logical switch port that backs a VM or a
>> >>>>>> container.
>> >>>>>> + * Extract its addresses. For each of the address, go
>> >>>>>> through all
>> >>>>>> + * the router ports attached to the switch (to which this
>> >>>>>> port
>> >>>>>> + * connects) and if the address in question is reachable
>> >>>>>> from the
>> >>>>>> + * router port, add an ARP entry in that router's
>> >>> pipeline.
>> >>>>>> */
>> >>>>>> +
>> >>>>>> for (size_t i = 0; i < op->nbs->n_addresses; i++) {
>> >>>>>> struct lport_addresses laddrs;
>> >>>>>> if (!extract_lport_addresses(op->nbs->addresses[i],
>> >>>>>> &laddrs,
>> >>>>>> @@ -2158,8 +2164,56 @@ build_lrouter_flows(struct hmap *datapaths,
>> >>>>>> struct hmap *ports,
>> >>>>>>
>> >>>>>> free(laddrs.ipv4_addrs);
>> >>>>>> }
>> >>>>>> + } else if (!strcmp(op->nbs->type, "router")) {
>> >>>>>> + /* This is a logical switch port that connects to a
>> >>> router.
>> >>>>>> */
>> >>>>>> +
>> >>>>>> + /* The peer of this switch port is the router port for
>> >>> which
>> >>>>>> + * we need to add logical flows such that it can resolve
>> >>>>>> + * ARP entries for all the other router ports connected
>> >>> to
>> >>>>>> + * the switch in question. */
>> >>>>>> +
>> >>>>>> + const char *peer_name = smap_get(&op->nbs->options,
>> >>>>>> + "router-port");
>> >>>>>> + if (!peer_name) {
>> >>>>>> + continue;
>> >>>>>> + }
>> >>>>>> +
>> >>>>>> + struct ovn_port *peer = ovn_port_find(ports, peer_name);
>> >>>>>> + if (!peer || !peer->nbr || !peer->ip) {
>> >>>>>> + continue;
>> >>>>>> + }
>> >>>>>> +
>> >>>>>> + for (size_t j = 0; j < op->od->n_router_ports; j++) {
>> >>>>>> + const char *router_port_name = smap_get(
>> >>>>>> +
>> >>>>>> &op->od->router_ports[j]->nbs->options,
>> >>>>>> + "router-port");
>> >>>>>> + struct ovn_port *router_port = ovn_port_find(ports,
>> >>>>>> +
>> >>>>>> router_port_name);
>> >>>>>> + if (!router_port || !router_port->nbr ||
>> >>>>>> !router_port->ip) {
>> >>>>>> + continue;
>> >>>>>> + }
>> >>>>>> +
>> >>>>>> + /* Skip the router port under consideration. */
>> >>>>>> + if (router_port == peer) {
>> >>>>>> + continue;
>> >>>>>> + }
>> >>>>>> +
>> >>>>>> + if (!router_port->ip) {
>> >>>>>> + continue;
>> >>>>>> + }
>> >>>>>> + char *match = xasprintf("outport == %s && reg0 ==
>> >>>>>> "IP_FMT,
>> >>>>>> + peer->json_key,
>> >>>>>> + IP_ARGS(router_port->ip));
>> >>>>>> + char *actions = xasprintf("eth.dst = "ETH_ADDR_FMT";
>> >>>>>> next;",
>> >>>>>> +
>> >>>>>> ETH_ADDR_ARGS(router_port->mac));
>> >>>>>> + ovn_lflow_add(lflows, peer->od,
>> >>> S_ROUTER_IN_ARP_RESOLVE,
>> >>>>>> + 101, match, actions);
>> >>>>>> + free(actions);
>> >>>>>> + free(match);
>> >>>>>> + }
>> >>>>>> }
>> >>>>>> }
>> >>>>>> +
>> >>>>>> HMAP_FOR_EACH (od, key_node, datapaths) {
>> >>>>>> if (!od->nbr) {
>> >>>>>> continue;
>> >>>>>> diff --git a/ovn/ovn-nb.xml b/ovn/ovn-nb.xml
>> >>>>>> index c01455d..d7fd595 100644
>> >>>>>> --- a/ovn/ovn-nb.xml
>> >>>>>> +++ b/ovn/ovn-nb.xml
>> >>>>>> @@ -154,13 +154,6 @@
>> >>>>>> These options apply when <ref column="type"/> is
>> >>>>>> <code>router</code>.
>> >>>>>> </p>
>> >>>>>>
>> >>>>>> - <p>
>> >>>>>> - If a given logical switch has multiple <code>router</code>
>> >>>>>> ports, the
>> >>>>>> - <ref table="Logical_Router_Port"/> rows that they reference
>> >>>>>> must be
>> >>>>>> - all on the same <ref table="Logical_Router"/> (for
>> >>> different
>> >>>>>> - subnets).
>> >>>>>> - </p>
>> >>>>>> -
>> >>>>>> <column name="options" key="router-port">
>> >>>>>> Required. The <ref column="name"/> of the <ref
>> >>>>>> table="Logical_Router_Port"/> to which this logical switch
>> >>>>>> port is
>> >>>>>> diff --git a/tests/ovn.at b/tests/ovn.at
>> >>>>>> index b9d7ada..6961be0 100644
>> >>>>>> --- a/tests/ovn.at
>> >>>>>> +++ b/tests/ovn.at
>> >>>>>> @@ -2545,3 +2545,239 @@ OVS_APP_EXIT_AND_WAIT([ovsdb-server])
>> >>>>>>
>> >>>>>> AT_CLEANUP
>> >>>>>>
>> >>>>>> +AT_SETUP([ovn -- 2 HVs, 3 LRs connected via LS, static routes])
>> >>>>>> +AT_KEYWORDS([ovnstaticroutes])
>> >>>>>> +AT_SKIP_IF([test $HAVE_PYTHON = no])
>> >>>>>> +ovn_start
>> >>>>>> +
>> >>>>>> +# Logical network:
>> >>>>>> +# Three LRs - R1, R2 and R3 that are connected to each other via LS
>> >>>>>> "join"
>> >>>>>> +# in 20.0.0.0/24 network. R1 has switchess foo (192.168.1.0/24)
>> >>>>>> +# connected to it. R2 has alice (172.16.1.0/24) and R3 has bob (
>> >>>>>> 10.32.1.0/24)
>> >>>>>> +# connected to it.
>> >>>>>> +
>> >>>>>> +ovn-nbctl create Logical_Router name=R1
>> >>>>>> +ovn-nbctl create Logical_Router name=R2
>> >>>>>> +ovn-nbctl create Logical_Router name=R3
>> >>>>>> +
>> >>>>>> +ovn-nbctl lswitch-add foo
>> >>>>>> +ovn-nbctl lswitch-add alice
>> >>>>>> +ovn-nbctl lswitch-add bob
>> >>>>>> +ovn-nbctl lswitch-add join
>> >>>>>> +
>> >>>>>> +# Connect foo to R1
>> >>>>>> +ovn-nbctl -- --id=@lrp create Logical_Router_port name=foo \
>> >>>>>> +network=192.168.1.1/24 mac=\"00:00:01:01:02:03\" -- add
>> >>> Logical_Router
>> >>>>>> R1 \
>> >>>>>> +ports @lrp -- lport-add foo rp-foo
>> >>>>>> +
>> >>>>>> +ovn-nbctl set Logical_port rp-foo type=router
>> >>> options:router-port=foo \
>> >>>>>> +addresses=\"00:00:01:01:02:03\"
>> >>>>>> +
>> >>>>>> +# Connect alice to R2
>> >>>>>> +ovn-nbctl -- --id=@lrp create Logical_Router_port name=alice \
>> >>>>>> +network=172.16.1.1/24 mac=\"00:00:02:01:02:03\" -- add
>> >>> Logical_Router
>> >>>>>> R2 \
>> >>>>>> +ports @lrp -- lport-add alice rp-alice
>> >>>>>> +
>> >>>>>> +ovn-nbctl set Logical_port rp-alice type=router
>> >>>>>> options:router-port=alice \
>> >>>>>> +addresses=\"00:00:02:01:02:03\"
>> >>>>>> +
>> >>>>>> +# Connect bob to R3
>> >>>>>> +ovn-nbctl -- --id=@lrp create Logical_Router_port name=bob \
>> >>>>>> +network=10.32.1.1/24 mac=\"00:00:03:01:02:03\" -- add Logical_Router
>> >>>>>> R3 \
>> >>>>>> +ports @lrp -- lport-add bob rp-bob
>> >>>>>> +
>> >>>>>> +ovn-nbctl set Logical_port rp-bob type=router
>> >>> options:router-port=bob \
>> >>>>>> +addresses=\"00:00:03:01:02:03\"
>> >>>>>> +
>> >>>>>> +# Connect R1 to join
>> >>>>>> +ovn-nbctl -- --id=@lrp create Logical_Router_port name=R1_join \
>> >>>>>> +network=20.0.0.1/24 mac=\"00:00:04:01:02:03\" -- add Logical_Router
>> >>> R1
>> >>>>>> \
>> >>>>>> +ports @lrp -- lport-add join r1-join
>> >>>>>> +
>> >>>>>> +ovn-nbctl set Logical_port r1-join type=router
>> >>>>>> options:router-port=R1_join \
>> >>>>>> +addresses='"00:00:04:01:02:03"'
>> >>>>>> +
>> >>>>>> +# Connect R2 to join
>> >>>>>> +ovn-nbctl -- --id=@lrp create Logical_Router_port name=R2_join \
>> >>>>>> +network=20.0.0.2/24 mac=\"00:00:04:01:02:04\" -- add Logical_Router
>> >>> R2
>> >>>>>> \
>> >>>>>> +ports @lrp -- lport-add join r2-join
>> >>>>>> +
>> >>>>>> +ovn-nbctl set Logical_port r2-join type=router
>> >>>>>> options:router-port=R2_join \
>> >>>>>> +addresses='"00:00:04:01:02:04"'
>> >>>>>> +
>> >>>>>> +
>> >>>>>> +# Connect R3 to join
>> >>>>>> +ovn-nbctl -- --id=@lrp create Logical_Router_port name=R3_join \
>> >>>>>> +network=20.0.0.3/24 mac=\"00:00:04:01:02:05\" -- add Logical_Router
>> >>> R3
>> >>>>>> \
>> >>>>>> +ports @lrp -- lport-add join r3-join
>> >>>>>> +
>> >>>>>> +ovn-nbctl set Logical_port r3-join type=router
>> >>>>>> options:router-port=R3_join \
>> >>>>>> +addresses='"00:00:04:01:02:05"'
>> >>>>>> +
>> >>>>>> +#install static routes
>> >>>>>> +ovn-nbctl -- --id=@lrt create Logical_Router_Static_Route \
>> >>>>>> +ip_prefix=172.16.1.0/24 nexthop=20.0.0.2 -- add Logical_Router \
>> >>>>>> +R1 static_routes @lrt
>> >>>>>> +
>> >>>>>> +ovn-nbctl -- --id=@lrt create Logical_Router_Static_Route \
>> >>>>>> +ip_prefix=10.32.1.0/24 nexthop=20.0.0.3 -- add Logical_Router \
>> >>>>>> +R1 static_routes @lrt
>> >>>>>> +
>> >>>>>> +ovn-nbctl -- --id=@lrt create Logical_Router_Static_Route \
>> >>>>>> +ip_prefix=192.168.1.0/24 nexthop=20.0.0.1 -- add Logical_Router \
>> >>>>>> +R2 static_routes @lrt
>> >>>>>> +
>> >>>>>> +ovn-nbctl -- --id=@lrt create Logical_Router_Static_Route \
>> >>>>>> +ip_prefix=10.32.1.0/24 nexthop=20.0.0.3 -- add Logical_Router \
>> >>>>>> +R2 static_routes @lrt
>> >>>>>> +
>> >>>>>> +ovn-nbctl -- --id=@lrt create Logical_Router_Static_Route \
>> >>>>>> +ip_prefix=192.168.1.0/24 nexthop=20.0.0.1 -- add Logical_Router \
>> >>>>>> +R3 static_routes @lrt
>> >>>>>> +
>> >>>>>> +ovn-nbctl -- --id=@lrt create Logical_Router_Static_Route \
>> >>>>>> +ip_prefix=172.16.1.0/24 nexthop=20.0.0.2 -- add Logical_Router \
>> >>>>>> +R3 static_routes @lrt
>> >>>>>> +
>> >>>>>> +# Create logical port foo1 in foo
>> >>>>>> +ovn-nbctl lport-add foo foo1 \
>> >>>>>> +-- lport-set-addresses foo1 "f0:00:00:01:02:03 192.168.1.2"
>> >>>>>> +
>> >>>>>> +# Create logical port alice1 in alice
>> >>>>>> +ovn-nbctl lport-add alice alice1 \
>> >>>>>> +-- lport-set-addresses alice1 "f0:00:00:01:02:04 172.16.1.2"
>> >>>>>> +
>> >>>>>> +# Create logical port bob1 in bob
>> >>>>>> +ovn-nbctl lport-add bob bob1 \
>> >>>>>> +-- lport-set-addresses bob1 "f0:00:00:01:02:05 10.32.1.2"
>> >>>>>> +
>> >>>>>> +# Create two hypervisor and create OVS ports corresponding to logical
>> >>>>>> ports.
>> >>>>>> +net_add n1
>> >>>>>> +
>> >>>>>> +sim_add hv1
>> >>>>>> +as hv1
>> >>>>>> +ovs-vsctl add-br br-phys
>> >>>>>> +ovn_attach n1 br-phys 192.168.0.1
>> >>>>>> +ovs-vsctl -- add-port br-int hv1-vif1 -- \
>> >>>>>> + set interface hv1-vif1 external-ids:iface-id=foo1 \
>> >>>>>> + options:tx_pcap=hv1/vif1-tx.pcap \
>> >>>>>> + options:rxq_pcap=hv1/vif1-rx.pcap \
>> >>>>>> + ofport-request=1
>> >>>>>> +
>> >>>>>> +ovs-vsctl -- add-port br-int hv1-vif2 -- \
>> >>>>>> + set interface hv1-vif2 external-ids:iface-id=alice1 \
>> >>>>>> + options:tx_pcap=hv1/vif2-tx.pcap \
>> >>>>>> + options:rxq_pcap=hv1/vif2-rx.pcap \
>> >>>>>> + ofport-request=2
>> >>>>>> +
>> >>>>>> +sim_add hv2
>> >>>>>> +as hv2
>> >>>>>> +ovs-vsctl add-br br-phys
>> >>>>>> +ovn_attach n1 br-phys 192.168.0.2
>> >>>>>> +ovs-vsctl -- add-port br-int hv2-vif1 -- \
>> >>>>>> + set interface hv2-vif1 external-ids:iface-id=bob1 \
>> >>>>>> + options:tx_pcap=hv2/vif1-tx.pcap \
>> >>>>>> + options:rxq_pcap=hv2/vif1-rx.pcap \
>> >>>>>> + ofport-request=1
>> >>>>>> +
>> >>>>>> +
>> >>>>>> +# Pre-populate the hypervisors' ARP tables so that we don't lose any
>> >>>>>> +# packets for ARP resolution (native tunneling doesn't queue packets
>> >>>>>> +# for ARP resolution).
>> >>>>>> +ovn_populate_arp
>> >>>>>> +
>> >>>>>> +# Allow some time for ovn-northd and ovn-controller to catch up.
>> >>>>>> +# XXX This should be more systematic.
>> >>>>>> +sleep 1
>> >>>>>> +
>> >>>>>> +ip_to_hex() {
>> >>>>>> + printf "%02x%02x%02x%02x" "$@"
>> >>>>>> +}
>> >>>>>> +trim_zeros() {
>> >>>>>> + sed 's/\(00\)\{1,\}$//'
>> >>>>>> +}
>> >>>>>> +
>> >>>>>> +# Send ip packets between foo1 and alice1
>> >>>>>> +src_mac="f00000010203"
>> >>>>>> +dst_mac="000001010203"
>> >>>>>> +src_ip=`ip_to_hex 192 168 1 2`
>> >>>>>> +dst_ip=`ip_to_hex 172 16 1 2`
>> >>> +packet=${dst_mac}${src_mac}08004500001c0000000040110000${src_ip}${dst_ip}0035111100080000
>> >>>>>> +as hv1 ovs-appctl netdev-dummy/receive hv1-vif1 $packet
>> >>>>>> +as hv1 ovs-appctl ofproto/trace br-int in_port=1 $packet
>> >>>>>> +
>> >>>>>> +# Send ip packets between foo1 and bob1
>> >>>>>> +src_mac="f00000010203"
>> >>>>>> +dst_mac="000001010203"
>> >>>>>> +src_ip=`ip_to_hex 192 168 1 2`
>> >>>>>> +dst_ip=`ip_to_hex 10 32 1 2`
>> >>> +packet=${dst_mac}${src_mac}08004500001c0000000040110000${src_ip}${dst_ip}0035111100080000
>> >>>>>> +as hv1 ovs-appctl netdev-dummy/receive hv1-vif1 $packet
>> >>>>>> +
>> >>>>>> +echo "---------NB dump-----"
>> >>>>>> +ovn-nbctl show
>> >>>>>> +echo "---------------------"
>> >>>>>> +ovn-nbctl list logical_router
>> >>>>>> +echo "---------------------"
>> >>>>>> +ovn-nbctl list logical_router_port
>> >>>>>> +echo "---------------------"
>> >>>>>> +
>> >>>>>> +echo "---------SB dump-----"
>> >>>>>> +ovn-sbctl list datapath_binding
>> >>>>>> +echo "---------------------"
>> >>>>>> +ovn-sbctl list port_binding
>> >>>>>> +echo "---------------------"
>> >>>>>> +ovn-sbctl dump-flows
>> >>>>>> +echo "---------------------"
>> >>>>>> +
>> >>>>>> +echo "------ hv1 dump ----------"
>> >>>>>> +as hv1 ovs-ofctl show br-int
>> >>>>>> +as hv1 ovs-ofctl dump-flows br-int
>> >>>>>> +echo "------ hv2 dump ----------"
>> >>>>>> +as hv2 ovs-ofctl show br-int
>> >>>>>> +as hv2 ovs-ofctl dump-flows br-int
>> >>>>>> +echo "----------------------------"
>> >>>>>> +
>> >>>>>> +# Packet to Expect at bob1
>> >>>>>> +src_mac="000003010203"
>> >>>>>> +dst_mac="f00000010205"
>> >>>>>> +src_ip=`ip_to_hex 192 168 1 2`
>> >>>>>> +dst_ip=`ip_to_hex 10 32 1 2`
>> >>> +expected=${dst_mac}${src_mac}08004500001c000000003e110200${src_ip}${dst_ip}0035111100080000
>> >>>>>> +
>> >>>>>> +$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv2/vif1-tx.pcap |
>> >>>>>> trim_zeros > received.packets
>> >>>>>> +echo $expected | trim_zeros > expout
>> >>>>>> +AT_CHECK([cat received.packets], [0], [expout])
>> >>>>>> +
>> >>>>>> +# Packet to Expect at alice1
>> >>>>>> +src_mac="000002010203"
>> >>>>>> +dst_mac="f00000010204"
>> >>>>>> +src_ip=`ip_to_hex 192 168 1 2`
>> >>>>>> +dst_ip=`ip_to_hex 172 16 1 2`
>> >>> +expected=${dst_mac}${src_mac}08004500001c000000003e110200${src_ip}${dst_ip}0035111100080000
>> >>>>>> +
>> >>>>>> +$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/vif2-tx.pcap |
>> >>>>>> trim_zeros > received1.packets
>> >>>>>> +echo $expected | trim_zeros > expout
>> >>>>>> +AT_CHECK([cat received1.packets], [0], [expout])
>> >>>>>> +
>> >>>>>> +for sim in hv1 hv2; do
>> >>>>>> + as $sim
>> >>>>>> + OVS_APP_EXIT_AND_WAIT([ovn-controller])
>> >>>>>> + OVS_APP_EXIT_AND_WAIT([ovs-vswitchd])
>> >>>>>> + OVS_APP_EXIT_AND_WAIT([ovsdb-server])
>> >>>>>> +done
>> >>>>>> +
>> >>>>>> +as ovn-sb
>> >>>>>> +OVS_APP_EXIT_AND_WAIT([ovsdb-server])
>> >>>>>> +
>> >>>>>> +as ovn-nb
>> >>>>>> +OVS_APP_EXIT_AND_WAIT([ovsdb-server])
>> >>>>>> +
>> >>>>>> +as northd
>> >>>>>> +OVS_APP_EXIT_AND_WAIT([ovn-northd])
>> >>>>>> +
>> >>>>>> +as main
>> >>>>>> +OVS_APP_EXIT_AND_WAIT([ovs-vswitchd])
>> >>>>>> +OVS_APP_EXIT_AND_WAIT([ovsdb-server])
>> >>>>>> +
>> >>>>>> +AT_CLEANUP
>> >>>>>> --
>> >>>>>> 1.7.9.5
>> >>>>>>
>> >>>>>> _______________________________________________
>> >>>>>> dev mailing list
>> >>>>>> dev@openvswitch.org
>> >>>>>> http://openvswitch.org/mailman/listinfo/dev
>> >>> _______________________________________________
>> >>> dev mailing list
>> >>> dev@openvswitch.org
>> >>> http://openvswitch.org/mailman/listinfo/dev
>> > _______________________________________________
>> > dev mailing list
>> > dev@openvswitch.org
>> > http://openvswitch.org/mailman/listinfo/dev
>
_______________________________________________
dev mailing list
dev@openvswitch.org
http://openvswitch.org/mailman/listinfo/dev
