Hi Yves, inline > -----Original Message----- > From: [email protected] [mailto:[email protected]] On Behalf Of > Yves Hertoghs (yhertogh) > Sent: Monday, October 28, 2013 6:39 PM > To: Xialiang (Frank) > Cc: 'Thomas Narten'; [email protected]; [email protected]; Lucy yong; > Xuxiaohu; Larry Kreeger (kreeger) > Subject: Re: [nvo3] Distributed Gateways [was Re: NVO3 Architecture > document] > > Frank, > > inline > > On 28/10/13 03:49, "Xialiang (Frank)" <[email protected]> wrote: > > >Hi Yves, > >Thanks for your response. > >Please see my response with [Frank]. > > > >B.R. > >Frank > > > >> -----Original Message----- > >> From: [email protected] [mailto:[email protected]] On Behalf > >> Of Yves Hertoghs (yhertogh) > >> Sent: Saturday, October 26, 2013 6:12 PM > >> To: Xialiang (C); Lizhong Jin > >> Cc: 'Thomas Narten'; [email protected]; [email protected]; Lucy yong; > >> Xuxiaohu; Larry Kreeger (kreeger) > >> Subject: Re: [nvo3] Distributed Gateways [was Re: NVO3 Architecture > >> document] > >> > >> Inline. > >> > >> From: "Xialiang (C)" <[email protected]> > >> Date: Saturday 26 October 2013 11:42 > >> To: Yves Hertoghs <[email protected]>, Lizhong Jin > >> <[email protected]> > >> Cc: 'Thomas Narten' <[email protected]>, "[email protected]" > >> <[email protected]>, "[email protected]" <[email protected]>, Lucy yong > >> <[email protected]>, Xuxiaohu <[email protected]>, "Larry > >> Kreeger (kreeger)" <[email protected]> > >> Subject: RE: [nvo3] Distributed Gateways [was Re: NVO3 Architecture > >> document] > >> > >> > >> >Hi all, > >> >Distributed Gateway is a very interesting topic. It would also be > >> >valuable to solve some challenge introduced by network virtualization. > >> > > >> >To support the intra/inter-subnet connections through the same or > >> >different NVEs in one tenant, the Distributed Gateway should > >> >implement the bridging function and routing function at the same time. > >> > > >> >But, there are still some points need to be studied and solved. > >> >Below are some examples (NVE1 and NVE2 are all Distributed Gateway. > >> >Hosts of subnet > >> >1.1.1.0/24 and 2.2.2.0/24 are attached to them): > >> > ------- / > >> > ------- ------ / > >> > // \\ / > >> > // \\ / > >> > // \\ / > >> > / \ / > >> > / +--------+ +-------+\/Internet > >> > / | | | |/\ > >> > / | CORE |------+Gateway/ \ > >> > / | | | | \ > >> > | +--/----\* +-------+ | > >> > | // \\ | > >> >| // \\ | > >> >| // \\ | > >> >| +----/----+ +----\----+ | > >> >| | | | | | > >> >| | NVE1 | NVE2 | | > >> >| | / \ | | / \ | | > >> >| +-/-----\-+ +-/-----\-+ | > >> >| / \ / \ | > >> >| / \ / \ | > >> >| --- ---- / \ | > >> > | // \\ / \ / -- \ --- | > >> > | |1.1.1.0| | | // \\ / \ | > >> > \ | | |2.2.2.0 | | 2.2.2.0/ > >> > \ \\ // \ / |1.1.1.0 \ // > >> > \ --- ---- | | ---/ > >> > \ \\ // / > >> > \ -- / > >> > \\ // > >> > \\ // > >> > \\ // > >> > ------- ------- > >> > > >> >Problems 1: If we configure the same gateway address (i.e. 1.1.1.1) > >> >for the two subnets--1.1.1.0/24 attached to NVE1 and NVE2, and there > >> >is intra-subnet connection between them, it will result in an > >> >gateway address collision in one subnet. If we configure different > >> >gateway address for same subnet attached to different NVEs to solve > >> >the address collision problem, it will bring a huge configuration > >> >workload and waste of address space, and also it will limit the VM > mobility. > >> > >> Why ? You are thinking that the default gateway has subnets > >>configured, feeding into its routing table. It doesn't. It learns > >>all hosts as host-routes, with optional aggregation in case the > >>host-routing table can be aggregated on bit boundaries. Host might > >>have mask/prefix-lengths configured, as that is how they are > >>operating today, but in reality these denote an address-pool rather > >>than a subnet/prefix. For IP traffic these two parts of a subnet are > >>not interconnected by a L2 link. Surely there might be a L2 overlay > >>spanning both NVE's, but that overlay does not switch IP packets. > >> > >[Frank]: Maybe my description is not so clear. What I mean is every > >host in the subnet configures its gateway for IP forwarding. For the > >same subnet distributed in different NVEs, Same gateway address located > >in different NVEs can exist. When these gateways in different NVEs send > >gratuitous ARP messages in the subnet, or several VRRP groups in same > >subnet send VRRP messages in the subnet, the IP address collision > >between gateways will be detected. How to solve this kind of problem? > In the topology drawn by you i dont see any L2 interconnection between > NVE1 and NVE2. You dont even need VRRP is this topology. You are again > assuming a distributed subnets equals one L2 domain. It doesn't need to be > like this, as long as all hosts are treated as host-routes on the NVEs i.e. > The > NVE is not configured for a certain TS prefix on an interface. > It dynamically installs the host-routes based on Tses registering (via VQP or > some other means) and pushes them to the NVA. The subnet is no longer a > subnet, it is merely an address-pool. > [Frank]: Your can treat all the hosts as routes. But gateways are still needed for all the hosts for inter-subnet connections. And these gateways are configured and located in NVEs and within same L2 domain of hosts. That's why I think the IP address collision between gateways exists. In addition, I think there can be several options of how to treat and handle hosts' traffic, just like the previous discussion in this thread. For example, we have Non-IP traffic and intra-subnet traffic. I think switching function is suitable for them.
> > > > > >> > >> > > >> >Problem 2: How to aggregate or decrease the address number learned > >> >by NVEs or hosts? > >> In any case you will perform the same or better than when learning > >>MAC-addresses. With a MAC address namespace you cannot aggregate. > >> With an IP namespace and host-route learning there is at least a > >>change of aggregation, either at the NVE, or higher up in the data > >>centre. And actually the aggregation function isn't really a part of > >>the NVE, it can be a function of the NVA to aggregate address-space, > >>if a contiguous block of host-routes are all pointing to the same > >>NVE. > >> > >[Frank]: Right, I agree your analysis about address aggregation. > >Actually, what I am concerned is the possibility to decrease or need > >not to learn all host's address by some gateway-like method. > > > >> > > >> >Problem 3: How to deal with multi-tenants? > >> > >> This is orthogonal. The NVE learns about hosts and their IP > >> addresses within the NV instance. I dont see why multi tenancy is a > problem ? > >> > >> > > >> >In addition to above problems, there are maybe more points around > >> >the Distributed Gateway issue. > >> > > >> >I think the distributed gateway will solve those problems. But > >> >before we define any functions of it, we need a problem statement > >> >document for those scenarios firstly. > >> I think the problem statement of how the solution should look like > >>from a user's perspective is already there. A distributed gateway, > >>with a unified > >> L2/L3 NVE on it, is an option to solve those problems. > >> > >[Frank]: It depends on how many problems we find by discussion and come > >to the agreement. We can keep on discussing. > Haven't seen any problem yet, as this model is implemented already. The > biggest problem is that people dont fully understand it, and perhaps we need > a separate draft to discuss this in detail. Draft-hertoghs does explain it, > but > the LISP machinery around it might confuse people i.e the distributed, unified > L2/L3 NVE is orthogonal to the NVE-NVA CP being used. > [Frank] By my understanding, distributed gateways are new functions needed to support in NVE and can work together with NVE-NVA CP well. > Yves > > > >> > > >> >Your comments? > >> > > >> >B.R. > >> >Frank > >> > > >> > > >> >From: [email protected] [mailto:[email protected]] On Behalf > >> >Of Yves Hertoghs (yhertogh) > >> >Sent: Friday, October 25, 2013 11:05 PM > >> >To: Lizhong Jin > >> >Cc: 'Thomas Narten'; [email protected]; [email protected]; Lucy yong; > >> >Xuxiaohu; Larry Kreeger (kreeger) > >> >Subject: Re: [nvo3] Distributed Gateways [was Re: NVO3 Architecture > >> >document] > >> > > >> > > >> > > >> >True, but the unified L2/L3 solution gives you the choice on a per > >> >VNI basis whether you want to do 'enhanced IP forwarding' or > >> >'Ethernet based forwarding'. > >> > > >> > > >> > > >> >I also wouldn't call RDMA over Converged Ethernet a prime use case > >> >for a multi tenant DC. Typically this would be deployed standalone > >> >and across a few ethernet switches. > >> > > >> > > >> > > >> >Yves > >> > > >> > > >> > > >> >From: > >> >Lizhong Jin <[email protected]> > >> >Date: Friday 25 October 2013 16:52 > >> >To: Yves Hertoghs <[email protected]> > >> >Cc: Lucy yong <[email protected]>, "[email protected]" > >> ><[email protected]>, 'Thomas Narten' <[email protected]>, > >> 'Xuxiaohu' > >> ><[email protected]>, "[email protected]" <[email protected]>, "Larry > >> >Kreeger (kreeger)" <[email protected]> > >> >Subject: RE: [nvo3] Distributed Gateways [was Re: NVO3 Architecture > >> >document] > >> > > >> > > >> > > >> >>Yves, > >> >>In that scenario, I agree. But my concern is, is it realistic for a > >> >>datacenter with only IP traffic. I am not sure. But at least as I > >> >>know, if you want to deploy RoCE, there will be non-IP traffic. > >> >> > >> >>Lizhong > >> >> > >> >>From: Yves Hertoghs (yhertogh) [mailto:[email protected]] > >> >> > >> >>Sent: Friday, October 25, 2013 6:07 PM > >> >>To: Lizhong Jin > >> >>Cc: 'Lucy yong'; [email protected]; 'Thomas Narten'; 'Xuxiaohu'; > >> >>[email protected]; Larry Kreeger (kreeger) > >> >>Subject: Re: [nvo3] Distributed Gateways [was Re: NVO3 Architecture > >> >>document] > >> >> > >> >> > >> >> > >> >>Lizhong, > >> >> > >> >> > >> >> > >> >>My main point is that , in the absence of non-IP traffic, there is > >> >>a lot of value of doing IP based forwarding for all traffic, as > >> >>this effectively isolates the failure domains to the L2 domains > >> >>underneath the NVE's. > >> >> > >> >> > >> >> > >> >>Yves > >> >> > >> >> > >> >> > >> >>From: > >> >>Lizhong Jin <[email protected]> > >> >>Date: Friday 25 October 2013 04:33 > >> >>To: Yves Hertoghs <[email protected]> > >> >>Cc: Lucy yong <[email protected]>, "[email protected]" > >> >><[email protected]>, 'Thomas Narten' <[email protected]>, > >> 'Xuxiaohu' > >> >><[email protected]>, "[email protected]" <[email protected]>, "Larry > >> Kreeger > >> >>(kreeger)" <[email protected]> > >> >>Subject: RE: [nvo3] Distributed Gateways [was Re: NVO3 Architecture > >> >>document] > >> >> > >> >> > >> >> > >> >>>See inline. > >> >>> > >> >>>From: Yves Hertoghs (yhertogh) [mailto:[email protected]] > >> >>> > >> >>>Sent: 2013年10月24日 > >> >>> 23:54 > >> >>>To: Lizhong Jin > >> >>>Cc: Lucy yong; [email protected]; Thomas Narten; Xuxiaohu; > >> >>>[email protected]; Larry Kreeger (kreeger) > >> >>>Subject: Re: [nvo3] Distributed Gateways [was Re: NVO3 > >> >>>Architecture document] > >> >>> > >> >>> > >> >>> > >> >>>Inline > >> >>> > >> >>> > >> >>> > >> >>>From: > >> >>>Lizhong Jin <[email protected]> > >> >>>Date: Thursday 24 October 2013 17:06 > >> >>>To: Yves Hertoghs <[email protected]> > >> >>>Cc: Lucy yong <[email protected]>, "[email protected]" > >> >>><[email protected]>, Thomas Narten <[email protected]>, > >> Xuxiaohu > >> >>><[email protected]>, "[email protected]" <[email protected]>, "Larry > >> >>>Kreeger (kreeger)" <[email protected]> > >> >>>Subject: Re: [nvo3] Distributed Gateways [was Re: NVO3 > >> >>>Architecture document] > >> >>> > >> >>> > >> >>> > >> >>>>Hi Yves, > >> >>>>Are you referring the benefits below: > >> >>>>Effectively this allows the 'spread' of > >> >>>> subnets across the Datacenter(s) without leading to network wide > >> >>>> broadcast and associated failure domains, while allowing free > >> >>>> mobility of the end-stations. > >> >>>>MAC based forwarding will also not lead to network wide > >> >>>>broadcast, if all the MAC mapping entries are generated by NVA. > >> >>>> > >> >>>> > >> >>>> > >> >>>Since when is that a requirement for the NVA ? An NVA will store > >> >>>the mappings, not generate it. The mapping generation can be done > >> >>>through orchestration of the NVA, or through the NVE to NVA > >> >>>interactions as a result of the NVE detection new Tenant Systems. > >> >>> > >> >>>Or are you saying that, independent of how the NVA knows about > >> >>>them, tenant broadcast will only get flooded within the scope of the > VNI? > >> >>>If yes, why would this prevent from sending L2 broadcasts to all > >> >>>participating NVEs, and attached TS'es? There might be some > >> >>>optimisations in the underlay to make this work 'better' , but it > >> >>>doesn't stop those ugly L2 broadcast packets from hitting all TS'es. > >> >>>A > >> >>> L2 overlay creates a L2 failure domain, in the context of > >> >>>broadcast and multicast. Nothing you can do about it. > >> >>> > >> >>>[Lizhong] Sorry for the confusion, I mean the MAC mapping of > >> >>>remote will be got from NVA, and MAC mapping of local will be got > >> >>>from NVA or generated by itself. I did not invent new things, and > >> >>>all of that is described in draft-narten-nvo3-arch-01. The NVE > >> >>>will reply the ARP from local TS if corresponding MAC mapping is > >> >>>already existed. For EVPN, broadcast only happen when there is no > >> >>>MAC-IP bindings locally, and that works well. There maybe some > >> >>>benefits existed from the multicast view, and I should think about it. > >> >>> > >> >>>Thanks > >> >>>Lizhong > >> >>> > >> >>> > >> >>> > >> >>>>I remember this has already been discussed on NVO3 list last year. > >> >>>>The only benefit I could see is, for an IP only data center, you > >> >>>>only need to have IP forwarding. > >> >>>>Regards > >> >>>>Lizhong > >> >>>>On Oct 24, 2013 7:24 PM, "Yves Hertoghs (yhertogh)" > >> >>>><[email protected]> wrote: > >> >>>>> > >> >>>>> inline > >> >>>>> > >> >>>>> From: Lizhong Jin <[email protected]> > >> >>>>> Date: Thursday 24 October 2013 03:54 > >> >>>>> To: "Larry Kreeger (kreeger)" <[email protected]>, Xuxiaohu > >> >>>>><[email protected]>, Yves Hertoghs <[email protected]>, > Lucy > >> >>>>>yong <[email protected]>, > >> >>>> Thomas Narten <[email protected]> > >> >>>>> > >> >>>>> Cc: "[email protected]" <[email protected]>, "[email protected]" > >> >>>>><[email protected]> > >> >>>>> Subject: RE: [nvo3] Distributed Gateways [was Re: NVO3 > >> >>>>>Architecture document] > >> >>>>> > >> >>>>>> Hi Larry, > >> >>>>>> > >> >>>>>> 1) A name for the service (lets's pick one, is it hybrid, > >> >>>>>>combined or unified?) > >> >>>>>> > >> >>>>>> [Lizhong] IRB, which have been used before. We could change it > >> >>>>>>to virtual IRB. > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> 2) To address it from a solution point of view (e.g. are > >> >>>>>>packets forwarded based on MAC or based on IP when source and > >> >>>>>>dest are in the same subnet?). > >> >>>>>> > >> >>>>>> [Lizhong] Yes, we should do that, and there are already some > >> >>>>>>drafts there > >> >>>>>>(http://tools.ietf.org/html/draft-jin-nvo3-virb-centralized-00). > >> >>>>>>For IP forwarding for intra subnet traffic: > >> >>>>>> > >> >>>>>> 1. I don’t think it could be wildly accepted in next two or > >> >>>>>>three years to have a pure IP datacenter. Pure Ethernet traffic > >> >>>>>>and MAC forwarding will be there anyway. > >> >>>>>> > >> >>>>>> 2. From the dataplane point, you have to design larger IP > >> >>>>>>routing table, and possibly larger TCAM (for prefix entry), and > >> >>>>>>higher cost consequently. What’s more, should we even design > >> >>>>>>aging for IP entry like MAC entry? > >> >>>>>> > >> >>>>>> 3. From the control plane point, you need to change the > >> >>>>>>mapping entry rule, and may have new interoperable issue with > >> >>>>>>legacy > >> >>>>>>L2/L3 VPN. > >> >>>>>> > >> >>>>>> I don’t see much benefit. Could anyone give me some hints? > >> >>>>> > >> >>>>> [yves] read draft-hertoghs-nvo3-lisp-unified-control-plane > >> >>>>> > >> >>>>> Yves > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> Thanks > >> >>>>>> > >> >>>>>> Lizhong > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> From: Larry Kreeger (kreeger) [mailto:[email protected]] > >> >>>> > >> >>>>>> Sent: 2013年10月24日 > >> >>>> 8:32 > >> >>>>>> To: Xuxiaohu; Lizhong Jin; Yves Hertoghs (yhertogh); Lucy > >> >>>>>>yong; Thomas Narten > >> >>>>>> Cc: [email protected]; > >> >>>>[email protected] <mailto:[email protected]> > >> >>>>>> Subject: Re: 答复: [nvo3] Distributed Gateways [was Re: NVO3 > >> >>>>>>Architecture document] > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> I've now read through this whole thread twice. I see now > >> >>>>>>that there may be some confusion between an L2/L3 service (what > >> >>>>>>tenant's > >> >>>>>>see) and an L2/L3 solution (how NVO3 provides the service). > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> Prior to this thread, I may have agreed that we don't need to > >> >>>>>>define a hybrid/combined/unified L2/L3 "service", but now I'm > >> >>>>>>less sure. Just be be clear, I was not calling so much for a > >> >>>>>>definition of the service, but that we need: > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> 1) A name for the service (lets's pick one, is it hybrid, > >> >>>>>>combined or unified?) > >> >>>>>> > >> >>>>>> 2) To address it from a solution point of view (e.g. are > >> >>>>>>packets forwarded based on MAC or based on IP when source and > >> >>>>>>dest are in the same subnet?). > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> As I stated before, I believe the control plane requirements > >> >>>>>>will be different for a combined service than for a pure L2 or > >> >>>>>>pure L3 service, and that we if don't start specifically > >> >>>>>>calling out the control plane requirements for a combined > >> >>>>>>service (from a > >> >>>> solution point of view), we will have a very low likelihood of > >> >>>>interoperable solutions as each implementer fills in the gaps in > >> >>>>their own way. > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> Does the WG agree that we should do 1) and 2) above? > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> I would also not object if we had a short description in the > >> >>>>>>framework of what the combined service looks like compared to a > >> >>>>>>pure > >> >>>>>>L2 or pure L3 service, just so we can level set for everyone. > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> Thanks, Larry > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> From: Xuxiaohu <[email protected]> > >> >>>>>> Date: Tuesday, October 22, 2013 11:38 PM > >> >>>>>> To: Larry Kreeger <[email protected]>, Lizhong Jin > >> >>>>>><[email protected]>, "Yves Hertoghs (yhertogh)" > >> >>>>>><[email protected]>, Lucy yong <[email protected]>, > >> >>>> Thomas Narten <[email protected]> > >> >>>>>> Cc: "[email protected]" <[email protected]>, "[email protected]" > >> >>>>>><[email protected]> > >> >>>>>> Subject: 答复: [nvo3] Distributed Gateways [was Re: NVO3 > >> >>>>>>Architecture document] > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> 发件人: > >> >>>>[email protected] [mailto:[email protected]] > >> >>>>代表 Larry Kreeger (kreeger) > >> >>>>>> 发送时间: 2013年10月23日 > >> >>>> 11:27 > >> >>>>>> 收件人: Lizhong Jin; Yves Hertoghs (yhertogh); Lucy yong; Thomas > >> >>>>>> Narten > >> >>>>>> 抄送: > >> >>>>[email protected]; > >> >>>>[email protected] <mailto:[email protected]> > >> >>>>>> 主题: Re: [nvo3] Distributed Gateways [was Re: NVO3 > Architecture > >> >>>>>>document] > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> Hi Lizong, > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> Let me give more details on my thinking. > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> A pure L2 service only requires inner MAC address to outer IP > >> >>>>>>address mappings. When a frame is delivered between two TS, the > >> >>>>>>source and dest MAC addresses have not been changed. > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> A pure L3 service only requires inner IP to outer IP address > >> >>>>>>mappings. There is no expectation that two TS are on the same > >> >>>>>>L2 segment, and the TS would always ARP for a router to reach > >> >>>>>>another TS. When a frame is delivered between the two TS, both > >> >>>>>>the > >> >>>> source and dest MAC will have been rewritten. TTL will have > >> >>>>been decremented at least once. > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> I'm expecting a hybrid L2/L3 service to preserve L2 semantics > >> >>>>>>within a subnet. TS ARP for the other TS on its subnet instead > >> >>>>>>of the router. The source and dest MAC are not rewritten. TTL > >> >>>>>>is not decremented. From the perspective of the two TS on the > >> >>>>>>same > >> >>>> subnet, they are connected to the same L2 segment. To achieve > >> >>>>this, the NVEs must behave differently when forwarding IP packets > >> >>>>within the same L2 VN vs between different L2 VNs. For example, > >> >>>>the NVE would need to respond to ARP, not with its own MAC > >> >>>>address, but with the remote TSs MAC address if the TS are in the > same L2 VN. > >> >>>>Furthermore, as I wrote below, there needs to be a mapping of the > >> >>>>set of L2 VNs that belong to the L3 VN. > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> If so, what’s the difference between hybrid L2/L3 service and > >> >>>>>>IRB (Integrated Routing and Bridging)? In addition, I believe > >> >>>>>>the hybrid > >> >>>>>>L2/L3 service you mentioned above is different from what Yves > >> >>>>>>meant by “combined L2/L3”. The former is about bridging > >> >>>> for intra-subnet traffic and routing for inter-subnet traffic > >> >>>>while the latter is about routing for (both intra-subnet and > >> >>>>inter-subnet) IP traffic while bridging for non-IP traffic. > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> Best regards, > >> >>>>>> > >> >>>>>> Xiaohu > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> So, I still think NVEs will need some additional information > >> >>>>>>for a hybrid service, and I think we will need to describe the > >> >>>>>>behavior of a hybrid service beyond a sentence in the framework > >> >>>>>>saying it is possible. Otherwise, I think it is quite likely > >> >>>>>>that > >> >>>> two different hybrid implementations will not interoperate. > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> - Larry > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> From: Lizhong Jin <[email protected]> > >> >>>>>> Date: Tuesday, October 22, 2013 7:31 PM > >> >>>>>> To: Larry Kreeger <[email protected]>, "Yves Hertoghs > (yhertogh)" > >> >>>>>><[email protected]>, Lucy yong <[email protected]>, > Thomas > >> >>>>>>Narten > >> >>>> <[email protected]> > >> >>>>>> Cc: "[email protected]" <[email protected]>, "[email protected]" > >> >>>>>><[email protected]> > >> >>>>>> Subject: RE: [nvo3] Distributed Gateways [was Re: NVO3 > >> >>>>>>Architecture document] > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> > > >> >>>>>> >With regards to the 'new service type' of a combined L2/L3 > >> >>>>>> >NVE, > >> >>>>>> > >> >>>>>>>http://tools.ietf.org/html/draft-hertoghs-nvo3-lisp-controlpla > >> >>>>>>>ne- > >> >>>>>>>uni > >> >>>>>>>fie > >> >>>>>> >d-0 > >> >>>>>> >0 > >> >>>>>> > describes what you are referring to , but i am not sure if > >> >>>>>> >it > >> >>>>>>needs to > >> >>>>>> >be called a new service-type. I see it more as a deployment > >> >>>>>> >choice where you choose to deploy distributed gateways with > >> >>>>>> >uniform mac/IP addressing across the DC, collocate them with > >> >>>>>> >the > >> >>>>>> >L2 overlay, and > >> >>>>>>do > >> >>>>>> >some traffic steering to make sure IP traffic gets sent to > >> >>>>>> >the > >> >>>>>> >L3 overlay (at all times), and non-IP traffic gets sent > >> >>>>>> >across the L2 > >> >>>>>> overlay. > >> >>>>>> > >> >>>>>> LK> I'll give you an example of why I think the protocol > >> >>>>>>requirements will > >> >>>>>> be different for L2 vs L3 vs a combined L2/L3 service. For an > >> >>>>>>L2 VN, the VN needs to be identified (e.g. with a Name or ID). > >> >>>>>>For an L3 VN, it similarly needs to be identified. For a > >> >>>>>>combined > >> >>>>>>L2/L3 service, I think the NVEs need to know both the L3 VN > >> >>>>>>identity (one), and all the L2 VN identities that are part of > >> >>>>>>the > >> >>>>>>L3 VN. When doing distributed L3 forwarding between a TS on > >> >>>>>>one > >> >>>>>>L2 VN to one on another L2 VN, it will need to know not just > >> >>>>>>the mapping of inner to outer address, but the mapping of > >> >>>>>>inner L3 address to destination L2 VN and MAC address (so it > >> >>>>>>can rewrite the MAC and the L2 VN). > >> >>>>>> > >> >>>>>> [Lizhong] I tend to agree with Yves, it is more of a > >> >>>>>>deployment choice. > >> >>>>>> The behavior you described for L3 forwarding between two L2 > >> >>>>>>VPN is more of a standard L3 forwarding function, looking up > >> >>>>>>with destination IP address, then getting the destination MAC, > >> >>>>>>source MAC, VLAN and NVO3 tunnel from the table. The MAC > and > >> >>>>>>VLAN > >> are > >> >>>>>>resolved from the next hop within same L3 VN. We write a > >> combined > >> >>>>>>L2/L3 forwarding for NVO3 last year, just for reference > >> >>>>http://tools.ietf.org/html/draft-jin-nvo3-virb-centralized-00 > >> >>>><http://tools.ietf.org/html/draft-jin-nvo3-virb-centralized-00>. > >> >>>>>> BTW, we may update it soon. > >> >>>>>> > >> >>>>>> Thanks > >> >>>>>> Lizhong > >> >>>>>> > >> >>>>>> > > >> >>>>>> >Yves > >> >>>>>> > > >> >>>>>> >On 21/10/13 10:28, "Larry Kreeger (kreeger)" > >> >>>>>> ><[email protected]> > >> >>>>>>wrote: > >> >>>>>> > > >> >>>>>> >>Lucy, > >> >>>>>> >> > >> >>>>>> >>See inline with LK2>. > >> >>>>>> >> > >> >>>>>> >> - Larry > >> >>>>>> >> > >> >>>>>> >>On 10/20/13 8:20 PM, "Lucy yong" <[email protected]> > >> wrote: > >> >>>>>> >> > >> >>>>>> >>>Hi Larry, > >> >>>>>> >>> > >> >>>>>> >>>Please see inline with [Lucy] > >> >>>>>> >>> > >> >>>>>> >>>-----Original Message----- > >> >>>>>> >>>From: Larry Kreeger (kreeger) [mailto:[email protected]] > >> >>>>>> >>>Sent: Friday, October 18, 2013 6:39 PM > >> >>>>>> >>>To: Lucy yong; Thomas Narten > >> >>>>>> >>>Cc: [email protected] > >> >>>>>> >>>Subject: Re: [nvo3] Distributed Gateways [was Re: NVO3 > >> >>>>>>Architecture > >> >>>>>> >>>document] > >> >>>>>> >>> > >> >>>>>> >>>Lucy, > >> >>>>>> >>> > >> >>>>>> >>>See inline with LK>. > >> >>>>>> >>> > >> >>>>>> >>>Thanks, Larry > >> >>>>>> >>> > >> >>>>>> >>>On 10/18/13 3:21 PM, "Lucy yong" <[email protected]> > >> wrote: > >> >>>>>> >>> > >> >>>>>> >>>>Larry, > >> >>>>>> >>>> > >> >>>>>> >>>>Distributed L3 gateway is very useful and some vendors > >> >>>>>> >>>>already implement that. > >> >>>>>> >>> > >> >>>>>> >>>I agree. > >> >>>>>> >>> > >> >>>>>> >>>>Current framework also includes L2/L3 service as Marc > >> >>>>>> >>>>agrees to > >> >>>>>>add > >> >>>>>> >>>>the text I proposed although we did not explicitly define > >> >>>>>> >>>>as a service type. > >> >>>>>> >>> > >> >>>>>> >>>I think we will need to because the control plane > >> >>>>>> >>>requirements > >> >>>>>>will > >> >>>>>> >>>likely be different for a hybrid service. > >> >>>>>> >>>[Lucy] Do you mean that we should define an l2/L3 service > >> >>>>>> >>>type? I full agree and hope see more people support that too. > >> >>>>>> >>>We can > >> >>>>>>either > >> >>>>>> >>>define it in the existing framework, or in the framework > >> >>>>>> >>>addition draft I submitted while ago. > >> >>>>>> >> > >> >>>>>> >>LK2> Yes, if the group is to seriously address a hybrid > >> >>>>>> >>LK2> L2/L3 > >> >>>>>>service, > >> >>>>>> >>LK2> I > >> >>>>>> >>think we need to define it as we will need to identify its > >> >>>>>> >>requirements independently from a pure L2 or pure L3 service. > >> >>>>>> >> > >> >>>>>> >>> > >> >>>>>> >>>> > >> >>>>>> >>>>Current architecture clear states NVE and NVA roles, i.e. > >> >>>>>> >>>>NVE performs forwarding, and NVA performs routing. > >> >>>>>> >>> > >> >>>>>> >>>I'm not sure what you mean by routing. Are you referring to > >> >>>>>> >>>L3 > >> >>>>>> service? > >> >>>>>> >>>If so, I don't agree. > >> >>>>>> >>>[Lucy] Sorry to make you confuse. The routing here does not > >> >>>>>> >>>mean > >> >>>>>>L3 > >> >>>>>> >>>services. I should state that NVE performs data plan > >> >>>>>> >>>forwarding, > >> >>>>>>NVA > >> >>>>>> >>>performs control plane routing, which, IMO, it is not just > >> >>>>>>simple DB > >> >>>>>> >>>to have inner to outer mappings. NVA may get the routing > >> >>>>>> >>>policy > >> >>>>>>from > >> >>>>>> >>>operators or customer, then interpret the policy, then > >> >>>>>> >>>generates > >> >>>>>>the > >> >>>>>> >>>mapping of tenant/next-hop location and send to NVEs. An > >> >>>>>> >>>NVE > >> >>>>>>receives > >> >>>>>> >>>the mapping in which, if the location is the same as > >> >>>>>> >>>itself, it translates to a tenant/VAP mapping; If not, it > >> >>>>>> >>>installs as an inner/outer mapping. > >> >>>>>> >>>Thus, it works regardless whether sender tenant and > >> >>>>>> >>>destination tenant are one the same NVE or on different NVEs. > >> >>>>>> >>>The mapping > >> >>>>>>between > >> >>>>>> >>>tenant/location is fully controlled under NVA. In this way, > >> >>>>>>operator > >> >>>>>> >>>only needs to input the routing policies to NVA. NVE simply > >> >>>>>>performs > >> >>>>>> >>>the forwarding accordingly. This is also my view about the > >> >>>>>> >>>SDN > >> >>>>>>based > >> >>>>>> >>>architecture. > >> >>>>>> >>> > >> >>>>>> >>>>If NVA is not able to distribute routing policy to NVE at > >> >>>>>> >>>>all, > >> >>>>>>I do > >> >>>>>> >>>>not know how NVA can perform route distribution control? > >> >>>>>> >>> > >> >>>>>> >>>You lost me. Are you referring to a hybrid L2/L3 service? > >> >>>>>> >>>[Lucy] Let me know above explanation help or not. No, not > >> >>>>>>particulate > >> >>>>>> >>>to > >> >>>>>> >>>L2/L3 service but that certainly applies to L2/L3 service. > >> >>>>>> >> > >> >>>>>> >>LK2> OK, if it isn't specific to L2/L3 service, than I'd > >> >>>>>> >>LK2> rather > >> >>>>>>leave > >> >>>>>> >>LK2> it > >> >>>>>> >>out of the discussion for now. And yes, the above helped me > >> >>>>>> >>understand what you meant. The term "routing" is very > >> overloaded. > >> >>>>>> >> > >> >>>>>> >>> > >> >>>>>> >>>> If NVA only supports simple inner to outer mappings, how > >> >>>>>> >>>> can > >> >>>>>>NVE > >> >>>>>> >>>>get information to perform local forwarding? > >> >>>>>> >>> > >> >>>>>> >>>Inner to outer mapping resolution works fine for pure L2 or > >> >>>>>> >>>pure > >> >>>>>>L3 > >> >>>>>> >>>service. Local forwarding doesnšt need mappings, the local > >> >>>>>> >>>NVE > >> >>>>>>knows > >> >>>>>> >>>what VAPs the TSI are connected to. > >> >>>>>> >>>[Lucy] does pure L3 service means an L3VPN w/o any policy? > >> >>>>>> >>>For > >> >>>>>>an L3 > >> >>>>>> >>>service, you can implement it w/o policy or w policy. > >> >>>>>> >> > >> >>>>>> >>LK2> Yes, I was thinking of pure L3 service without adding > >>policy. > >> >>>>>> >>LK2> I'm > >> >>>>>> >>pretty sure you need more than just inner to outer mappings > >> >>>>>> >>to implement policy. > >> >>>>>> >> > >> >>>>>> >>>IMO: NVE is not the entity to enforce the policy. NVA is > >> >>>>>> >>>the > >> >>>>>>entity > >> >>>>>> >>>to enforce the policy regardless the tenant locations. > >> >>>>>> >>>Again, > >> >>>>>>Network > >> >>>>>> >>>virtualization overlay has to address how to support the > >> >>>>>>policies and > >> >>>>>> >>>tenant mobility. IMO: current architecture draft is vague > >> >>>>>> >>>in or > >> >>>>>>lacks > >> >>>>>> >>>of describing this but it is important to architect this > >> >>>>>> >>>when > >> >>>>>>having > >> >>>>>> >>>data plan and control plane separated on different entities. > >> >>>>>> >> > >> >>>>>> >>LK2> I'm not sure if I agree with you about the NVE not > >> >>>>>> >>LK2> being the entity > >> >>>>>> >>to enforce policy, but I think it is something better > >> >>>>>> >>discussed > >> >>>>>>in a > >> >>>>>> >>conversation (not email). > >> >>>>>> >> > >> >>>>>> >>> > >> >>>>>> >>>Thanks, > >> >>>>>> >>>Lucy > >> >>>>>> >>> > >> >>>>>> >>>> > >> >>>>>> >>>>Thanks, > >> >>>>>> >>>>Lucy > >> >>>>>> >>>> > >> >>>>>> >>>>-----Original Message----- > >> >>>>>> >>>>From: Larry Kreeger (kreeger) [mailto:[email protected]] > >> >>>>>> >>>>Sent: Friday, October 18, 2013 5:00 PM > >> >>>>>> >>>>To: Thomas Narten; Lucy yong > >> >>>>>> >>>>Cc: [email protected] > >> >>>>>> >>>>Subject: Re: [nvo3] Distributed Gateways [was Re: NVO3 > >> >>>>>>Architecture > >> >>>>>> >>>>document] > >> >>>>>> >>>> > >> >>>>>> >>>>Hi Thomas and Lucy, > >> >>>>>> >>>> > >> >>>>>> >>>>The WG needs to think hard about this one. > >> >>>>>> >>>> > >> >>>>>> >>>>Support of a distributed L3 gateway function between L2 > >> >>>>>> >>>>VNs is a significant increase in scope of the NVA, and the > >> >>>>>> >>>>NVE to NVA > >> >>>>>>protocol. > >> >>>>>> >>>>Where we had previously stated L2 service or L3 service > >> >>>>>> >>>>and > >> >>>>>>pretty > >> >>>>>> >>>>much left a combined L2/L3 service as an exercise for the > >> >>>>>>reader, we > >> >>>>>> >>>>would now be adding whatever mechanisms are needed to > the > >> >>>>>>protocols. > >> >>>>>> >>>>We will need to add cases for > >> >>>>>> >>>>L2 service, L3 service and L2/L3 service. We no longer > >> >>>>>> >>>>have > >> >>>>>>simple > >> >>>>>> >>>>inner to outer mappings, but now need NVEs to do MAC > >> >>>>>> >>>>rewrites, > >> >>>>>>local > >> >>>>>> >>>>NVE ARP termination, and multiple lookups depending on > the > >> >>>>>> >>>>destination MAC address (first L2, then potentially L3). > >> >>>>>> >>>>We will also need to distribute two different VN > >> >>>>>> >>>>identifiers (one for > >> >>>>>>L2 and > >> >>>>>> >>>>one for L3), and somehow convey the containment > >> >>>>>> >>>>relationship > >> >>>>>>between > >> >>>>>> >>>>the two (multiple L2 VNs within one > >> >>>>>> >>>>L3 VN). While I think this is all very useful, I just want > >> >>>>>> >>>>to > >> >>>>>>make > >> >>>>>> >>>>sure the WG agrees to this since I feel it is a > >> >>>>>> >>>>significant change/increase in scope from my perspective. > >> >>>>>> >>>> > >> >>>>>> >>>>Thanks, Larry > >> >>>>>> >>>> > >> >>>>>> >>>> > >> >>>>>> >>>> > >> >>>>>> >>>>On 10/18/13 2:52 PM, "Thomas Narten" > <[email protected]> > >> wrote: > >> >>>>>> >>>> > >> >>>>>> >>>>>Hi Lucy. > >> >>>>>> >>>>> > >> >>>>>> >>>>>Lucy yong <[email protected]> writes: > >> >>>>>> >>>>> > >> >>>>>> >>>>>> Section 5.3 describes gateways. IMO: it misses an > >> >>>>>> >>>>>> important > >> >>>>>>use > >> >>>>>> >>>>>> case. A Gateway, say overlay gateway, may be used to > >> >>>>>>interconnect > >> >>>>>> >>>>>> two or more overlay VNs. In this case, the traffic > >> >>>>>> >>>>>> traversing between two overlay VNs must go through the > >> >>>>>> >>>>>> gateway where the policy can be enforced. Furthermore, > >> >>>>>> >>>>>> it is possible to > >> >>>>>>implement > >> >>>>>> >>>>>> centralized or distributed overlay gateway. The latter > >> >>>>>> >>>>>> has overlay gateway function implemented on NVEs. Thus, > >> >>>>>> >>>>>> it > >> >>>>>>requests > >> >>>>>> >>>>>> the cross-VN policies to be distributed to NVEs. > >> >>>>>> >>>>> > >> >>>>>> >>>>>> Current section seems very focus on overlay VN > >> >>>>>> >>>>>> interconnect a non-overlay network, which centralized > >> >>>>>> >>>>>> gateway architecture > >> >>>>>>is > >> >>>>>> >>>>>> practical. But in overlay networks, both centralized or > >> >>>>>> >>>>>> distributed are possible and depend on the applications. > >> >>>>>> >>>>> > >> >>>>>> >>>>>Agreed. I propose adding a new section after 5.3 that says: > >> >>>>>> >>>>> > >> >>>>>> >>>>> <section title="Distributed Gateways"> <t> The relaying > >> >>>>>> >>>>> of traffic from one VN to another deserves special > >> >>>>>> >>>>> consideration. The previous section described gateways > >> >>>>>> >>>>> performing this function. If such gateways are > >> >>>>>> >>>>> centralized, traffic between TSes on different VNs can > >> >>>>>> >>>>> take suboptimal paths, i.e., triangular routing results > >> >>>>>> >>>>> in paths that always traverse the gateway. As an > >> >>>>>> >>>>> optimization, individual NVEs can be part of a > >> >>>>>> >>>>> distributed gateway that performs such relaying, > >> >>>>>> >>>>> reducing or completely eliminating triangular routing. > >> >>>>>> >>>>> In a distributed gateway, each ingress NVE can perform > >> >>>>>> >>>>> such relaying activity directly, so long as it has > >> >>>>>> >>>>> access to the policy information needed to determine > >> >>>>>> >>>>> whether cross-VN communication is allowed. Having > >> >>>>>> >>>>> individual NVEs be part of a distributed gateway allows > >> >>>>>> >>>>> them to tunnel traffic directly to the destination NVE > without the need to take suboptimal paths. > >> >>>>>> >>>>> </t> > >> >>>>>> >>>>> <t> > >> >>>>>> >>>>> The NVO3 architecture should [must? or just say it > >> >>>>>> >>>>> does?] support distributed gateways. Such support > >> >>>>>> >>>>> requires that > >> >>>>>> >>>>> NVO3 control protocols include mechanisms for the > >> >>>>>> >>>>> maintenance and distribution of policy information about > >> >>>>>> >>>>> what type of cross-VN communication is allowed so that > >> >>>>>> >>>>> NVEs acting as distributed gateways can tunnel traffic > >> >>>>>> >>>>> from one VN to another as appropriate. > >> >>>>>> >>>>> </t> > >> >>>>>> >>>>> </section> > >> >>>>>> >>>>> > >> >>>>>> >>>>>Thoughts? > >> >>>>>> >>>>> > >> >>>>>> >>>>>Thomas > >> >>>>>> >>>>> > >> >>>>>> >>>>>_______________________________________________ > >> >>>>>> >>>>>nvo3 mailing list > >> >>>>>> >>>>>[email protected] > >> >>>>>> >>>>>https://www.ietf.org/mailman/listinfo/nvo3 > >> >>>>>> >>>> > >> >>>>>> >>> > >> >>>>>> >> > >> >>>>>> >>_______________________________________________ > >> >>>>>> >>nvo3 mailing list > >> >>>>>> >>[email protected] > >> >>>>>> >>https://www.ietf.org/mailman/listinfo/nvo3 > >> >>>>>> > > >> > >> _______________________________________________ > >> nvo3 mailing list > >> [email protected] > >> https://www.ietf.org/mailman/listinfo/nvo3 > > _______________________________________________ > nvo3 mailing list > [email protected] > https://www.ietf.org/mailman/listinfo/nvo3 _______________________________________________ nvo3 mailing list [email protected] https://www.ietf.org/mailman/listinfo/nvo3
