Frank, inline
On 28/10/13 03:49, "Xialiang (Frank)" <[email protected]> wrote: >Hi Yves, >Thanks for your response. >Please see my response with [Frank]. > >B.R. >Frank > >> -----Original Message----- >> From: [email protected] [mailto:[email protected]] On Behalf Of >> Yves Hertoghs (yhertogh) >> Sent: Saturday, October 26, 2013 6:12 PM >> To: Xialiang (C); Lizhong Jin >> Cc: 'Thomas Narten'; [email protected]; [email protected]; Lucy yong; >> Xuxiaohu; Larry Kreeger (kreeger) >> Subject: Re: [nvo3] Distributed Gateways [was Re: NVO3 Architecture >> document] >> >> Inline. >> >> From: "Xialiang (C)" <[email protected]> >> Date: Saturday 26 October 2013 11:42 >> To: Yves Hertoghs <[email protected]>, Lizhong Jin >> <[email protected]> >> Cc: 'Thomas Narten' <[email protected]>, "[email protected]" >> <[email protected]>, "[email protected]" <[email protected]>, Lucy yong >> <[email protected]>, Xuxiaohu <[email protected]>, "Larry >> Kreeger (kreeger)" <[email protected]> >> Subject: RE: [nvo3] Distributed Gateways [was Re: NVO3 Architecture >> document] >> >> >> >Hi all, >> >Distributed Gateway is a very interesting topic. It would also be >> >valuable to solve some challenge introduced by network virtualization. >> > >> >To support the intra/inter-subnet connections through the same or >> >different NVEs in one tenant, the Distributed Gateway should implement >> >the bridging function and routing function at the same time. >> > >> >But, there are still some points need to be studied and solved. Below >> >are some examples (NVE1 and NVE2 are all Distributed Gateway. Hosts of >> >subnet >> >1.1.1.0/24 and 2.2.2.0/24 are attached to them): >> > ------- / >> > ------- ------ / >> > // \\ / >> > // \\ / >> > // \\ / >> > / \ / >> > / +--------+ +-------+\/Internet >> > / | | | |/\ >> > / | CORE |------+Gateway/ \ >> > / | | | | \ >> > | +--/----\* +-------+ | >> > | // \\ | >> >| // \\ | >> >| // \\ | >> >| +----/----+ +----\----+ | >> >| | | | | | >> >| | NVE1 | NVE2 | | >> >| | / \ | | / \ | | >> >| +-/-----\-+ +-/-----\-+ | >> >| / \ / \ | >> >| / \ / \ | >> >| --- ---- / \ | >> > | // \\ / \ / -- \ --- | >> > | |1.1.1.0| | | // \\ / \ | >> > \ | | |2.2.2.0 | | 2.2.2.0/ >> > \ \\ // \ / |1.1.1.0 \ // >> > \ --- ---- | | ---/ >> > \ \\ // / >> > \ -- / >> > \\ // >> > \\ // >> > \\ // >> > ------- ------- >> > >> >Problems 1: If we configure the same gateway address (i.e. 1.1.1.1) for >> >the two subnets--1.1.1.0/24 attached to NVE1 and NVE2, and there is >> >intra-subnet connection between them, it will result in an gateway >> >address collision in one subnet. If we configure different gateway >> >address for same subnet attached to different NVEs to solve the address >> >collision problem, it will bring a huge configuration workload and >> >waste of address space, and also it will limit the VM mobility. >> >> Why ? You are thinking that the default gateway has subnets configured, >> feeding into its routing table. It doesn't. It learns all hosts as >>host-routes, >> with optional aggregation in case the host-routing table can be >>aggregated >> on bit boundaries. Host might have mask/prefix-lengths configured, as >>that >> is how they are operating today, but in reality these denote an >>address-pool >> rather than a subnet/prefix. For IP traffic these two parts of a >>subnet are >> not interconnected by a L2 link. Surely there might be a L2 overlay >>spanning >> both NVE's, but that overlay does not switch IP packets. >> >[Frank]: Maybe my description is not so clear. What I mean is every host >in the subnet configures its gateway for IP forwarding. For the same >subnet distributed in different NVEs, Same gateway address located in >different NVEs can exist. When these gateways in different NVEs send >gratuitous ARP messages in the subnet, or several VRRP groups in same >subnet send VRRP messages in the subnet, the IP address collision between >gateways will be detected. How to solve this kind of problem? In the topology drawn by you i dont see any L2 interconnection between NVE1 and NVE2. You dont even need VRRP is this topology. You are again assuming a distributed subnets equals one L2 domain. It doesn't need to be like this, as long as all hosts are treated as host-routes on the NVEs i.e. The NVE is not configured for a certain TS prefix on an interface. It dynamically installs the host-routes based on Tses registering (via VQP or some other means) and pushes them to the NVA. The subnet is no longer a subnet, it is merely an address-pool. > > >> >> > >> >Problem 2: How to aggregate or decrease the address number learned by >> >NVEs or hosts? >> In any case you will perform the same or better than when learning >> MAC-addresses. With a MAC address namespace you cannot aggregate. >> With an IP namespace and host-route learning there is at least a change >>of >> aggregation, either at the NVE, or higher up in the data centre. And >>actually >> the aggregation function isn't really a part of the NVE, it can be a >>function of >> the NVA to aggregate address-space, if a contiguous block of >>host-routes are >> all pointing to the same NVE. >> >[Frank]: Right, I agree your analysis about address aggregation. >Actually, what I am concerned is the possibility to decrease or need not >to learn all host's address by some gateway-like method. > >> > >> >Problem 3: How to deal with multi-tenants? >> >> This is orthogonal. The NVE learns about hosts and their IP addresses >> within the NV instance. I dont see why multi tenancy is a problem ? >> >> > >> >In addition to above problems, there are maybe more points around the >> >Distributed Gateway issue. >> > >> >I think the distributed gateway will solve those problems. But before >> >we define any functions of it, we need a problem statement document for >> >those scenarios firstly. >> I think the problem statement of how the solution should look like from >>a >> user's perspective is already there. A distributed gateway, with a >>unified >> L2/L3 NVE on it, is an option to solve those problems. >> >[Frank]: It depends on how many problems we find by discussion and come >to the agreement. We can keep on discussing. Haven't seen any problem yet, as this model is implemented already. The biggest problem is that people dont fully understand it, and perhaps we need a separate draft to discuss this in detail. Draft-hertoghs does explain it, but the LISP machinery around it might confuse people i.e the distributed, unified L2/L3 NVE is orthogonal to the NVE-NVA CP being used. Yves > >> > >> >Your comments? >> > >> >B.R. >> >Frank >> > >> > >> >From: [email protected] [mailto:[email protected]] On Behalf Of >> >Yves Hertoghs (yhertogh) >> >Sent: Friday, October 25, 2013 11:05 PM >> >To: Lizhong Jin >> >Cc: 'Thomas Narten'; [email protected]; [email protected]; Lucy yong; >> >Xuxiaohu; Larry Kreeger (kreeger) >> >Subject: Re: [nvo3] Distributed Gateways [was Re: NVO3 Architecture >> >document] >> > >> > >> > >> >True, but the unified L2/L3 solution gives you the choice on a per VNI >> >basis whether you want to do 'enhanced IP forwarding' or 'Ethernet >> >based forwarding'. >> > >> > >> > >> >I also wouldn't call RDMA over Converged Ethernet a prime use case for >> >a multi tenant DC. Typically this would be deployed standalone and >> >across a few ethernet switches. >> > >> > >> > >> >Yves >> > >> > >> > >> >From: >> >Lizhong Jin <[email protected]> >> >Date: Friday 25 October 2013 16:52 >> >To: Yves Hertoghs <[email protected]> >> >Cc: Lucy yong <[email protected]>, "[email protected]" >> ><[email protected]>, 'Thomas Narten' <[email protected]>, >> 'Xuxiaohu' >> ><[email protected]>, "[email protected]" <[email protected]>, "Larry Kreeger >> >(kreeger)" <[email protected]> >> >Subject: RE: [nvo3] Distributed Gateways [was Re: NVO3 Architecture >> >document] >> > >> > >> > >> >>Yves, >> >>In that scenario, I agree. But my concern is, is it realistic for a >> >>datacenter with only IP traffic. I am not sure. But at least as I >> >>know, if you want to deploy RoCE, there will be non-IP traffic. >> >> >> >>Lizhong >> >> >> >>From: Yves Hertoghs (yhertogh) [mailto:[email protected]] >> >> >> >>Sent: Friday, October 25, 2013 6:07 PM >> >>To: Lizhong Jin >> >>Cc: 'Lucy yong'; [email protected]; 'Thomas Narten'; 'Xuxiaohu'; >> >>[email protected]; Larry Kreeger (kreeger) >> >>Subject: Re: [nvo3] Distributed Gateways [was Re: NVO3 Architecture >> >>document] >> >> >> >> >> >> >> >>Lizhong, >> >> >> >> >> >> >> >>My main point is that , in the absence of non-IP traffic, there is a >> >>lot of value of doing IP based forwarding for all traffic, as this >> >>effectively isolates the failure domains to the L2 domains underneath >> >>the NVE's. >> >> >> >> >> >> >> >>Yves >> >> >> >> >> >> >> >>From: >> >>Lizhong Jin <[email protected]> >> >>Date: Friday 25 October 2013 04:33 >> >>To: Yves Hertoghs <[email protected]> >> >>Cc: Lucy yong <[email protected]>, "[email protected]" >> >><[email protected]>, 'Thomas Narten' <[email protected]>, >> 'Xuxiaohu' >> >><[email protected]>, "[email protected]" <[email protected]>, "Larry >> Kreeger >> >>(kreeger)" <[email protected]> >> >>Subject: RE: [nvo3] Distributed Gateways [was Re: NVO3 Architecture >> >>document] >> >> >> >> >> >> >> >>>See inline. >> >>> >> >>>From: Yves Hertoghs (yhertogh) [mailto:[email protected]] >> >>> >> >>>Sent: 2013年10月24日 >> >>> 23:54 >> >>>To: Lizhong Jin >> >>>Cc: Lucy yong; [email protected]; Thomas Narten; Xuxiaohu; >> >>>[email protected]; Larry Kreeger (kreeger) >> >>>Subject: Re: [nvo3] Distributed Gateways [was Re: NVO3 Architecture >> >>>document] >> >>> >> >>> >> >>> >> >>>Inline >> >>> >> >>> >> >>> >> >>>From: >> >>>Lizhong Jin <[email protected]> >> >>>Date: Thursday 24 October 2013 17:06 >> >>>To: Yves Hertoghs <[email protected]> >> >>>Cc: Lucy yong <[email protected]>, "[email protected]" >> >>><[email protected]>, Thomas Narten <[email protected]>, >> Xuxiaohu >> >>><[email protected]>, "[email protected]" <[email protected]>, "Larry >> >>>Kreeger (kreeger)" <[email protected]> >> >>>Subject: Re: [nvo3] Distributed Gateways [was Re: NVO3 Architecture >> >>>document] >> >>> >> >>> >> >>> >> >>>>Hi Yves, >> >>>>Are you referring the benefits below: >> >>>>Effectively this allows the 'spread' of >> >>>> subnets across the Datacenter(s) without leading to network wide >> >>>> broadcast and associated failure domains, while allowing free >> >>>> mobility of the end-stations. >> >>>>MAC based forwarding will also not lead to network wide broadcast, >> >>>>if all the MAC mapping entries are generated by NVA. >> >>>> >> >>>> >> >>>> >> >>>Since when is that a requirement for the NVA ? An NVA will store the >> >>>mappings, not generate it. The mapping generation can be done >> >>>through orchestration of the NVA, or through the NVE to NVA >> >>>interactions as a result of the NVE detection new Tenant Systems. >> >>> >> >>>Or are you saying that, independent of how the NVA knows about them, >> >>>tenant broadcast will only get flooded within the scope of the VNI? >> >>>If yes, why would this prevent from sending L2 broadcasts to all >> >>>participating NVEs, and attached TS'es? There might be some >> >>>optimisations in the underlay to make this work 'better' , but it >> >>>doesn't stop those ugly L2 broadcast packets from hitting all TS'es. >> >>>A >> >>> L2 overlay creates a L2 failure domain, in the context of broadcast >> >>>and multicast. Nothing you can do about it. >> >>> >> >>>[Lizhong] Sorry for the confusion, I mean the MAC mapping of remote >> >>>will be got from NVA, and MAC mapping of local will be got from NVA >> >>>or generated by itself. I did not invent new things, and all of that >> >>>is described in draft-narten-nvo3-arch-01. The NVE will reply the ARP >> >>>from local TS if corresponding MAC mapping is already existed. For >> >>>EVPN, broadcast only happen when there is no MAC-IP bindings locally, >> >>>and that works well. There maybe some benefits existed from the >> >>>multicast view, and I should think about it. >> >>> >> >>>Thanks >> >>>Lizhong >> >>> >> >>> >> >>> >> >>>>I remember this has already been discussed on NVO3 list last year. >> >>>>The only benefit I could see is, for an IP only data center, you >> >>>>only need to have IP forwarding. >> >>>>Regards >> >>>>Lizhong >> >>>>On Oct 24, 2013 7:24 PM, "Yves Hertoghs (yhertogh)" >> >>>><[email protected]> wrote: >> >>>>> >> >>>>> inline >> >>>>> >> >>>>> From: Lizhong Jin <[email protected]> >> >>>>> Date: Thursday 24 October 2013 03:54 >> >>>>> To: "Larry Kreeger (kreeger)" <[email protected]>, Xuxiaohu >> >>>>><[email protected]>, Yves Hertoghs <[email protected]>, Lucy >> >>>>>yong <[email protected]>, >> >>>> Thomas Narten <[email protected]> >> >>>>> >> >>>>> Cc: "[email protected]" <[email protected]>, "[email protected]" >> >>>>><[email protected]> >> >>>>> Subject: RE: [nvo3] Distributed Gateways [was Re: NVO3 >> >>>>>Architecture document] >> >>>>> >> >>>>>> Hi Larry, >> >>>>>> >> >>>>>> 1) A name for the service (lets's pick one, is it hybrid, >> >>>>>>combined or unified?) >> >>>>>> >> >>>>>> [Lizhong] IRB, which have been used before. We could change it to >> >>>>>>virtual IRB. >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> 2) To address it from a solution point of view (e.g. are packets >> >>>>>>forwarded based on MAC or based on IP when source and dest are in >> >>>>>>the same subnet?). >> >>>>>> >> >>>>>> [Lizhong] Yes, we should do that, and there are already some >> >>>>>>drafts there >> >>>>>>(http://tools.ietf.org/html/draft-jin-nvo3-virb-centralized-00). >> >>>>>>For IP forwarding for intra subnet traffic: >> >>>>>> >> >>>>>> 1. I don’t think it could be wildly accepted in next two or >> >>>>>>three years to have a pure IP datacenter. Pure Ethernet traffic >> >>>>>>and MAC forwarding will be there anyway. >> >>>>>> >> >>>>>> 2. From the dataplane point, you have to design larger IP >> >>>>>>routing table, and possibly larger TCAM (for prefix entry), and >> >>>>>>higher cost consequently. What’s more, should we even design aging >> >>>>>>for IP entry like MAC entry? >> >>>>>> >> >>>>>> 3. From the control plane point, you need to change the >> >>>>>>mapping entry rule, and may have new interoperable issue with >> >>>>>>legacy >> >>>>>>L2/L3 VPN. >> >>>>>> >> >>>>>> I don’t see much benefit. Could anyone give me some hints? >> >>>>> >> >>>>> [yves] read draft-hertoghs-nvo3-lisp-unified-control-plane >> >>>>> >> >>>>> Yves >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> Thanks >> >>>>>> >> >>>>>> Lizhong >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> From: Larry Kreeger (kreeger) [mailto:[email protected]] >> >>>> >> >>>>>> Sent: 2013年10月24日 >> >>>> 8:32 >> >>>>>> To: Xuxiaohu; Lizhong Jin; Yves Hertoghs (yhertogh); Lucy yong; >> >>>>>>Thomas Narten >> >>>>>> Cc: [email protected]; >> >>>>[email protected] <mailto:[email protected]> >> >>>>>> Subject: Re: 答复: [nvo3] Distributed Gateways [was Re: NVO3 >> >>>>>>Architecture document] >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> I've now read through this whole thread twice. I see now that >> >>>>>>there may be some confusion between an L2/L3 service (what >> >>>>>>tenant's >> >>>>>>see) and an L2/L3 solution (how NVO3 provides the service). >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> Prior to this thread, I may have agreed that we don't need to >> >>>>>>define a hybrid/combined/unified L2/L3 "service", but now I'm less >> >>>>>>sure. Just be be clear, I was not calling so much for a definition >> >>>>>>of the service, but that we need: >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> 1) A name for the service (lets's pick one, is it hybrid, >> >>>>>>combined or unified?) >> >>>>>> >> >>>>>> 2) To address it from a solution point of view (e.g. are packets >> >>>>>>forwarded based on MAC or based on IP when source and dest are in >> >>>>>>the same subnet?). >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> As I stated before, I believe the control plane requirements will >> >>>>>>be different for a combined service than for a pure L2 or pure L3 >> >>>>>>service, and that we if don't start specifically calling out the >> >>>>>>control plane requirements for a combined service (from a >> >>>> solution point of view), we will have a very low likelihood of >> >>>>interoperable solutions as each implementer fills in the gaps in >> >>>>their own way. >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> Does the WG agree that we should do 1) and 2) above? >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> I would also not object if we had a short description in the >> >>>>>>framework of what the combined service looks like compared to a >> >>>>>>pure >> >>>>>>L2 or pure L3 service, just so we can level set for everyone. >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> Thanks, Larry >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> From: Xuxiaohu <[email protected]> >> >>>>>> Date: Tuesday, October 22, 2013 11:38 PM >> >>>>>> To: Larry Kreeger <[email protected]>, Lizhong Jin >> >>>>>><[email protected]>, "Yves Hertoghs (yhertogh)" >> >>>>>><[email protected]>, Lucy yong <[email protected]>, >> >>>> Thomas Narten <[email protected]> >> >>>>>> Cc: "[email protected]" <[email protected]>, "[email protected]" >> >>>>>><[email protected]> >> >>>>>> Subject: 答复: [nvo3] Distributed Gateways [was Re: NVO3 >> >>>>>>Architecture document] >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> 发件人: >> >>>>[email protected] [mailto:[email protected]] >> >>>>代表 Larry Kreeger (kreeger) >> >>>>>> 发送时间: 2013年10月23日 >> >>>> 11:27 >> >>>>>> 收件人: Lizhong Jin; Yves Hertoghs (yhertogh); Lucy yong; Thomas >> >>>>>> Narten >> >>>>>> 抄送: >> >>>>[email protected]; >> >>>>[email protected] <mailto:[email protected]> >> >>>>>> 主题: Re: [nvo3] Distributed Gateways [was Re: NVO3 Architecture >> >>>>>>document] >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> Hi Lizong, >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> Let me give more details on my thinking. >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> A pure L2 service only requires inner MAC address to outer IP >> >>>>>>address mappings. When a frame is delivered between two TS, the >> >>>>>>source and dest MAC addresses have not been changed. >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> A pure L3 service only requires inner IP to outer IP address >> >>>>>>mappings. There is no expectation that two TS are on the same L2 >> >>>>>>segment, and the TS would always ARP for a router to reach another >> >>>>>>TS. When a frame is delivered between the two TS, both the >> >>>> source and dest MAC will have been rewritten. TTL will have been >> >>>>decremented at least once. >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> I'm expecting a hybrid L2/L3 service to preserve L2 semantics >> >>>>>>within a subnet. TS ARP for the other TS on its subnet instead of >> >>>>>>the router. The source and dest MAC are not rewritten. TTL is >> >>>>>>not decremented. From the perspective of the two TS on the same >> >>>> subnet, they are connected to the same L2 segment. To achieve >> >>>>this, the NVEs must behave differently when forwarding IP packets >> >>>>within the same L2 VN vs between different L2 VNs. For example, the >> >>>>NVE would need to respond to ARP, not with its own MAC address, but >> >>>>with the remote TSs MAC address if the TS are in the same L2 VN. >> >>>>Furthermore, as I wrote below, there needs to be a mapping of the >> >>>>set of L2 VNs that belong to the L3 VN. >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> If so, what’s the difference between hybrid L2/L3 service and IRB >> >>>>>>(Integrated Routing and Bridging)? In addition, I believe the >> >>>>>>hybrid >> >>>>>>L2/L3 service you mentioned above is different from what Yves >> >>>>>>meant by “combined L2/L3”. The former is about bridging >> >>>> for intra-subnet traffic and routing for inter-subnet traffic while >> >>>>the latter is about routing for (both intra-subnet and inter-subnet) >> >>>>IP traffic while bridging for non-IP traffic. >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> Best regards, >> >>>>>> >> >>>>>> Xiaohu >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> So, I still think NVEs will need some additional information for >> >>>>>>a hybrid service, and I think we will need to describe the >> >>>>>>behavior of a hybrid service beyond a sentence in the framework >> >>>>>>saying it is possible. Otherwise, I think it is quite likely that >> >>>> two different hybrid implementations will not interoperate. >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> - Larry >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> From: Lizhong Jin <[email protected]> >> >>>>>> Date: Tuesday, October 22, 2013 7:31 PM >> >>>>>> To: Larry Kreeger <[email protected]>, "Yves Hertoghs (yhertogh)" >> >>>>>><[email protected]>, Lucy yong <[email protected]>, Thomas >> >>>>>>Narten >> >>>> <[email protected]> >> >>>>>> Cc: "[email protected]" <[email protected]>, "[email protected]" >> >>>>>><[email protected]> >> >>>>>> Subject: RE: [nvo3] Distributed Gateways [was Re: NVO3 >> >>>>>>Architecture document] >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> > >> >>>>>> >With regards to the 'new service type' of a combined L2/L3 NVE, >> >>>>>> >> >>>>>>>http://tools.ietf.org/html/draft-hertoghs-nvo3-lisp-controlplane- >> >>>>>>>uni >> >>>>>>>fie >> >>>>>> >d-0 >> >>>>>> >0 >> >>>>>> > describes what you are referring to , but i am not sure if it >> >>>>>>needs to >> >>>>>> >be called a new service-type. I see it more as a deployment >> >>>>>> >choice where you choose to deploy distributed gateways with >> >>>>>> >uniform mac/IP addressing across the DC, collocate them with the >> >>>>>> >L2 overlay, and >> >>>>>>do >> >>>>>> >some traffic steering to make sure IP traffic gets sent to the >> >>>>>> >L3 overlay (at all times), and non-IP traffic gets sent across >> >>>>>> >the L2 >> >>>>>> overlay. >> >>>>>> >> >>>>>> LK> I'll give you an example of why I think the protocol >> >>>>>>requirements will >> >>>>>> be different for L2 vs L3 vs a combined L2/L3 service. For an L2 >> >>>>>>VN, the VN needs to be identified (e.g. with a Name or ID). For >> >>>>>>an L3 VN, it similarly needs to be identified. For a combined >> >>>>>>L2/L3 service, I think the NVEs need to know both the L3 VN >> >>>>>>identity (one), and all the L2 VN identities that are part of the >> >>>>>>L3 VN. When doing distributed L3 forwarding between a TS on one >> >>>>>>L2 VN to one on another L2 VN, it will need to know not just the >> >>>>>>mapping of inner to outer address, but the mapping of inner L3 >> >>>>>>address to destination L2 VN and MAC address (so it can rewrite >> >>>>>>the MAC and the L2 VN). >> >>>>>> >> >>>>>> [Lizhong] I tend to agree with Yves, it is more of a deployment >> >>>>>>choice. >> >>>>>> The behavior you described for L3 forwarding between two L2 VPN >> >>>>>>is more of a standard L3 forwarding function, looking up with >> >>>>>>destination IP address, then getting the destination MAC, source >> >>>>>>MAC, VLAN and NVO3 tunnel from the table. The MAC and VLAN >> are >> >>>>>>resolved from the next hop within same L3 VN. We write a >> combined >> >>>>>>L2/L3 forwarding for NVO3 last year, just for reference >> >>>>http://tools.ietf.org/html/draft-jin-nvo3-virb-centralized-00 >> >>>><http://tools.ietf.org/html/draft-jin-nvo3-virb-centralized-00>. >> >>>>>> BTW, we may update it soon. >> >>>>>> >> >>>>>> Thanks >> >>>>>> Lizhong >> >>>>>> >> >>>>>> > >> >>>>>> >Yves >> >>>>>> > >> >>>>>> >On 21/10/13 10:28, "Larry Kreeger (kreeger)" <[email protected]> >> >>>>>>wrote: >> >>>>>> > >> >>>>>> >>Lucy, >> >>>>>> >> >> >>>>>> >>See inline with LK2>. >> >>>>>> >> >> >>>>>> >> - Larry >> >>>>>> >> >> >>>>>> >>On 10/20/13 8:20 PM, "Lucy yong" <[email protected]> >> wrote: >> >>>>>> >> >> >>>>>> >>>Hi Larry, >> >>>>>> >>> >> >>>>>> >>>Please see inline with [Lucy] >> >>>>>> >>> >> >>>>>> >>>-----Original Message----- >> >>>>>> >>>From: Larry Kreeger (kreeger) [mailto:[email protected]] >> >>>>>> >>>Sent: Friday, October 18, 2013 6:39 PM >> >>>>>> >>>To: Lucy yong; Thomas Narten >> >>>>>> >>>Cc: [email protected] >> >>>>>> >>>Subject: Re: [nvo3] Distributed Gateways [was Re: NVO3 >> >>>>>>Architecture >> >>>>>> >>>document] >> >>>>>> >>> >> >>>>>> >>>Lucy, >> >>>>>> >>> >> >>>>>> >>>See inline with LK>. >> >>>>>> >>> >> >>>>>> >>>Thanks, Larry >> >>>>>> >>> >> >>>>>> >>>On 10/18/13 3:21 PM, "Lucy yong" <[email protected]> >> wrote: >> >>>>>> >>> >> >>>>>> >>>>Larry, >> >>>>>> >>>> >> >>>>>> >>>>Distributed L3 gateway is very useful and some vendors >> >>>>>> >>>>already implement that. >> >>>>>> >>> >> >>>>>> >>>I agree. >> >>>>>> >>> >> >>>>>> >>>>Current framework also includes L2/L3 service as Marc agrees >> >>>>>> >>>>to >> >>>>>>add >> >>>>>> >>>>the text I proposed although we did not explicitly define as >> >>>>>> >>>>a service type. >> >>>>>> >>> >> >>>>>> >>>I think we will need to because the control plane requirements >> >>>>>>will >> >>>>>> >>>likely be different for a hybrid service. >> >>>>>> >>>[Lucy] Do you mean that we should define an l2/L3 service >> >>>>>> >>>type? I full agree and hope see more people support that too. >> >>>>>> >>>We can >> >>>>>>either >> >>>>>> >>>define it in the existing framework, or in the framework >> >>>>>> >>>addition draft I submitted while ago. >> >>>>>> >> >> >>>>>> >>LK2> Yes, if the group is to seriously address a hybrid L2/L3 >> >>>>>>service, >> >>>>>> >>LK2> I >> >>>>>> >>think we need to define it as we will need to identify its >> >>>>>> >>requirements independently from a pure L2 or pure L3 service. >> >>>>>> >> >> >>>>>> >>> >> >>>>>> >>>> >> >>>>>> >>>>Current architecture clear states NVE and NVA roles, i.e. NVE >> >>>>>> >>>>performs forwarding, and NVA performs routing. >> >>>>>> >>> >> >>>>>> >>>I'm not sure what you mean by routing. Are you referring to L3 >> >>>>>> service? >> >>>>>> >>>If so, I don't agree. >> >>>>>> >>>[Lucy] Sorry to make you confuse. The routing here does not >> >>>>>> >>>mean >> >>>>>>L3 >> >>>>>> >>>services. I should state that NVE performs data plan >> >>>>>> >>>forwarding, >> >>>>>>NVA >> >>>>>> >>>performs control plane routing, which, IMO, it is not just >> >>>>>>simple DB >> >>>>>> >>>to have inner to outer mappings. NVA may get the routing >> >>>>>> >>>policy >> >>>>>>from >> >>>>>> >>>operators or customer, then interpret the policy, then >> >>>>>> >>>generates >> >>>>>>the >> >>>>>> >>>mapping of tenant/next-hop location and send to NVEs. An NVE >> >>>>>>receives >> >>>>>> >>>the mapping in which, if the location is the same as itself, >> >>>>>> >>>it translates to a tenant/VAP mapping; If not, it installs as >> >>>>>> >>>an inner/outer mapping. >> >>>>>> >>>Thus, it works regardless whether sender tenant and >> >>>>>> >>>destination tenant are one the same NVE or on different NVEs. >> >>>>>> >>>The mapping >> >>>>>>between >> >>>>>> >>>tenant/location is fully controlled under NVA. In this way, >> >>>>>>operator >> >>>>>> >>>only needs to input the routing policies to NVA. NVE simply >> >>>>>>performs >> >>>>>> >>>the forwarding accordingly. This is also my view about the SDN >> >>>>>>based >> >>>>>> >>>architecture. >> >>>>>> >>> >> >>>>>> >>>>If NVA is not able to distribute routing policy to NVE at >> >>>>>> >>>>all, >> >>>>>>I do >> >>>>>> >>>>not know how NVA can perform route distribution control? >> >>>>>> >>> >> >>>>>> >>>You lost me. Are you referring to a hybrid L2/L3 service? >> >>>>>> >>>[Lucy] Let me know above explanation help or not. No, not >> >>>>>>particulate >> >>>>>> >>>to >> >>>>>> >>>L2/L3 service but that certainly applies to L2/L3 service. >> >>>>>> >> >> >>>>>> >>LK2> OK, if it isn't specific to L2/L3 service, than I'd rather >> >>>>>>leave >> >>>>>> >>LK2> it >> >>>>>> >>out of the discussion for now. And yes, the above helped me >> >>>>>> >>understand what you meant. The term "routing" is very >> overloaded. >> >>>>>> >> >> >>>>>> >>> >> >>>>>> >>>> If NVA only supports simple inner to outer mappings, how can >> >>>>>>NVE >> >>>>>> >>>>get information to perform local forwarding? >> >>>>>> >>> >> >>>>>> >>>Inner to outer mapping resolution works fine for pure L2 or >> >>>>>> >>>pure >> >>>>>>L3 >> >>>>>> >>>service. Local forwarding doesnšt need mappings, the local NVE >> >>>>>>knows >> >>>>>> >>>what VAPs the TSI are connected to. >> >>>>>> >>>[Lucy] does pure L3 service means an L3VPN w/o any policy? For >> >>>>>>an L3 >> >>>>>> >>>service, you can implement it w/o policy or w policy. >> >>>>>> >> >> >>>>>> >>LK2> Yes, I was thinking of pure L3 service without adding >>policy. >> >>>>>> >>LK2> I'm >> >>>>>> >>pretty sure you need more than just inner to outer mappings to >> >>>>>> >>implement policy. >> >>>>>> >> >> >>>>>> >>>IMO: NVE is not the entity to enforce the policy. NVA is the >> >>>>>>entity >> >>>>>> >>>to enforce the policy regardless the tenant locations. Again, >> >>>>>>Network >> >>>>>> >>>virtualization overlay has to address how to support the >> >>>>>>policies and >> >>>>>> >>>tenant mobility. IMO: current architecture draft is vague in >> >>>>>> >>>or >> >>>>>>lacks >> >>>>>> >>>of describing this but it is important to architect this when >> >>>>>>having >> >>>>>> >>>data plan and control plane separated on different entities. >> >>>>>> >> >> >>>>>> >>LK2> I'm not sure if I agree with you about the NVE not being >> >>>>>> >>LK2> the entity >> >>>>>> >>to enforce policy, but I think it is something better discussed >> >>>>>>in a >> >>>>>> >>conversation (not email). >> >>>>>> >> >> >>>>>> >>> >> >>>>>> >>>Thanks, >> >>>>>> >>>Lucy >> >>>>>> >>> >> >>>>>> >>>> >> >>>>>> >>>>Thanks, >> >>>>>> >>>>Lucy >> >>>>>> >>>> >> >>>>>> >>>>-----Original Message----- >> >>>>>> >>>>From: Larry Kreeger (kreeger) [mailto:[email protected]] >> >>>>>> >>>>Sent: Friday, October 18, 2013 5:00 PM >> >>>>>> >>>>To: Thomas Narten; Lucy yong >> >>>>>> >>>>Cc: [email protected] >> >>>>>> >>>>Subject: Re: [nvo3] Distributed Gateways [was Re: NVO3 >> >>>>>>Architecture >> >>>>>> >>>>document] >> >>>>>> >>>> >> >>>>>> >>>>Hi Thomas and Lucy, >> >>>>>> >>>> >> >>>>>> >>>>The WG needs to think hard about this one. >> >>>>>> >>>> >> >>>>>> >>>>Support of a distributed L3 gateway function between L2 VNs >> >>>>>> >>>>is a significant increase in scope of the NVA, and the NVE to >> >>>>>> >>>>NVA >> >>>>>>protocol. >> >>>>>> >>>>Where we had previously stated L2 service or L3 service and >> >>>>>>pretty >> >>>>>> >>>>much left a combined L2/L3 service as an exercise for the >> >>>>>>reader, we >> >>>>>> >>>>would now be adding whatever mechanisms are needed to the >> >>>>>>protocols. >> >>>>>> >>>>We will need to add cases for >> >>>>>> >>>>L2 service, L3 service and L2/L3 service. We no longer have >> >>>>>>simple >> >>>>>> >>>>inner to outer mappings, but now need NVEs to do MAC >> >>>>>> >>>>rewrites, >> >>>>>>local >> >>>>>> >>>>NVE ARP termination, and multiple lookups depending on the >> >>>>>> >>>>destination MAC address (first L2, then potentially L3). We >> >>>>>> >>>>will also need to distribute two different VN identifiers >> >>>>>> >>>>(one for >> >>>>>>L2 and >> >>>>>> >>>>one for L3), and somehow convey the containment relationship >> >>>>>>between >> >>>>>> >>>>the two (multiple L2 VNs within one >> >>>>>> >>>>L3 VN). While I think this is all very useful, I just want to >> >>>>>>make >> >>>>>> >>>>sure the WG agrees to this since I feel it is a significant >> >>>>>> >>>>change/increase in scope from my perspective. >> >>>>>> >>>> >> >>>>>> >>>>Thanks, Larry >> >>>>>> >>>> >> >>>>>> >>>> >> >>>>>> >>>> >> >>>>>> >>>>On 10/18/13 2:52 PM, "Thomas Narten" <[email protected]> >> wrote: >> >>>>>> >>>> >> >>>>>> >>>>>Hi Lucy. >> >>>>>> >>>>> >> >>>>>> >>>>>Lucy yong <[email protected]> writes: >> >>>>>> >>>>> >> >>>>>> >>>>>> Section 5.3 describes gateways. IMO: it misses an >> >>>>>> >>>>>> important >> >>>>>>use >> >>>>>> >>>>>> case. A Gateway, say overlay gateway, may be used to >> >>>>>>interconnect >> >>>>>> >>>>>> two or more overlay VNs. In this case, the traffic >> >>>>>> >>>>>> traversing between two overlay VNs must go through the >> >>>>>> >>>>>> gateway where the policy can be enforced. Furthermore, it >> >>>>>> >>>>>> is possible to >> >>>>>>implement >> >>>>>> >>>>>> centralized or distributed overlay gateway. The latter has >> >>>>>> >>>>>> overlay gateway function implemented on NVEs. Thus, it >> >>>>>>requests >> >>>>>> >>>>>> the cross-VN policies to be distributed to NVEs. >> >>>>>> >>>>> >> >>>>>> >>>>>> Current section seems very focus on overlay VN >> >>>>>> >>>>>> interconnect a non-overlay network, which centralized >> >>>>>> >>>>>> gateway architecture >> >>>>>>is >> >>>>>> >>>>>> practical. But in overlay networks, both centralized or >> >>>>>> >>>>>> distributed are possible and depend on the applications. >> >>>>>> >>>>> >> >>>>>> >>>>>Agreed. I propose adding a new section after 5.3 that says: >> >>>>>> >>>>> >> >>>>>> >>>>> <section title="Distributed Gateways"> <t> The relaying of >> >>>>>> >>>>> traffic from one VN to another deserves special >> >>>>>> >>>>> consideration. The previous section described gateways >> >>>>>> >>>>> performing this function. If such gateways are centralized, >> >>>>>> >>>>> traffic between TSes on different VNs can take suboptimal >> >>>>>> >>>>> paths, i.e., triangular routing results in paths that >> >>>>>> >>>>> always traverse the gateway. As an optimization, individual >> >>>>>> >>>>> NVEs can be part of a distributed gateway that performs >> >>>>>> >>>>> such relaying, reducing or completely eliminating >> >>>>>> >>>>> triangular routing. In a distributed gateway, each ingress >> >>>>>> >>>>> NVE can perform such relaying activity directly, so long as >> >>>>>> >>>>> it has access to the policy information needed to determine >> >>>>>> >>>>> whether cross-VN communication is allowed. Having >> >>>>>> >>>>> individual NVEs be part of a distributed gateway allows >> >>>>>> >>>>> them to tunnel traffic directly to the destination NVE >> >>>>>> >>>>> without the need to take suboptimal paths. >> >>>>>> >>>>> </t> >> >>>>>> >>>>> <t> >> >>>>>> >>>>> The NVO3 architecture should [must? or just say it does?] >> >>>>>> >>>>> support distributed gateways. Such support requires that >> >>>>>> >>>>> NVO3 control protocols include mechanisms for the >> >>>>>> >>>>> maintenance and distribution of policy information about >> >>>>>> >>>>> what type of cross-VN communication is allowed so that NVEs >> >>>>>> >>>>> acting as distributed gateways can tunnel traffic from one >> >>>>>> >>>>> VN to another as appropriate. >> >>>>>> >>>>> </t> >> >>>>>> >>>>> </section> >> >>>>>> >>>>> >> >>>>>> >>>>>Thoughts? >> >>>>>> >>>>> >> >>>>>> >>>>>Thomas >> >>>>>> >>>>> >> >>>>>> >>>>>_______________________________________________ >> >>>>>> >>>>>nvo3 mailing list >> >>>>>> >>>>>[email protected] >> >>>>>> >>>>>https://www.ietf.org/mailman/listinfo/nvo3 >> >>>>>> >>>> >> >>>>>> >>> >> >>>>>> >> >> >>>>>> >>_______________________________________________ >> >>>>>> >>nvo3 mailing list >> >>>>>> >>[email protected] >> >>>>>> >>https://www.ietf.org/mailman/listinfo/nvo3 >> >>>>>> > >> >> _______________________________________________ >> nvo3 mailing list >> [email protected] >> https://www.ietf.org/mailman/listinfo/nvo3 _______________________________________________ nvo3 mailing list [email protected] https://www.ietf.org/mailman/listinfo/nvo3
