On 29/10/13 04:41, "Xialiang (Frank)" <[email protected]> wrote:
> >> >> >> >> >> >> >Hi all, >> >> >Distributed Gateway is a very interesting topic. It would also be >> >> >valuable to solve some challenge introduced by network >>virtualization. >> >> > >> >> >To support the intra/inter-subnet connections through the same or >> >> >different NVEs in one tenant, the Distributed Gateway should >> >> >implement the bridging function and routing function at the same >>time. >> >> > >> >> >But, there are still some points need to be studied and solved. >> >> >Below are some examples (NVE1 and NVE2 are all Distributed Gateway. >> >> >Hosts of subnet >> >> >1.1.1.0/24 and 2.2.2.0/24 are attached to them): >> >> > ------- / >> >> > ------- ------ / >> >> > // \\ / >> >> > // \\ / >> >> > // \\ / >> >> > / \ / >> >> > / +--------+ +-------+\/Internet >> >> > / | | | |/\ >> >> > / | CORE |------+Gateway/ \ >> >> > / | | | | \ >> >> > | +--/----\* +-------+ | >> >> > | // \\ | >> >> >| // \\ | >> >> >| // \\ | >> >> >| +----/----+ +----\----+ | >> >> >| | | | | | >> >> >| | NVE1 | NVE2 | | >> >> >| | / \ | | / \ | | >> >> >| +-/-----\-+ +-/-----\-+ | >> >> >| / \ / \ | >> >> >| / \ / \ | >> >> >| --- ---- / \ | >> >> > | // \\ / \ / -- \ --- | >> >> > | |1.1.1.0| | | // \\ / \ | >> >> > \ | | |2.2.2.0 | | 2.2.2.0/ >> >> > \ \\ // \ / |1.1.1.0 \ // >> >> > \ --- ---- | | ---/ >> >> > \ \\ // / >> >> > \ -- / >> >> > \\ // >> >> > \\ // >> >> > \\ // >> >> > ------- ------- >> >> > >> >> >Problems 1: If we configure the same gateway address (i.e. 1.1.1.1) >> >> >for the two subnets--1.1.1.0/24 attached to NVE1 and NVE2, and there >> >> >is intra-subnet connection between them, it will result in an >> >> >gateway address collision in one subnet. If we configure different >> >> >gateway address for same subnet attached to different NVEs to solve >> >> >the address collision problem, it will bring a huge configuration >> >> >workload and waste of address space, and also it will limit the VM >> mobility. >> >> >> >> Why ? You are thinking that the default gateway has subnets >> >>configured, feeding into its routing table. It doesn't. It learns >> >>all hosts as host-routes, with optional aggregation in case the >> >>host-routing table can be aggregated on bit boundaries. Host might >> >>have mask/prefix-lengths configured, as that is how they are >> >>operating today, but in reality these denote an address-pool rather >> >>than a subnet/prefix. For IP traffic these two parts of a subnet are >> >>not interconnected by a L2 link. Surely there might be a L2 overlay >> >>spanning both NVE's, but that overlay does not switch IP packets. >> >> >> >[Frank]: Maybe my description is not so clear. What I mean is every >> >host in the subnet configures its gateway for IP forwarding. For the >> >same subnet distributed in different NVEs, Same gateway address located >> >in different NVEs can exist. When these gateways in different NVEs send >> >gratuitous ARP messages in the subnet, or several VRRP groups in same >> >subnet send VRRP messages in the subnet, the IP address collision >> >between gateways will be detected. How to solve this kind of problem? >> In the topology drawn by you i dont see any L2 interconnection between >> NVE1 and NVE2. You dont even need VRRP is this topology. You are again >> assuming a distributed subnets equals one L2 domain. It doesn't need >>to be >> like this, as long as all hosts are treated as host-routes on the NVEs >>i.e. The >> NVE is not configured for a certain TS prefix on an interface. >> It dynamically installs the host-routes based on Tses registering (via >>VQP or >> some other means) and pushes them to the NVA. The subnet is no longer a >> subnet, it is merely an address-pool. >> >[Frank]: Your can treat all the hosts as routes. But gateways are still >needed for all the hosts for inter-subnet connections. And these gateways >are configured and located in NVEs and within same L2 domain of hosts. >That's why I think the IP address collision between gateways exists. In >addition, I think there can be several options of how to treat and handle >hosts' traffic, just like the previous discussion in this thread. For >example, we have Non-IP traffic and intra-subnet traffic. I think >switching function is suitable for them. Frank, why do you keep on insisting that there is a L2 overlay connecting the NVE's for IP traffic? In my proposal there isn't. Non-IP traffic is switches across the L2 overlay, all IP traffic (including intra-subnet) is routed across the L3 overlay. NVE's will respond to all ARP's, but with the 'right' MAC address. Intersubnet traffic is send to the default gateway, which has the same identity across the entire DC. Obviously the default gateway IP address is not part of the L3 overlay routing table. Yves > > >> > >> > >> >> >> >> > >> >> >Problem 2: How to aggregate or decrease the address number learned >> >> >by NVEs or hosts? >> >> In any case you will perform the same or better than when learning >> >>MAC-addresses. With a MAC address namespace you cannot aggregate. >> >> With an IP namespace and host-route learning there is at least a >> >>change of aggregation, either at the NVE, or higher up in the data >> >>centre. And actually the aggregation function isn't really a part of >> >>the NVE, it can be a function of the NVA to aggregate address-space, >> >>if a contiguous block of host-routes are all pointing to the same >> >>NVE. >> >> >> >[Frank]: Right, I agree your analysis about address aggregation. >> >Actually, what I am concerned is the possibility to decrease or need >> >not to learn all host's address by some gateway-like method. >> > >> >> > >> >> >Problem 3: How to deal with multi-tenants? >> >> >> >> This is orthogonal. The NVE learns about hosts and their IP >> >> addresses within the NV instance. I dont see why multi tenancy is a >> problem ? >> >> >> >> > >> >> >In addition to above problems, there are maybe more points around >> >> >the Distributed Gateway issue. >> >> > >> >> >I think the distributed gateway will solve those problems. But >> >> >before we define any functions of it, we need a problem statement >> >> >document for those scenarios firstly. >> >> I think the problem statement of how the solution should look like >> >>from a user's perspective is already there. A distributed gateway, >> >>with a unified >> >> L2/L3 NVE on it, is an option to solve those problems. >> >> >> >[Frank]: It depends on how many problems we find by discussion and come >> >to the agreement. We can keep on discussing. >> Haven't seen any problem yet, as this model is implemented already. The >> biggest problem is that people dont fully understand it, and perhaps we >>need >> a separate draft to discuss this in detail. Draft-hertoghs does >>explain it, but >> the LISP machinery around it might confuse people i.e the distributed, >>unified >> L2/L3 NVE is orthogonal to the NVE-NVA CP being used. >> >[Frank] By my understanding, distributed gateways are new functions >needed to support in NVE and can work together with NVE-NVA CP well. > >> Yves >> > >> >> > >> >> >Your comments? >> >> > >> >> >B.R. >> >> >Frank >> >> > >> >> > >> >> >From: [email protected] [mailto:[email protected]] On Behalf >> >> >Of Yves Hertoghs (yhertogh) >> >> >Sent: Friday, October 25, 2013 11:05 PM >> >> >To: Lizhong Jin >> >> >Cc: 'Thomas Narten'; [email protected]; [email protected]; Lucy yong; >> >> >Xuxiaohu; Larry Kreeger (kreeger) >> >> >Subject: Re: [nvo3] Distributed Gateways [was Re: NVO3 Architecture >> >> >document] >> >> > >> >> > >> >> > >> >> >True, but the unified L2/L3 solution gives you the choice on a per >> >> >VNI basis whether you want to do 'enhanced IP forwarding' or >> >> >'Ethernet based forwarding'. >> >> > >> >> > >> >> > >> >> >I also wouldn't call RDMA over Converged Ethernet a prime use case >> >> >for a multi tenant DC. Typically this would be deployed standalone >> >> >and across a few ethernet switches. >> >> > >> >> > >> >> > >> >> >Yves >> >> > >> >> > >> >> > >> >> >From: >> >> >Lizhong Jin <[email protected]> >> >> >Date: Friday 25 October 2013 16:52 >> >> >To: Yves Hertoghs <[email protected]> >> >> >Cc: Lucy yong <[email protected]>, "[email protected]" >> >> ><[email protected]>, 'Thomas Narten' <[email protected]>, >> >> 'Xuxiaohu' >> >> ><[email protected]>, "[email protected]" <[email protected]>, "Larry >> >> >Kreeger (kreeger)" <[email protected]> >> >> >Subject: RE: [nvo3] Distributed Gateways [was Re: NVO3 Architecture >> >> >document] >> >> > >> >> > >> >> > >> >> >>Yves, >> >> >>In that scenario, I agree. But my concern is, is it realistic for a >> >> >>datacenter with only IP traffic. I am not sure. But at least as I >> >> >>know, if you want to deploy RoCE, there will be non-IP traffic. >> >> >> >> >> >>Lizhong >> >> >> >> >> >>From: Yves Hertoghs (yhertogh) [mailto:[email protected]] >> >> >> >> >> >>Sent: Friday, October 25, 2013 6:07 PM >> >> >>To: Lizhong Jin >> >> >>Cc: 'Lucy yong'; [email protected]; 'Thomas Narten'; 'Xuxiaohu'; >> >> >>[email protected]; Larry Kreeger (kreeger) >> >> >>Subject: Re: [nvo3] Distributed Gateways [was Re: NVO3 Architecture >> >> >>document] >> >> >> >> >> >> >> >> >> >> >> >>Lizhong, >> >> >> >> >> >> >> >> >> >> >> >>My main point is that , in the absence of non-IP traffic, there is >> >> >>a lot of value of doing IP based forwarding for all traffic, as >> >> >>this effectively isolates the failure domains to the L2 domains >> >> >>underneath the NVE's. >> >> >> >> >> >> >> >> >> >> >> >>Yves >> >> >> >> >> >> >> >> >> >> >> >>From: >> >> >>Lizhong Jin <[email protected]> >> >> >>Date: Friday 25 October 2013 04:33 >> >> >>To: Yves Hertoghs <[email protected]> >> >> >>Cc: Lucy yong <[email protected]>, "[email protected]" >> >> >><[email protected]>, 'Thomas Narten' <[email protected]>, >> >> 'Xuxiaohu' >> >> >><[email protected]>, "[email protected]" <[email protected]>, "Larry >> >> Kreeger >> >> >>(kreeger)" <[email protected]> >> >> >>Subject: RE: [nvo3] Distributed Gateways [was Re: NVO3 Architecture >> >> >>document] >> >> >> >> >> >> >> >> >> >> >> >>>See inline. >> >> >>> >> >> >>>From: Yves Hertoghs (yhertogh) [mailto:[email protected]] >> >> >>> >> >> >>>Sent: 2013年10月24日 >> >> >>> 23:54 >> >> >>>To: Lizhong Jin >> >> >>>Cc: Lucy yong; [email protected]; Thomas Narten; Xuxiaohu; >> >> >>>[email protected]; Larry Kreeger (kreeger) >> >> >>>Subject: Re: [nvo3] Distributed Gateways [was Re: NVO3 >> >> >>>Architecture document] >> >> >>> >> >> >>> >> >> >>> >> >> >>>Inline >> >> >>> >> >> >>> >> >> >>> >> >> >>>From: >> >> >>>Lizhong Jin <[email protected]> >> >> >>>Date: Thursday 24 October 2013 17:06 >> >> >>>To: Yves Hertoghs <[email protected]> >> >> >>>Cc: Lucy yong <[email protected]>, "[email protected]" >> >> >>><[email protected]>, Thomas Narten <[email protected]>, >> >> Xuxiaohu >> >> >>><[email protected]>, "[email protected]" <[email protected]>, "Larry >> >> >>>Kreeger (kreeger)" <[email protected]> >> >> >>>Subject: Re: [nvo3] Distributed Gateways [was Re: NVO3 >> >> >>>Architecture document] >> >> >>> >> >> >>> >> >> >>> >> >> >>>>Hi Yves, >> >> >>>>Are you referring the benefits below: >> >> >>>>Effectively this allows the 'spread' of >> >> >>>> subnets across the Datacenter(s) without leading to network >>wide >> >> >>>> broadcast and associated failure domains, while allowing free >> >> >>>> mobility of the end-stations. >> >> >>>>MAC based forwarding will also not lead to network wide >> >> >>>>broadcast, if all the MAC mapping entries are generated by NVA. >> >> >>>> >> >> >>>> >> >> >>>> >> >> >>>Since when is that a requirement for the NVA ? An NVA will store >> >> >>>the mappings, not generate it. The mapping generation can be done >> >> >>>through orchestration of the NVA, or through the NVE to NVA >> >> >>>interactions as a result of the NVE detection new Tenant Systems. >> >> >>> >> >> >>>Or are you saying that, independent of how the NVA knows about >> >> >>>them, tenant broadcast will only get flooded within the scope of >>the >> VNI? >> >> >>>If yes, why would this prevent from sending L2 broadcasts to all >> >> >>>participating NVEs, and attached TS'es? There might be some >> >> >>>optimisations in the underlay to make this work 'better' , but it >> >> >>>doesn't stop those ugly L2 broadcast packets from hitting all >>TS'es. >> >> >>>A >> >> >>> L2 overlay creates a L2 failure domain, in the context of >> >> >>>broadcast and multicast. Nothing you can do about it. >> >> >>> >> >> >>>[Lizhong] Sorry for the confusion, I mean the MAC mapping of >> >> >>>remote will be got from NVA, and MAC mapping of local will be got >> >> >>>from NVA or generated by itself. I did not invent new things, and >> >> >>>all of that is described in draft-narten-nvo3-arch-01. The NVE >> >> >>>will reply the ARP from local TS if corresponding MAC mapping is >> >> >>>already existed. For EVPN, broadcast only happen when there is no >> >> >>>MAC-IP bindings locally, and that works well. There maybe some >> >> >>>benefits existed from the multicast view, and I should think >>about it. >> >> >>> >> >> >>>Thanks >> >> >>>Lizhong >> >> >>> >> >> >>> >> >> >>> >> >> >>>>I remember this has already been discussed on NVO3 list last >>year. >> >> >>>>The only benefit I could see is, for an IP only data center, you >> >> >>>>only need to have IP forwarding. >> >> >>>>Regards >> >> >>>>Lizhong >> >> >>>>On Oct 24, 2013 7:24 PM, "Yves Hertoghs (yhertogh)" >> >> >>>><[email protected]> wrote: >> >> >>>>> >> >> >>>>> inline >> >> >>>>> >> >> >>>>> From: Lizhong Jin <[email protected]> >> >> >>>>> Date: Thursday 24 October 2013 03:54 >> >> >>>>> To: "Larry Kreeger (kreeger)" <[email protected]>, Xuxiaohu >> >> >>>>><[email protected]>, Yves Hertoghs <[email protected]>, >> Lucy >> >> >>>>>yong <[email protected]>, >> >> >>>> Thomas Narten <[email protected]> >> >> >>>>> >> >> >>>>> Cc: "[email protected]" <[email protected]>, "[email protected]" >> >> >>>>><[email protected]> >> >> >>>>> Subject: RE: [nvo3] Distributed Gateways [was Re: NVO3 >> >> >>>>>Architecture document] >> >> >>>>> >> >> >>>>>> Hi Larry, >> >> >>>>>> >> >> >>>>>> 1) A name for the service (lets's pick one, is it hybrid, >> >> >>>>>>combined or unified?) >> >> >>>>>> >> >> >>>>>> [Lizhong] IRB, which have been used before. We could change it >> >> >>>>>>to virtual IRB. >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> 2) To address it from a solution point of view (e.g. are >> >> >>>>>>packets forwarded based on MAC or based on IP when source and >> >> >>>>>>dest are in the same subnet?). >> >> >>>>>> >> >> >>>>>> [Lizhong] Yes, we should do that, and there are already some >> >> >>>>>>drafts there >> >> >>>>>>>>(http://tools.ietf.org/html/draft-jin-nvo3-virb-centralized-00). >> >> >>>>>>For IP forwarding for intra subnet traffic: >> >> >>>>>> >> >> >>>>>> 1. I don’t think it could be wildly accepted in next >>two or >> >> >>>>>>three years to have a pure IP datacenter. Pure Ethernet traffic >> >> >>>>>>and MAC forwarding will be there anyway. >> >> >>>>>> >> >> >>>>>> 2. From the dataplane point, you have to design larger >>IP >> >> >>>>>>routing table, and possibly larger TCAM (for prefix entry), and >> >> >>>>>>higher cost consequently. What’s more, should we even design >> >> >>>>>>aging for IP entry like MAC entry? >> >> >>>>>> >> >> >>>>>> 3. From the control plane point, you need to change the >> >> >>>>>>mapping entry rule, and may have new interoperable issue with >> >> >>>>>>legacy >> >> >>>>>>L2/L3 VPN. >> >> >>>>>> >> >> >>>>>> I don’t see much benefit. Could anyone give me some hints? >> >> >>>>> >> >> >>>>> [yves] read draft-hertoghs-nvo3-lisp-unified-control-plane >> >> >>>>> >> >> >>>>> Yves >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> Thanks >> >> >>>>>> >> >> >>>>>> Lizhong >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> From: Larry Kreeger (kreeger) [mailto:[email protected]] >> >> >>>> >> >> >>>>>> Sent: 2013年10月24日 >> >> >>>> 8:32 >> >> >>>>>> To: Xuxiaohu; Lizhong Jin; Yves Hertoghs (yhertogh); Lucy >> >> >>>>>>yong; Thomas Narten >> >> >>>>>> Cc: [email protected]; >> >> >>>>[email protected] <mailto:[email protected]> >> >> >>>>>> Subject: Re: 答复: [nvo3] Distributed Gateways [was Re: NVO3 >> >> >>>>>>Architecture document] >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> I've now read through this whole thread twice. I see now >> >> >>>>>>that there may be some confusion between an L2/L3 service (what >> >> >>>>>>tenant's >> >> >>>>>>see) and an L2/L3 solution (how NVO3 provides the service). >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> Prior to this thread, I may have agreed that we don't need to >> >> >>>>>>define a hybrid/combined/unified L2/L3 "service", but now I'm >> >> >>>>>>less sure. Just be be clear, I was not calling so much for a >> >> >>>>>>definition of the service, but that we need: >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> 1) A name for the service (lets's pick one, is it hybrid, >> >> >>>>>>combined or unified?) >> >> >>>>>> >> >> >>>>>> 2) To address it from a solution point of view (e.g. are >> >> >>>>>>packets forwarded based on MAC or based on IP when source and >> >> >>>>>>dest are in the same subnet?). >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> As I stated before, I believe the control plane requirements >> >> >>>>>>will be different for a combined service than for a pure L2 or >> >> >>>>>>pure L3 service, and that we if don't start specifically >> >> >>>>>>calling out the control plane requirements for a combined >> >> >>>>>>service (from a >> >> >>>> solution point of view), we will have a very low likelihood of >> >> >>>>interoperable solutions as each implementer fills in the gaps in >> >> >>>>their own way. >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> Does the WG agree that we should do 1) and 2) above? >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> I would also not object if we had a short description in the >> >> >>>>>>framework of what the combined service looks like compared to a >> >> >>>>>>pure >> >> >>>>>>L2 or pure L3 service, just so we can level set for everyone. >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> Thanks, Larry >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> From: Xuxiaohu <[email protected]> >> >> >>>>>> Date: Tuesday, October 22, 2013 11:38 PM >> >> >>>>>> To: Larry Kreeger <[email protected]>, Lizhong Jin >> >> >>>>>><[email protected]>, "Yves Hertoghs (yhertogh)" >> >> >>>>>><[email protected]>, Lucy yong <[email protected]>, >> >> >>>> Thomas Narten <[email protected]> >> >> >>>>>> Cc: "[email protected]" <[email protected]>, "[email protected]" >> >> >>>>>><[email protected]> >> >> >>>>>> Subject: 答复: [nvo3] Distributed Gateways [was Re: NVO3 >> >> >>>>>>Architecture document] >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> 发件人: >> >> >>>>[email protected] [mailto:[email protected]] >> >> >>>>代表 Larry Kreeger (kreeger) >> >> >>>>>> 发送时间: 2013年10月23日 >> >> >>>> 11:27 >> >> >>>>>> 收件人: Lizhong Jin; Yves Hertoghs (yhertogh); Lucy yong; Thomas >> >> >>>>>> Narten >> >> >>>>>> 抄送: >> >> >>>>[email protected]; >> >> >>>>[email protected] <mailto:[email protected]> >> >> >>>>>> 主题: Re: [nvo3] Distributed Gateways [was Re: NVO3 >> Architecture >> >> >>>>>>document] >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> Hi Lizong, >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> Let me give more details on my thinking. >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> A pure L2 service only requires inner MAC address to outer IP >> >> >>>>>>address mappings. When a frame is delivered between two TS, the >> >> >>>>>>source and dest MAC addresses have not been changed. >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> A pure L3 service only requires inner IP to outer IP address >> >> >>>>>>mappings. There is no expectation that two TS are on the same >> >> >>>>>>L2 segment, and the TS would always ARP for a router to reach >> >> >>>>>>another TS. When a frame is delivered between the two TS, both >> >> >>>>>>the >> >> >>>> source and dest MAC will have been rewritten. TTL will have >> >> >>>>been decremented at least once. >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> I'm expecting a hybrid L2/L3 service to preserve L2 semantics >> >> >>>>>>within a subnet. TS ARP for the other TS on its subnet instead >> >> >>>>>>of the router. The source and dest MAC are not rewritten. TTL >> >> >>>>>>is not decremented. From the perspective of the two TS on the >> >> >>>>>>same >> >> >>>> subnet, they are connected to the same L2 segment. To achieve >> >> >>>>this, the NVEs must behave differently when forwarding IP packets >> >> >>>>within the same L2 VN vs between different L2 VNs. For example, >> >> >>>>the NVE would need to respond to ARP, not with its own MAC >> >> >>>>address, but with the remote TSs MAC address if the TS are in >>the >> same L2 VN. >> >> >>>>Furthermore, as I wrote below, there needs to be a mapping of the >> >> >>>>set of L2 VNs that belong to the L3 VN. >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> If so, what’s the difference between hybrid L2/L3 service and >> >> >>>>>>IRB (Integrated Routing and Bridging)? In addition, I believe >> >> >>>>>>the hybrid >> >> >>>>>>L2/L3 service you mentioned above is different from what Yves >> >> >>>>>>meant by “combined L2/L3”. The former is about bridging >> >> >>>> for intra-subnet traffic and routing for inter-subnet traffic >> >> >>>>while the latter is about routing for (both intra-subnet and >> >> >>>>inter-subnet) IP traffic while bridging for non-IP traffic. >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> Best regards, >> >> >>>>>> >> >> >>>>>> Xiaohu >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> So, I still think NVEs will need some additional information >> >> >>>>>>for a hybrid service, and I think we will need to describe the >> >> >>>>>>behavior of a hybrid service beyond a sentence in the framework >> >> >>>>>>saying it is possible. Otherwise, I think it is quite likely >> >> >>>>>>that >> >> >>>> two different hybrid implementations will not interoperate. >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> - Larry >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> From: Lizhong Jin <[email protected]> >> >> >>>>>> Date: Tuesday, October 22, 2013 7:31 PM >> >> >>>>>> To: Larry Kreeger <[email protected]>, "Yves Hertoghs >> (yhertogh)" >> >> >>>>>><[email protected]>, Lucy yong <[email protected]>, >> Thomas >> >> >>>>>>Narten >> >> >>>> <[email protected]> >> >> >>>>>> Cc: "[email protected]" <[email protected]>, "[email protected]" >> >> >>>>>><[email protected]> >> >> >>>>>> Subject: RE: [nvo3] Distributed Gateways [was Re: NVO3 >> >> >>>>>>Architecture document] >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> > >> >> >>>>>> >With regards to the 'new service type' of a combined L2/L3 >> >> >>>>>> >NVE, >> >> >>>>>> >> >> >>>>>>>http://tools.ietf.org/html/draft-hertoghs-nvo3-lisp-controlpla >> >> >>>>>>>ne- >> >> >>>>>>>uni >> >> >>>>>>>fie >> >> >>>>>> >d-0 >> >> >>>>>> >0 >> >> >>>>>> > describes what you are referring to , but i am not sure if >> >> >>>>>> >it >> >> >>>>>>needs to >> >> >>>>>> >be called a new service-type. I see it more as a deployment >> >> >>>>>> >choice where you choose to deploy distributed gateways with >> >> >>>>>> >uniform mac/IP addressing across the DC, collocate them with >> >> >>>>>> >the >> >> >>>>>> >L2 overlay, and >> >> >>>>>>do >> >> >>>>>> >some traffic steering to make sure IP traffic gets sent to >> >> >>>>>> >the >> >> >>>>>> >L3 overlay (at all times), and non-IP traffic gets sent >> >> >>>>>> >across the L2 >> >> >>>>>> overlay. >> >> >>>>>> >> >> >>>>>> LK> I'll give you an example of why I think the protocol >> >> >>>>>>requirements will >> >> >>>>>> be different for L2 vs L3 vs a combined L2/L3 service. For an >> >> >>>>>>L2 VN, the VN needs to be identified (e.g. with a Name or ID). >> >> >>>>>>For an L3 VN, it similarly needs to be identified. For a >> >> >>>>>>combined >> >> >>>>>>L2/L3 service, I think the NVEs need to know both the L3 VN >> >> >>>>>>identity (one), and all the L2 VN identities that are part of >> >> >>>>>>the >> >> >>>>>>L3 VN. When doing distributed L3 forwarding between a TS on >> >> >>>>>>one >> >> >>>>>>L2 VN to one on another L2 VN, it will need to know not just >> >> >>>>>>the mapping of inner to outer address, but the mapping of >> >> >>>>>>inner L3 address to destination L2 VN and MAC address (so it >> >> >>>>>>can rewrite the MAC and the L2 VN). >> >> >>>>>> >> >> >>>>>> [Lizhong] I tend to agree with Yves, it is more of a >> >> >>>>>>deployment choice. >> >> >>>>>> The behavior you described for L3 forwarding between two L2 >> >> >>>>>>VPN is more of a standard L3 forwarding function, looking up >> >> >>>>>>with destination IP address, then getting the destination MAC, >> >> >>>>>>source MAC, VLAN and NVO3 tunnel from the table. The MAC >> and >> >> >>>>>>VLAN >> >> are >> >> >>>>>>resolved from the next hop within same L3 VN. We write a >> >> combined >> >> >>>>>>L2/L3 forwarding for NVO3 last year, just for reference >> >> >>>>http://tools.ietf.org/html/draft-jin-nvo3-virb-centralized-00 >> >> >>>><http://tools.ietf.org/html/draft-jin-nvo3-virb-centralized-00>. >> >> >>>>>> BTW, we may update it soon. >> >> >>>>>> >> >> >>>>>> Thanks >> >> >>>>>> Lizhong >> >> >>>>>> >> >> >>>>>> > >> >> >>>>>> >Yves >> >> >>>>>> > >> >> >>>>>> >On 21/10/13 10:28, "Larry Kreeger (kreeger)" >> >> >>>>>> ><[email protected]> >> >> >>>>>>wrote: >> >> >>>>>> > >> >> >>>>>> >>Lucy, >> >> >>>>>> >> >> >> >>>>>> >>See inline with LK2>. >> >> >>>>>> >> >> >> >>>>>> >> - Larry >> >> >>>>>> >> >> >> >>>>>> >>On 10/20/13 8:20 PM, "Lucy yong" <[email protected]> >> >> wrote: >> >> >>>>>> >> >> >> >>>>>> >>>Hi Larry, >> >> >>>>>> >>> >> >> >>>>>> >>>Please see inline with [Lucy] >> >> >>>>>> >>> >> >> >>>>>> >>>-----Original Message----- >> >> >>>>>> >>>From: Larry Kreeger (kreeger) [mailto:[email protected]] >> >> >>>>>> >>>Sent: Friday, October 18, 2013 6:39 PM >> >> >>>>>> >>>To: Lucy yong; Thomas Narten >> >> >>>>>> >>>Cc: [email protected] >> >> >>>>>> >>>Subject: Re: [nvo3] Distributed Gateways [was Re: NVO3 >> >> >>>>>>Architecture >> >> >>>>>> >>>document] >> >> >>>>>> >>> >> >> >>>>>> >>>Lucy, >> >> >>>>>> >>> >> >> >>>>>> >>>See inline with LK>. >> >> >>>>>> >>> >> >> >>>>>> >>>Thanks, Larry >> >> >>>>>> >>> >> >> >>>>>> >>>On 10/18/13 3:21 PM, "Lucy yong" <[email protected]> >> >> wrote: >> >> >>>>>> >>> >> >> >>>>>> >>>>Larry, >> >> >>>>>> >>>> >> >> >>>>>> >>>>Distributed L3 gateway is very useful and some vendors >> >> >>>>>> >>>>already implement that. >> >> >>>>>> >>> >> >> >>>>>> >>>I agree. >> >> >>>>>> >>> >> >> >>>>>> >>>>Current framework also includes L2/L3 service as Marc >> >> >>>>>> >>>>agrees to >> >> >>>>>>add >> >> >>>>>> >>>>the text I proposed although we did not explicitly define >> >> >>>>>> >>>>as a service type. >> >> >>>>>> >>> >> >> >>>>>> >>>I think we will need to because the control plane >> >> >>>>>> >>>requirements >> >> >>>>>>will >> >> >>>>>> >>>likely be different for a hybrid service. >> >> >>>>>> >>>[Lucy] Do you mean that we should define an l2/L3 service >> >> >>>>>> >>>type? I full agree and hope see more people support that >>too. >> >> >>>>>> >>>We can >> >> >>>>>>either >> >> >>>>>> >>>define it in the existing framework, or in the framework >> >> >>>>>> >>>addition draft I submitted while ago. >> >> >>>>>> >> >> >> >>>>>> >>LK2> Yes, if the group is to seriously address a hybrid >> >> >>>>>> >>LK2> L2/L3 >> >> >>>>>>service, >> >> >>>>>> >>LK2> I >> >> >>>>>> >>think we need to define it as we will need to identify its >> >> >>>>>> >>requirements independently from a pure L2 or pure L3 >>service. >> >> >>>>>> >> >> >> >>>>>> >>> >> >> >>>>>> >>>> >> >> >>>>>> >>>>Current architecture clear states NVE and NVA roles, i.e. >> >> >>>>>> >>>>NVE performs forwarding, and NVA performs routing. >> >> >>>>>> >>> >> >> >>>>>> >>>I'm not sure what you mean by routing. Are you referring to >> >> >>>>>> >>>L3 >> >> >>>>>> service? >> >> >>>>>> >>>If so, I don't agree. >> >> >>>>>> >>>[Lucy] Sorry to make you confuse. The routing here does not >> >> >>>>>> >>>mean >> >> >>>>>>L3 >> >> >>>>>> >>>services. I should state that NVE performs data plan >> >> >>>>>> >>>forwarding, >> >> >>>>>>NVA >> >> >>>>>> >>>performs control plane routing, which, IMO, it is not just >> >> >>>>>>simple DB >> >> >>>>>> >>>to have inner to outer mappings. NVA may get the routing >> >> >>>>>> >>>policy >> >> >>>>>>from >> >> >>>>>> >>>operators or customer, then interpret the policy, then >> >> >>>>>> >>>generates >> >> >>>>>>the >> >> >>>>>> >>>mapping of tenant/next-hop location and send to NVEs. An >> >> >>>>>> >>>NVE >> >> >>>>>>receives >> >> >>>>>> >>>the mapping in which, if the location is the same as >> >> >>>>>> >>>itself, it translates to a tenant/VAP mapping; If not, it >> >> >>>>>> >>>installs as an inner/outer mapping. >> >> >>>>>> >>>Thus, it works regardless whether sender tenant and >> >> >>>>>> >>>destination tenant are one the same NVE or on different >>NVEs. >> >> >>>>>> >>>The mapping >> >> >>>>>>between >> >> >>>>>> >>>tenant/location is fully controlled under NVA. In this way, >> >> >>>>>>operator >> >> >>>>>> >>>only needs to input the routing policies to NVA. NVE simply >> >> >>>>>>performs >> >> >>>>>> >>>the forwarding accordingly. This is also my view about the >> >> >>>>>> >>>SDN >> >> >>>>>>based >> >> >>>>>> >>>architecture. >> >> >>>>>> >>> >> >> >>>>>> >>>>If NVA is not able to distribute routing policy to NVE at >> >> >>>>>> >>>>all, >> >> >>>>>>I do >> >> >>>>>> >>>>not know how NVA can perform route distribution control? >> >> >>>>>> >>> >> >> >>>>>> >>>You lost me. Are you referring to a hybrid L2/L3 service? >> >> >>>>>> >>>[Lucy] Let me know above explanation help or not. No, not >> >> >>>>>>particulate >> >> >>>>>> >>>to >> >> >>>>>> >>>L2/L3 service but that certainly applies to L2/L3 service. >> >> >>>>>> >> >> >> >>>>>> >>LK2> OK, if it isn't specific to L2/L3 service, than I'd >> >> >>>>>> >>LK2> rather >> >> >>>>>>leave >> >> >>>>>> >>LK2> it >> >> >>>>>> >>out of the discussion for now. And yes, the above helped me >> >> >>>>>> >>understand what you meant. The term "routing" is very >> >> overloaded. >> >> >>>>>> >> >> >> >>>>>> >>> >> >> >>>>>> >>>> If NVA only supports simple inner to outer mappings, how >> >> >>>>>> >>>> can >> >> >>>>>>NVE >> >> >>>>>> >>>>get information to perform local forwarding? >> >> >>>>>> >>> >> >> >>>>>> >>>Inner to outer mapping resolution works fine for pure L2 or >> >> >>>>>> >>>pure >> >> >>>>>>L3 >> >> >>>>>> >>>service. Local forwarding doesnšt need mappings, the local >> >> >>>>>> >>>NVE >> >> >>>>>>knows >> >> >>>>>> >>>what VAPs the TSI are connected to. >> >> >>>>>> >>>[Lucy] does pure L3 service means an L3VPN w/o any policy? >> >> >>>>>> >>>For >> >> >>>>>>an L3 >> >> >>>>>> >>>service, you can implement it w/o policy or w policy. >> >> >>>>>> >> >> >> >>>>>> >>LK2> Yes, I was thinking of pure L3 service without adding >> >>policy. >> >> >>>>>> >>LK2> I'm >> >> >>>>>> >>pretty sure you need more than just inner to outer mappings >> >> >>>>>> >>to implement policy. >> >> >>>>>> >> >> >> >>>>>> >>>IMO: NVE is not the entity to enforce the policy. NVA is >> >> >>>>>> >>>the >> >> >>>>>>entity >> >> >>>>>> >>>to enforce the policy regardless the tenant locations. >> >> >>>>>> >>>Again, >> >> >>>>>>Network >> >> >>>>>> >>>virtualization overlay has to address how to support the >> >> >>>>>>policies and >> >> >>>>>> >>>tenant mobility. IMO: current architecture draft is vague >> >> >>>>>> >>>in or >> >> >>>>>>lacks >> >> >>>>>> >>>of describing this but it is important to architect this >> >> >>>>>> >>>when >> >> >>>>>>having >> >> >>>>>> >>>data plan and control plane separated on different >>entities. >> >> >>>>>> >> >> >> >>>>>> >>LK2> I'm not sure if I agree with you about the NVE not >> >> >>>>>> >>LK2> being the entity >> >> >>>>>> >>to enforce policy, but I think it is something better >> >> >>>>>> >>discussed >> >> >>>>>>in a >> >> >>>>>> >>conversation (not email). >> >> >>>>>> >> >> >> >>>>>> >>> >> >> >>>>>> >>>Thanks, >> >> >>>>>> >>>Lucy >> >> >>>>>> >>> >> >> >>>>>> >>>> >> >> >>>>>> >>>>Thanks, >> >> >>>>>> >>>>Lucy >> >> >>>>>> >>>> >> >> >>>>>> >>>>-----Original Message----- >> >> >>>>>> >>>>From: Larry Kreeger (kreeger) [mailto:[email protected]] >> >> >>>>>> >>>>Sent: Friday, October 18, 2013 5:00 PM >> >> >>>>>> >>>>To: Thomas Narten; Lucy yong >> >> >>>>>> >>>>Cc: [email protected] >> >> >>>>>> >>>>Subject: Re: [nvo3] Distributed Gateways [was Re: NVO3 >> >> >>>>>>Architecture >> >> >>>>>> >>>>document] >> >> >>>>>> >>>> >> >> >>>>>> >>>>Hi Thomas and Lucy, >> >> >>>>>> >>>> >> >> >>>>>> >>>>The WG needs to think hard about this one. >> >> >>>>>> >>>> >> >> >>>>>> >>>>Support of a distributed L3 gateway function between L2 >> >> >>>>>> >>>>VNs is a significant increase in scope of the NVA, and the >> >> >>>>>> >>>>NVE to NVA >> >> >>>>>>protocol. >> >> >>>>>> >>>>Where we had previously stated L2 service or L3 service >> >> >>>>>> >>>>and >> >> >>>>>>pretty >> >> >>>>>> >>>>much left a combined L2/L3 service as an exercise for the >> >> >>>>>>reader, we >> >> >>>>>> >>>>would now be adding whatever mechanisms are needed to >> the >> >> >>>>>>protocols. >> >> >>>>>> >>>>We will need to add cases for >> >> >>>>>> >>>>L2 service, L3 service and L2/L3 service. We no longer >> >> >>>>>> >>>>have >> >> >>>>>>simple >> >> >>>>>> >>>>inner to outer mappings, but now need NVEs to do MAC >> >> >>>>>> >>>>rewrites, >> >> >>>>>>local >> >> >>>>>> >>>>NVE ARP termination, and multiple lookups depending on >> the >> >> >>>>>> >>>>destination MAC address (first L2, then potentially L3). >> >> >>>>>> >>>>We will also need to distribute two different VN >> >> >>>>>> >>>>identifiers (one for >> >> >>>>>>L2 and >> >> >>>>>> >>>>one for L3), and somehow convey the containment >> >> >>>>>> >>>>relationship >> >> >>>>>>between >> >> >>>>>> >>>>the two (multiple L2 VNs within one >> >> >>>>>> >>>>L3 VN). While I think this is all very useful, I just want >> >> >>>>>> >>>>to >> >> >>>>>>make >> >> >>>>>> >>>>sure the WG agrees to this since I feel it is a >> >> >>>>>> >>>>significant change/increase in scope from my perspective. >> >> >>>>>> >>>> >> >> >>>>>> >>>>Thanks, Larry >> >> >>>>>> >>>> >> >> >>>>>> >>>> >> >> >>>>>> >>>> >> >> >>>>>> >>>>On 10/18/13 2:52 PM, "Thomas Narten" >> <[email protected]> >> >> wrote: >> >> >>>>>> >>>> >> >> >>>>>> >>>>>Hi Lucy. >> >> >>>>>> >>>>> >> >> >>>>>> >>>>>Lucy yong <[email protected]> writes: >> >> >>>>>> >>>>> >> >> >>>>>> >>>>>> Section 5.3 describes gateways. IMO: it misses an >> >> >>>>>> >>>>>> important >> >> >>>>>>use >> >> >>>>>> >>>>>> case. A Gateway, say overlay gateway, may be used to >> >> >>>>>>interconnect >> >> >>>>>> >>>>>> two or more overlay VNs. In this case, the traffic >> >> >>>>>> >>>>>> traversing between two overlay VNs must go through the >> >> >>>>>> >>>>>> gateway where the policy can be enforced. Furthermore, >> >> >>>>>> >>>>>> it is possible to >> >> >>>>>>implement >> >> >>>>>> >>>>>> centralized or distributed overlay gateway. The latter >> >> >>>>>> >>>>>> has overlay gateway function implemented on NVEs. Thus, >> >> >>>>>> >>>>>> it >> >> >>>>>>requests >> >> >>>>>> >>>>>> the cross-VN policies to be distributed to NVEs. >> >> >>>>>> >>>>> >> >> >>>>>> >>>>>> Current section seems very focus on overlay VN >> >> >>>>>> >>>>>> interconnect a non-overlay network, which centralized >> >> >>>>>> >>>>>> gateway architecture >> >> >>>>>>is >> >> >>>>>> >>>>>> practical. But in overlay networks, both centralized or >> >> >>>>>> >>>>>> distributed are possible and depend on the >>applications. >> >> >>>>>> >>>>> >> >> >>>>>> >>>>>Agreed. I propose adding a new section after 5.3 that >>says: >> >> >>>>>> >>>>> >> >> >>>>>> >>>>> <section title="Distributed Gateways"> <t> The relaying >> >> >>>>>> >>>>> of traffic from one VN to another deserves special >> >> >>>>>> >>>>> consideration. The previous section described gateways >> >> >>>>>> >>>>> performing this function. If such gateways are >> >> >>>>>> >>>>> centralized, traffic between TSes on different VNs can >> >> >>>>>> >>>>> take suboptimal paths, i.e., triangular routing results >> >> >>>>>> >>>>> in paths that always traverse the gateway. As an >> >> >>>>>> >>>>> optimization, individual NVEs can be part of a >> >> >>>>>> >>>>> distributed gateway that performs such relaying, >> >> >>>>>> >>>>> reducing or completely eliminating triangular routing. >> >> >>>>>> >>>>> In a distributed gateway, each ingress NVE can perform >> >> >>>>>> >>>>> such relaying activity directly, so long as it has >> >> >>>>>> >>>>> access to the policy information needed to determine >> >> >>>>>> >>>>> whether cross-VN communication is allowed. Having >> >> >>>>>> >>>>> individual NVEs be part of a distributed gateway allows >> >> >>>>>> >>>>> them to tunnel traffic directly to the destination NVE >> without the need to take suboptimal paths. >> >> >>>>>> >>>>> </t> >> >> >>>>>> >>>>> <t> >> >> >>>>>> >>>>> The NVO3 architecture should [must? or just say it >> >> >>>>>> >>>>> does?] support distributed gateways. Such support >> >> >>>>>> >>>>> requires that >> >> >>>>>> >>>>> NVO3 control protocols include mechanisms for the >> >> >>>>>> >>>>> maintenance and distribution of policy information about >> >> >>>>>> >>>>> what type of cross-VN communication is allowed so that >> >> >>>>>> >>>>> NVEs acting as distributed gateways can tunnel traffic >> >> >>>>>> >>>>> from one VN to another as appropriate. >> >> >>>>>> >>>>> </t> >> >> >>>>>> >>>>> </section> >> >> >>>>>> >>>>> >> >> >>>>>> >>>>>Thoughts? >> >> >>>>>> >>>>> >> >> >>>>>> >>>>>Thomas >> >> >>>>>> >>>>> >> >> >>>>>> >>>>>_______________________________________________ >> >> >>>>>> >>>>>nvo3 mailing list >> >> >>>>>> >>>>>[email protected] >> >> >>>>>> >>>>>https://www.ietf.org/mailman/listinfo/nvo3 >> >> >>>>>> >>>> >> >> >>>>>> >>> >> >> >>>>>> >> >> >> >>>>>> >>_______________________________________________ >> >> >>>>>> >>nvo3 mailing list >> >> >>>>>> >>[email protected] >> >> >>>>>> >>https://www.ietf.org/mailman/listinfo/nvo3 >> >> >>>>>> > >> >> >> >> _______________________________________________ >> >> nvo3 mailing list >> >> [email protected] >> >> https://www.ietf.org/mailman/listinfo/nvo3 >> >> _______________________________________________ >> nvo3 mailing list >> [email protected] >> https://www.ietf.org/mailman/listinfo/nvo3 >_______________________________________________ >nvo3 mailing list >[email protected] >https://www.ietf.org/mailman/listinfo/nvo3 _______________________________________________ nvo3 mailing list [email protected] https://www.ietf.org/mailman/listinfo/nvo3
