Inline. From: "Xialiang (C)" <[email protected]> Date: Saturday 26 October 2013 11:42 To: Yves Hertoghs <[email protected]>, Lizhong Jin <[email protected]> Cc: 'Thomas Narten' <[email protected]>, "[email protected]" <[email protected]>, "[email protected]" <[email protected]>, Lucy yong <[email protected]>, Xuxiaohu <[email protected]>, "Larry Kreeger (kreeger)" <[email protected]> Subject: RE: [nvo3] Distributed Gateways [was Re: NVO3 Architecture document]
>Hi all, >Distributed Gateway is a very interesting topic. It would also be >valuable to solve some challenge introduced by network virtualization. > >To support the intra/inter-subnet connections through the same or >different NVEs in one tenant, the Distributed Gateway should implement >the bridging function and routing function at the same time. > >But, there are still some points need to be studied and solved. Below are >some examples (NVE1 and NVE2 are all Distributed Gateway. Hosts of subnet >1.1.1.0/24 and 2.2.2.0/24 are attached to them): > ------- / > ------- ------ / > // \\ / > // \\ / > // \\ / > / \ / > / +--------+ +-------+\/Internet > / | | | |/\ > / | CORE |------+Gateway/ \ > / | | | | \ > | +--/----\* +-------+ | > | // \\ | >| // \\ | >| // \\ | >| +----/----+ +----\----+ | >| | | | | | >| | NVE1 | NVE2 | | >| | / \ | | / \ | | >| +-/-----\-+ +-/-----\-+ | >| / \ / \ | >| / \ / \ | >| --- ---- / \ | > | // \\ / \ / -- \ --- | > | |1.1.1.0| | | // \\ / \ | > \ | | |2.2.2.0 | | 2.2.2.0/ > \ \\ // \ / |1.1.1.0 \ // > \ --- ---- | | ---/ > \ \\ // / > \ -- / > \\ // > \\ // > \\ // > ------- ------- > >Problems 1: If we configure the same gateway address (i.e. 1.1.1.1) for >the two subnets--1.1.1.0/24 attached to NVE1 and NVE2, and there is >intra-subnet connection between them, it will result in an gateway >address collision > in one subnet. If we configure different gateway address for same subnet >attached to different NVEs to solve the address collision problem, it >will bring a huge configuration workload and waste of address space, and >also it will limit the VM mobility. Why ? You are thinking that the default gateway has subnets configured, feeding into its routing table. It doesn't. It learns all hosts as host-routes, with optional aggregation in case the host-routing table can be aggregated on bit boundaries. Host might have mask/prefix-lengths configured, as that is how they are operating today, but in reality these denote an address-pool rather than a subnet/prefix. For IP traffic these two parts of a subnet are not interconnected by a L2 link. Surely there might be a L2 overlay spanning both NVE's, but that overlay does not switch IP packets. > >Problem 2: How to aggregate or decrease the address number learned by >NVEs or hosts? In any case you will perform the same or better than when learning MAC-addresses. With a MAC address namespace you cannot aggregate. With an IP namespace and host-route learning there is at least a change of aggregation, either at the NVE, or higher up in the data centre. And actually the aggregation function isn't really a part of the NVE, it can be a function of the NVA to aggregate address-space, if a contiguous block of host-routes are all pointing to the same NVE. > >Problem 3: How to deal with multi-tenants? This is orthogonal. The NVE learns about hosts and their IP addresses within the NV instance. I dont see why multi tenancy is a problem ? > >In addition to above problems, there are maybe more points around the >Distributed Gateway issue. > >I think the distributed gateway will solve those problems. But before we >define any functions of it, we need a problem statement document for >those scenarios firstly. I think the problem statement of how the solution should look like from a user's perspective is already there. A distributed gateway, with a unified L2/L3 NVE on it, is an option to solve those problems. > >Your comments? > >B.R. >Frank > > >From: [email protected] [mailto:[email protected]] >On Behalf Of Yves Hertoghs (yhertogh) >Sent: Friday, October 25, 2013 11:05 PM >To: Lizhong Jin >Cc: 'Thomas Narten'; [email protected]; [email protected]; Lucy yong; >Xuxiaohu; Larry Kreeger (kreeger) >Subject: Re: [nvo3] Distributed Gateways [was Re: NVO3 Architecture >document] > > > >True, but the unified L2/L3 solution gives you the choice on a per VNI >basis whether you want to do 'enhanced IP forwarding' or 'Ethernet based >forwarding'. > > > >I also wouldn't call RDMA over Converged Ethernet a prime use case for a >multi tenant DC. Typically this would be deployed standalone and across > a few ethernet switches. > > > >Yves > > > >From: >Lizhong Jin <[email protected]> >Date: Friday 25 October 2013 16:52 >To: Yves Hertoghs <[email protected]> >Cc: Lucy yong <[email protected]>, "[email protected]" ><[email protected]>, 'Thomas Narten' <[email protected]>, > 'Xuxiaohu' <[email protected]>, "[email protected]" <[email protected]>, >"Larry Kreeger (kreeger)" <[email protected]> >Subject: RE: [nvo3] Distributed Gateways [was Re: NVO3 Architecture >document] > > > >>Yves, >>In that scenario, I agree. But my concern is, is it realistic for a >>datacenter with only IP traffic. I am not sure. But at least as I know, >>if >> you want to deploy RoCE, there will be non-IP traffic. >> >>Lizhong >> >>From: Yves Hertoghs (yhertogh) [mailto:[email protected]] >> >>Sent: Friday, October 25, 2013 6:07 PM >>To: Lizhong Jin >>Cc: 'Lucy yong'; [email protected]; 'Thomas Narten'; 'Xuxiaohu'; >>[email protected]; Larry Kreeger (kreeger) >>Subject: Re: [nvo3] Distributed Gateways [was Re: NVO3 Architecture >>document] >> >> >> >>Lizhong, >> >> >> >>My main point is that , in the absence of non-IP traffic, there is a lot >>of value of doing IP based forwarding for all traffic, as this >>effectively >> isolates the failure domains to the L2 domains underneath the NVE's. >> >> >> >>Yves >> >> >> >>From: >>Lizhong Jin <[email protected]> >>Date: Friday 25 October 2013 04:33 >>To: Yves Hertoghs <[email protected]> >>Cc: Lucy yong <[email protected]>, "[email protected]" >><[email protected]>, 'Thomas Narten' <[email protected]>, >> 'Xuxiaohu' <[email protected]>, "[email protected]" <[email protected]>, >>"Larry Kreeger (kreeger)" <[email protected]> >>Subject: RE: [nvo3] Distributed Gateways [was Re: NVO3 Architecture >>document] >> >> >> >>>See inline. >>> >>>From: Yves Hertoghs (yhertogh) [mailto:[email protected]] >>> >>>Sent: 2013年10月24日 >>> 23:54 >>>To: Lizhong Jin >>>Cc: Lucy yong; [email protected]; Thomas Narten; Xuxiaohu; >>>[email protected]; Larry Kreeger (kreeger) >>>Subject: Re: [nvo3] Distributed Gateways [was Re: NVO3 Architecture >>>document] >>> >>> >>> >>>Inline >>> >>> >>> >>>From: >>>Lizhong Jin <[email protected]> >>>Date: Thursday 24 October 2013 17:06 >>>To: Yves Hertoghs <[email protected]> >>>Cc: Lucy yong <[email protected]>, "[email protected]" >>><[email protected]>, Thomas Narten <[email protected]>, >>> Xuxiaohu <[email protected]>, "[email protected]" <[email protected]>, >>>"Larry Kreeger (kreeger)" <[email protected]> >>>Subject: Re: [nvo3] Distributed Gateways [was Re: NVO3 Architecture >>>document] >>> >>> >>> >>>>Hi Yves, >>>>Are you referring the benefits below: >>>>Effectively this allows the 'spread' of >>>> subnets across the Datacenter(s) without leading to network wide >>>> broadcast and associated failure domains, while allowing free >>>> mobility of the end-stations. >>>>MAC based forwarding will also not lead to network wide broadcast, if >>>>all the MAC mapping entries are generated by NVA. >>>> >>>> >>>> >>>Since when is that a requirement for the NVA ? An NVA will store the >>>mappings, not generate it. The mapping generation can be done through >>>orchestration >>> of the NVA, or through the NVE to NVA interactions as a result of the >>>NVE detection new Tenant Systems. >>> >>>Or are you saying that, independent of how the NVA knows about them, >>>tenant broadcast will only get flooded within the scope of the VNI? If >>>yes, >>> why would this prevent from sending L2 broadcasts to all participating >>>NVEs, and attached TS'es? There might be some optimisations in the >>>underlay to make this work 'better' , but it doesn't stop those ugly L2 >>>broadcast packets from hitting all TS'es. A >>> L2 overlay creates a L2 failure domain, in the context of broadcast >>>and multicast. Nothing you can do about it. >>> >>>[Lizhong] Sorry for the confusion, I mean the MAC mapping of remote >>>will be got from NVA, and MAC mapping of local will be got from NVA or >>>generated >>> by itself. I did not invent new things, and all of that is described >>>in draft-narten-nvo3-arch-01. The NVE will reply the ARP from local TS >>>if corresponding MAC mapping is already existed. For EVPN, broadcast >>>only happen when there is no MAC-IP bindings locally, >>> and that works well. There maybe some benefits existed from the >>>multicast view, and I should think about it. >>> >>>Thanks >>>Lizhong >>> >>> >>> >>>>I remember this has already been discussed on NVO3 list last year. >>>>The only benefit I could see is, for an IP only data center, you only >>>>need to have IP forwarding. >>>>Regards >>>>Lizhong >>>>On Oct 24, 2013 7:24 PM, "Yves Hertoghs (yhertogh)" >>>><[email protected]> wrote: >>>>> >>>>> inline >>>>> >>>>> From: Lizhong Jin <[email protected]> >>>>> Date: Thursday 24 October 2013 03:54 >>>>> To: "Larry Kreeger (kreeger)" <[email protected]>, Xuxiaohu >>>>><[email protected]>, Yves Hertoghs <[email protected]>, Lucy yong >>>>><[email protected]>, >>>> Thomas Narten <[email protected]> >>>>> >>>>> Cc: "[email protected]" <[email protected]>, "[email protected]" >>>>><[email protected]> >>>>> Subject: RE: [nvo3] Distributed Gateways [was Re: NVO3 Architecture >>>>>document] >>>>> >>>>>> Hi Larry, >>>>>> >>>>>> 1) A name for the service (lets's pick one, is it hybrid, combined >>>>>>or unified?) >>>>>> >>>>>> [Lizhong] IRB, which have been used before. We could change it to >>>>>>virtual IRB. >>>>>> >>>>>> >>>>>> >>>>>> 2) To address it from a solution point of view (e.g. are packets >>>>>>forwarded based on MAC or based on IP when source and dest are in >>>>>>the same subnet?). >>>>>> >>>>>> [Lizhong] Yes, we should do that, and there are already some drafts >>>>>>there >>>>>>(http://tools.ietf.org/html/draft-jin-nvo3-virb-centralized-00). For >>>>>>IP forwarding for intra subnet traffic: >>>>>> >>>>>> 1. I don’t think it could be wildly accepted in next two or >>>>>>three years to have a pure IP datacenter. Pure Ethernet traffic and >>>>>>MAC forwarding will be there anyway. >>>>>> >>>>>> 2. From the dataplane point, you have to design larger IP >>>>>>routing table, and possibly larger TCAM (for prefix entry), and >>>>>>higher cost consequently. What’s more, should we even design aging >>>>>>for IP entry like MAC entry? >>>>>> >>>>>> 3. From the control plane point, you need to change the >>>>>>mapping entry rule, and may have new interoperable issue with legacy >>>>>>L2/L3 VPN. >>>>>> >>>>>> I don’t see much benefit. Could anyone give me some hints? >>>>> >>>>> [yves] read draft-hertoghs-nvo3-lisp-unified-control-plane >>>>> >>>>> Yves >>>>>> >>>>>> >>>>>> >>>>>> Thanks >>>>>> >>>>>> Lizhong >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> From: Larry Kreeger (kreeger) [mailto:[email protected]] >>>> >>>>>> Sent: 2013年10月24日 >>>> 8:32 >>>>>> To: Xuxiaohu; Lizhong Jin; Yves Hertoghs (yhertogh); Lucy yong; >>>>>>Thomas Narten >>>>>> Cc: [email protected]; >>>>[email protected] <mailto:[email protected]> >>>>>> Subject: Re: 答复: [nvo3] Distributed Gateways [was Re: NVO3 >>>>>>Architecture document] >>>>>> >>>>>> >>>>>> >>>>>> I've now read through this whole thread twice. I see now that >>>>>>there may be some confusion between an L2/L3 service (what tenant's >>>>>>see) and an L2/L3 solution (how NVO3 provides the service). >>>>>> >>>>>> >>>>>> >>>>>> Prior to this thread, I may have agreed that we don't need to >>>>>>define a hybrid/combined/unified L2/L3 "service", but now I'm less >>>>>>sure. Just be be clear, I was not calling so much for a definition >>>>>>of the service, but that we need: >>>>>> >>>>>> >>>>>> >>>>>> 1) A name for the service (lets's pick one, is it hybrid, combined >>>>>>or unified?) >>>>>> >>>>>> 2) To address it from a solution point of view (e.g. are packets >>>>>>forwarded based on MAC or based on IP when source and dest are in >>>>>>the same subnet?). >>>>>> >>>>>> >>>>>> >>>>>> As I stated before, I believe the control plane requirements will >>>>>>be different for a combined service than for a pure L2 or pure L3 >>>>>>service, and that we if don't start specifically calling out the >>>>>>control plane requirements for a combined service (from a >>>> solution point of view), we will have a very low likelihood of >>>>interoperable solutions as each implementer fills in the gaps in >>>>their own way. >>>>>> >>>>>> >>>>>> >>>>>> Does the WG agree that we should do 1) and 2) above? >>>>>> >>>>>> >>>>>> >>>>>> I would also not object if we had a short description in the >>>>>>framework of what the combined service looks like compared to a pure >>>>>>L2 or pure L3 service, just so we can level set for everyone. >>>>>> >>>>>> >>>>>> >>>>>> Thanks, Larry >>>>>> >>>>>> >>>>>> >>>>>> From: Xuxiaohu <[email protected]> >>>>>> Date: Tuesday, October 22, 2013 11:38 PM >>>>>> To: Larry Kreeger <[email protected]>, Lizhong Jin >>>>>><[email protected]>, "Yves Hertoghs (yhertogh)" >>>>>><[email protected]>, Lucy yong <[email protected]>, >>>> Thomas Narten <[email protected]> >>>>>> Cc: "[email protected]" <[email protected]>, "[email protected]" >>>>>><[email protected]> >>>>>> Subject: 答复: [nvo3] Distributed Gateways [was Re: NVO3 Architecture >>>>>>document] >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> 发件人: >>>>[email protected] [mailto:[email protected]] >>>>代表 Larry Kreeger (kreeger) >>>>>> 发送时间: 2013年10月23日 >>>> 11:27 >>>>>> 收件人: Lizhong Jin; Yves Hertoghs (yhertogh); Lucy yong; Thomas Narten >>>>>> 抄送: >>>>[email protected]; >>>>[email protected] <mailto:[email protected]> >>>>>> 主题: Re: [nvo3] Distributed Gateways [was Re: NVO3 Architecture >>>>>>document] >>>>>> >>>>>> >>>>>> >>>>>> Hi Lizong, >>>>>> >>>>>> >>>>>> >>>>>> Let me give more details on my thinking. >>>>>> >>>>>> >>>>>> >>>>>> A pure L2 service only requires inner MAC address to outer IP >>>>>>address mappings. When a frame is delivered between two TS, the >>>>>>source and dest MAC addresses have not been changed. >>>>>> >>>>>> >>>>>> >>>>>> A pure L3 service only requires inner IP to outer IP address >>>>>>mappings. There is no expectation that two TS are on the same L2 >>>>>>segment, and the TS would always ARP for a router to reach another >>>>>>TS. When a frame is delivered between the two TS, both the >>>> source and dest MAC will have been rewritten. TTL will have been >>>>decremented at least once. >>>>>> >>>>>> >>>>>> >>>>>> I'm expecting a hybrid L2/L3 service to preserve L2 semantics >>>>>>within a subnet. TS ARP for the other TS on its subnet instead of >>>>>>the router. The source and dest MAC are not rewritten. TTL is not >>>>>>decremented. From the perspective of the two TS on the same >>>> subnet, they are connected to the same L2 segment. To achieve this, >>>>the NVEs must behave differently when forwarding IP packets within the >>>>same L2 VN vs between different L2 VNs. For example, the NVE would >>>>need to respond to ARP, not with its own MAC address, >>>> but with the remote TSs MAC address if the TS are in the same L2 VN. >>>>Furthermore, as I wrote below, there needs to be a mapping of the set >>>>of L2 VNs that belong to the L3 VN. >>>>>> >>>>>> >>>>>> >>>>>> If so, what’s the difference between hybrid L2/L3 service and IRB >>>>>>(Integrated Routing and Bridging)? In addition, I believe the hybrid >>>>>>L2/L3 service you mentioned above is different from what Yves meant >>>>>>by “combined L2/L3”. The former is about bridging >>>> for intra-subnet traffic and routing for inter-subnet traffic while >>>>the latter is about routing for (both intra-subnet and inter-subnet) >>>>IP traffic while bridging for non-IP traffic. >>>>>> >>>>>> >>>>>> >>>>>> Best regards, >>>>>> >>>>>> Xiaohu >>>>>> >>>>>> >>>>>> >>>>>> So, I still think NVEs will need some additional information for a >>>>>>hybrid service, and I think we will need to describe the behavior of >>>>>>a hybrid service beyond a sentence in the framework saying it is >>>>>>possible. Otherwise, I think it is quite likely that >>>> two different hybrid implementations will not interoperate. >>>>>> >>>>>> >>>>>> >>>>>> - Larry >>>>>> >>>>>> >>>>>> >>>>>> From: Lizhong Jin <[email protected]> >>>>>> Date: Tuesday, October 22, 2013 7:31 PM >>>>>> To: Larry Kreeger <[email protected]>, "Yves Hertoghs (yhertogh)" >>>>>><[email protected]>, Lucy yong <[email protected]>, Thomas Narten >>>> <[email protected]> >>>>>> Cc: "[email protected]" <[email protected]>, "[email protected]" >>>>>><[email protected]> >>>>>> Subject: RE: [nvo3] Distributed Gateways [was Re: NVO3 Architecture >>>>>>document] >>>>>> >>>>>> >>>>>> >>>>>> > >>>>>> >With regards to the 'new service type' of a combined L2/L3 NVE, >>>>>> >>>>>>>http://tools.ietf.org/html/draft-hertoghs-nvo3-lisp-controlplane-uni >>>>>>>fie >>>>>> >d-0 >>>>>> >0 >>>>>> > describes what you are referring to , but i am not sure if it >>>>>>needs to >>>>>> >be called a new service-type. I see it more as a deployment choice >>>>>> >where you choose to deploy distributed gateways with uniform mac/IP >>>>>> >addressing across the DC, collocate them with the L2 overlay, and >>>>>>do >>>>>> >some traffic steering to make sure IP traffic gets sent to the L3 >>>>>> >overlay (at all times), and non-IP traffic gets sent across the L2 >>>>>> overlay. >>>>>> >>>>>> LK> I'll give you an example of why I think the protocol >>>>>>requirements will >>>>>> be different for L2 vs L3 vs a combined L2/L3 service. For an L2 >>>>>>VN, the >>>>>> VN needs to be identified (e.g. with a Name or ID). For an L3 VN, it >>>>>> similarly needs to be identified. For a combined L2/L3 service, I >>>>>>think >>>>>> the NVEs need to know both the L3 VN identity (one), and all the L2 >>>>>>VN >>>>>> identities that are part of the L3 VN. When doing distributed L3 >>>>>> forwarding between a TS on one L2 VN to one on another L2 VN, it >>>>>>will need >>>>>> to know not just the mapping of inner to outer address, but the >>>>>>mapping of >>>>>> inner L3 address to destination L2 VN and MAC address (so it can >>>>>>rewrite >>>>>> the MAC and the L2 VN). >>>>>> >>>>>> [Lizhong] I tend to agree with Yves, it is more of a deployment >>>>>>choice. >>>>>> The behavior you described for L3 forwarding between two L2 VPN is >>>>>>more of >>>>>> a standard L3 forwarding function, looking up with destination IP >>>>>>address, >>>>>> then getting the destination MAC, source MAC, VLAN and NVO3 tunnel >>>>>>from >>>>>> the table. The MAC and VLAN are resolved from the next hop within >>>>>>same L3 >>>>>> VN. We write a combined L2/L3 forwarding for NVO3 last year, just >>>>>>for >>>>>> reference >>>>http://tools.ietf.org/html/draft-jin-nvo3-virb-centralized-00 >>>><http://tools.ietf.org/html/draft-jin-nvo3-virb-centralized-00>. >>>>>> BTW, we may update it soon. >>>>>> >>>>>> Thanks >>>>>> Lizhong >>>>>> >>>>>> > >>>>>> >Yves >>>>>> > >>>>>> >On 21/10/13 10:28, "Larry Kreeger (kreeger)" <[email protected]> >>>>>>wrote: >>>>>> > >>>>>> >>Lucy, >>>>>> >> >>>>>> >>See inline with LK2>. >>>>>> >> >>>>>> >> - Larry >>>>>> >> >>>>>> >>On 10/20/13 8:20 PM, "Lucy yong" <[email protected]> wrote: >>>>>> >> >>>>>> >>>Hi Larry, >>>>>> >>> >>>>>> >>>Please see inline with [Lucy] >>>>>> >>> >>>>>> >>>-----Original Message----- >>>>>> >>>From: Larry Kreeger (kreeger) [mailto:[email protected]] >>>>>> >>>Sent: Friday, October 18, 2013 6:39 PM >>>>>> >>>To: Lucy yong; Thomas Narten >>>>>> >>>Cc: [email protected] >>>>>> >>>Subject: Re: [nvo3] Distributed Gateways [was Re: NVO3 >>>>>>Architecture >>>>>> >>>document] >>>>>> >>> >>>>>> >>>Lucy, >>>>>> >>> >>>>>> >>>See inline with LK>. >>>>>> >>> >>>>>> >>>Thanks, Larry >>>>>> >>> >>>>>> >>>On 10/18/13 3:21 PM, "Lucy yong" <[email protected]> wrote: >>>>>> >>> >>>>>> >>>>Larry, >>>>>> >>>> >>>>>> >>>>Distributed L3 gateway is very useful and some vendors already >>>>>> >>>>implement that. >>>>>> >>> >>>>>> >>>I agree. >>>>>> >>> >>>>>> >>>>Current framework also includes L2/L3 service as Marc agrees to >>>>>>add >>>>>> >>>>the text I proposed although we did not explicitly define as a >>>>>> >>>>service type. >>>>>> >>> >>>>>> >>>I think we will need to because the control plane requirements >>>>>>will >>>>>> >>>likely be different for a hybrid service. >>>>>> >>>[Lucy] Do you mean that we should define an l2/L3 service type? I >>>>>> >>>full agree and hope see more people support that too. We can >>>>>>either >>>>>> >>>define it in the existing framework, or in the framework addition >>>>>> >>>draft I submitted while ago. >>>>>> >> >>>>>> >>LK2> Yes, if the group is to seriously address a hybrid L2/L3 >>>>>>service, >>>>>> >>LK2> I >>>>>> >>think we need to define it as we will need to identify its >>>>>> >>requirements independently from a pure L2 or pure L3 service. >>>>>> >> >>>>>> >>> >>>>>> >>>> >>>>>> >>>>Current architecture clear states NVE and NVA roles, i.e. NVE >>>>>> >>>>performs forwarding, and NVA performs routing. >>>>>> >>> >>>>>> >>>I'm not sure what you mean by routing. Are you referring to L3 >>>>>> service? >>>>>> >>>If so, I don't agree. >>>>>> >>>[Lucy] Sorry to make you confuse. The routing here does not mean >>>>>>L3 >>>>>> >>>services. I should state that NVE performs data plan forwarding, >>>>>>NVA >>>>>> >>>performs control plane routing, which, IMO, it is not just >>>>>>simple DB >>>>>> >>>to have inner to outer mappings. NVA may get the routing policy >>>>>>from >>>>>> >>>operators or customer, then interpret the policy, then generates >>>>>>the >>>>>> >>>mapping of tenant/next-hop location and send to NVEs. An NVE >>>>>>receives >>>>>> >>>the mapping in which, if the location is the same as itself, it >>>>>> >>>translates to a tenant/VAP mapping; If not, it installs as an >>>>>> >>>inner/outer mapping. >>>>>> >>>Thus, it works regardless whether sender tenant and destination >>>>>> >>>tenant are one the same NVE or on different NVEs. The mapping >>>>>>between >>>>>> >>>tenant/location is fully controlled under NVA. In this way, >>>>>>operator >>>>>> >>>only needs to input the routing policies to NVA. NVE simply >>>>>>performs >>>>>> >>>the forwarding accordingly. This is also my view about the SDN >>>>>>based >>>>>> >>>architecture. >>>>>> >>> >>>>>> >>>>If NVA is not able to distribute routing policy to NVE at all, >>>>>>I do >>>>>> >>>>not know how NVA can perform route distribution control? >>>>>> >>> >>>>>> >>>You lost me. Are you referring to a hybrid L2/L3 service? >>>>>> >>>[Lucy] Let me know above explanation help or not. No, not >>>>>>particulate >>>>>> >>>to >>>>>> >>>L2/L3 service but that certainly applies to L2/L3 service. >>>>>> >> >>>>>> >>LK2> OK, if it isn't specific to L2/L3 service, than I'd rather >>>>>>leave >>>>>> >>LK2> it >>>>>> >>out of the discussion for now. And yes, the above helped me >>>>>> >>understand what you meant. The term "routing" is very overloaded. >>>>>> >> >>>>>> >>> >>>>>> >>>> If NVA only supports simple inner to outer mappings, how can >>>>>>NVE >>>>>> >>>>get information to perform local forwarding? >>>>>> >>> >>>>>> >>>Inner to outer mapping resolution works fine for pure L2 or pure >>>>>>L3 >>>>>> >>>service. Local forwarding doesnšt need mappings, the local NVE >>>>>>knows >>>>>> >>>what VAPs the TSI are connected to. >>>>>> >>>[Lucy] does pure L3 service means an L3VPN w/o any policy? For >>>>>>an L3 >>>>>> >>>service, you can implement it w/o policy or w policy. >>>>>> >> >>>>>> >>LK2> Yes, I was thinking of pure L3 service without adding policy. >>>>>> >>LK2> I'm >>>>>> >>pretty sure you need more than just inner to outer mappings to >>>>>> >>implement policy. >>>>>> >> >>>>>> >>>IMO: NVE is not the entity to enforce the policy. NVA is the >>>>>>entity >>>>>> >>>to enforce the policy regardless the tenant locations. Again, >>>>>>Network >>>>>> >>>virtualization overlay has to address how to support the >>>>>>policies and >>>>>> >>>tenant mobility. IMO: current architecture draft is vague in or >>>>>>lacks >>>>>> >>>of describing this but it is important to architect this when >>>>>>having >>>>>> >>>data plan and control plane separated on different entities. >>>>>> >> >>>>>> >>LK2> I'm not sure if I agree with you about the NVE not being the >>>>>> >>LK2> entity >>>>>> >>to enforce policy, but I think it is something better discussed >>>>>>in a >>>>>> >>conversation (not email). >>>>>> >> >>>>>> >>> >>>>>> >>>Thanks, >>>>>> >>>Lucy >>>>>> >>> >>>>>> >>>> >>>>>> >>>>Thanks, >>>>>> >>>>Lucy >>>>>> >>>> >>>>>> >>>>-----Original Message----- >>>>>> >>>>From: Larry Kreeger (kreeger) [mailto:[email protected]] >>>>>> >>>>Sent: Friday, October 18, 2013 5:00 PM >>>>>> >>>>To: Thomas Narten; Lucy yong >>>>>> >>>>Cc: [email protected] >>>>>> >>>>Subject: Re: [nvo3] Distributed Gateways [was Re: NVO3 >>>>>>Architecture >>>>>> >>>>document] >>>>>> >>>> >>>>>> >>>>Hi Thomas and Lucy, >>>>>> >>>> >>>>>> >>>>The WG needs to think hard about this one. >>>>>> >>>> >>>>>> >>>>Support of a distributed L3 gateway function between L2 VNs is a >>>>>> >>>>significant increase in scope of the NVA, and the NVE to NVA >>>>>>protocol. >>>>>> >>>>Where we had previously stated L2 service or L3 service and >>>>>>pretty >>>>>> >>>>much left a combined L2/L3 service as an exercise for the >>>>>>reader, we >>>>>> >>>>would now be adding whatever mechanisms are needed to the >>>>>>protocols. >>>>>> >>>>We will need to add cases for >>>>>> >>>>L2 service, L3 service and L2/L3 service. We no longer have >>>>>>simple >>>>>> >>>>inner to outer mappings, but now need NVEs to do MAC rewrites, >>>>>>local >>>>>> >>>>NVE ARP termination, and multiple lookups depending on the >>>>>> >>>>destination MAC address (first L2, then potentially L3). We will >>>>>> >>>>also need to distribute two different VN identifiers (one for >>>>>>L2 and >>>>>> >>>>one for L3), and somehow convey the containment relationship >>>>>>between >>>>>> >>>>the two (multiple L2 VNs within one >>>>>> >>>>L3 VN). While I think this is all very useful, I just want to >>>>>>make >>>>>> >>>>sure the WG agrees to this since I feel it is a significant >>>>>> >>>>change/increase in scope from my perspective. >>>>>> >>>> >>>>>> >>>>Thanks, Larry >>>>>> >>>> >>>>>> >>>> >>>>>> >>>> >>>>>> >>>>On 10/18/13 2:52 PM, "Thomas Narten" <[email protected]> wrote: >>>>>> >>>> >>>>>> >>>>>Hi Lucy. >>>>>> >>>>> >>>>>> >>>>>Lucy yong <[email protected]> writes: >>>>>> >>>>> >>>>>> >>>>>> Section 5.3 describes gateways. IMO: it misses an important >>>>>>use >>>>>> >>>>>> case. A Gateway, say overlay gateway, may be used to >>>>>>interconnect >>>>>> >>>>>> two or more overlay VNs. In this case, the traffic traversing >>>>>> >>>>>> between two overlay VNs must go through the gateway where the >>>>>> >>>>>> policy can be enforced. Furthermore, it is possible to >>>>>>implement >>>>>> >>>>>> centralized or distributed overlay gateway. The latter has >>>>>> >>>>>> overlay gateway function implemented on NVEs. Thus, it >>>>>>requests >>>>>> >>>>>> the cross-VN policies to be distributed to NVEs. >>>>>> >>>>> >>>>>> >>>>>> Current section seems very focus on overlay VN interconnect a >>>>>> >>>>>> non-overlay network, which centralized gateway architecture >>>>>>is >>>>>> >>>>>> practical. But in overlay networks, both centralized or >>>>>> >>>>>> distributed are possible and depend on the applications. >>>>>> >>>>> >>>>>> >>>>>Agreed. I propose adding a new section after 5.3 that says: >>>>>> >>>>> >>>>>> >>>>> <section title="Distributed Gateways"> >>>>>> >>>>> <t> >>>>>> >>>>> The relaying of traffic from one VN to another deserves >>>>>> >>>>> special consideration. The previous section described >>>>>> >>>>> gateways performing this function. If such gateways are >>>>>> >>>>> centralized, traffic between TSes on different VNs can take >>>>>> >>>>> suboptimal paths, i.e., triangular routing results in paths >>>>>> >>>>> that always traverse the gateway. As an optimization, >>>>>> >>>>> individual NVEs can be part of a distributed gateway that >>>>>> >>>>> performs such relaying, reducing or completely eliminating >>>>>> >>>>> triangular routing. In a distributed gateway, each ingress >>>>>> >>>>> NVE can perform such relaying activity directly, so long as >>>>>> >>>>> it has access to the policy information needed to determine >>>>>> >>>>> whether cross-VN communication is allowed. Having individual >>>>>> >>>>> NVEs be part of a distributed gateway allows them to tunnel >>>>>> >>>>> traffic directly to the destination NVE without the need to >>>>>> >>>>> take suboptimal paths. >>>>>> >>>>> </t> >>>>>> >>>>> <t> >>>>>> >>>>> The NVO3 architecture should [must? or just say it does?] >>>>>> >>>>> support distributed gateways. Such support requires that >>>>>> >>>>> NVO3 control protocols include mechanisms for the >>>>>> >>>>> maintenance and distribution of policy information about >>>>>> >>>>> what type of cross-VN communication is allowed so that NVEs >>>>>> >>>>> acting as distributed gateways can tunnel traffic from one >>>>>> >>>>> VN to another as appropriate. >>>>>> >>>>> </t> >>>>>> >>>>> </section> >>>>>> >>>>> >>>>>> >>>>>Thoughts? >>>>>> >>>>> >>>>>> >>>>>Thomas >>>>>> >>>>> >>>>>> >>>>>_______________________________________________ >>>>>> >>>>>nvo3 mailing list >>>>>> >>>>>[email protected] >>>>>> >>>>>https://www.ietf.org/mailman/listinfo/nvo3 >>>>>> >>>> >>>>>> >>> >>>>>> >> >>>>>> >>_______________________________________________ >>>>>> >>nvo3 mailing list >>>>>> >>[email protected] >>>>>> >>https://www.ietf.org/mailman/listinfo/nvo3 >>>>>> > _______________________________________________ nvo3 mailing list [email protected] https://www.ietf.org/mailman/listinfo/nvo3
