H Marcus, We had a small session on how we plan to go about ipv6 configuration with your bullet list present. Comments are still brainstorming phase quality but hopefully they will lead to something.
> * VPC has no global IPv6 prefix (super CIDR as current private space), it's > simply IPv6 enabled or not. Admins can choose to route a /60 or a /48 to a > vpc and carve it up among the networks there, but it's not required or > enforced by cloudstack. The idea at Schuberg Philis is that VPCs should have a standard size network assigned to be taken from a pool. We should be able to configure the width of the network. The idea is that per level (Physical network/vpc/tier) only one width will be used but the size of it is configurable. Default block for a VR / VPC would be a /56 and each individual network would use a /64. Note that in a typical isolated network we only use the first. > * VPC networks get assigned one or multiple IPv6 prefixes, each prefix can be > marked 'SLAAC', 'DHCP', or 'Manual'. For networks we are in favor of using SLAAC for configuring routing and addressing, but would advise DHCPv6 to configure DNS and potentially other things (NTP?). No need to register the addresses in CloudStack. We already store the MAC of the NIC so we know which address will be configured on the interface with the configured prefix. Only the router will receive a specific address (prefix::1/64) As mentioned earlier first implementation will do SLAAC. How will SLAAC be configured with ACL's? > * Mgmt server allows calling external plugins for pushing routes to routers > upstream of VPCs (SDN or router API), but could be manually done by admins as > well We discussed the routing issues in front of the VPC / VR. The goal is to provide cloudstack with a way to deal with routable addresses inside an isolated network, as actually this problem is relevant for both IPv4 and IPv6. However for IPv4 we have a perfectly workable solution with NAT and in the IPv6 world we don't/. So we need to address this issue on how to route a network from the provider edge into an isolated network. For IPv6 we have several options: * Dynamic routing protocols, here we can think of iBGP, OSPF or any other protocol. Basically cloudstack assigns a block to a VR / VPC and the VPC / VR tells the outside world to route that block to the outside IP of the router. * Block allocator, cloudstack knows about the supernet (/48 for example) and assigns a block (/56) to a VR or VPC. Cloudstack tells the block allocator to allocate (route) this block to the VR/VPC. Block allocater needs to know about the device being managed (upstream router like cisco/juniper) and configured the route using telnet or an API * Broker, We introduce a new systemvm, the block broker. The admin routes the /48 to the block broker and CloudStack tells the blok broker to reroute a specific subblock to this VR / VPC * Static, Admin configures a list of block/ip pairs in the upstream router. CloudStack knows about these pairs and assigns the ip to the VR / VPC and the block will be routed. > * Work could be done in stages, e.g. SLAAC/manual network ranges would be > fairly straightforward, whereas DHCP ranges would require programming scripts > and ip allocation code. > * An issue was raised about privacy concerns with SLAAC using MAC, but we > think this revolves more around clients than servers; that is, a client > moving around the country would be traceable because of the MAC, but a server > always has the same address anyway. > * Still need to figure out what to do about ACLs, that's a whole separate > issue, but a plan needs to be in place. Are we going to put ACL's on networks or hosts or both? What is easiest to implement and enforce? Do we care about portforwarding, static NAT, Load Balancing etc as well? Or at least think about what impact these decisions have. This is not a version 1 consideration?!? > * We assume there will be an ipv6 public range assignable, allocated for VPC > routers/static nat/loadbalancers to pull from. How will this range be made > available? Is a cidr configured? Will we preconfigure all ranges or have a dynamic availability check? Radvd package is added to the systemvm template. (Done by Hugo this morning) Some standard UI components are needed. (test-)attention needs to go to: HaProxy version/feature matrix has to be reviewed to check for ipv6 compatability. Does dnsmasq work with ipv6? Does keepalived work seemslessly? Ipv6 tables configuration needs to be pushed to the VRs -----Original Message----- From: Marcus Sorensen [mailto:shadow...@gmail.com] Sent: maandag 6 januari 2014 9:11 To: cloudstack-...@incubator.apache.org Subject: IPv6 in VPC (was Re: IPv6 plan - questions) I've discussed this a bit with various subject matter experts at our datacenters/business, and so far we're leaning toward a rollout like this: * VPC has no global IPv6 prefix (super CIDR as current private space), it's simply IPv6 enabled or not. Admins can choose to route a /60 or a /48 to a vpc and carve it up among the networks there, but it's not required or enforced by cloudstack. * VPC networks get assigned one or multiple IPv6 prefixes, each prefix can be marked 'SLAAC', 'DHCP', or 'Manual'. * Mgmt server allows calling external plugins for pushing routes to routers upstream of VPCs (SDN or router API), but could be manually done by admins as well * Work could be done in stages, e.g. SLAAC/manual network ranges would be fairly straightforward, whereas DHCP ranges would require programming scripts and ip allocation code. * An issue was raised about privacy concerns with SLAAC using MAC, but we think this revolves more around clients than servers; that is, a client moving around the country would be traceable because of the MAC, but a server always has the same address anyway. * Still need to figure out what to do about ACLs, that's a whole separate issue, but a plan needs to be in place. Do we care about port forwarding, static NAT, Load Balancing etc as well? Or at least think about what impact these decisions have. * We assume there will be an ipv6 public range assignable, allocated for VPC routers/static nat/loadbalancers to pull from. On Sat, Jan 4, 2014 at 6:11 AM, Marcus Sorensen <shadow...@gmail.com> wrote: > I've put together a rough draft spec: > > https://cwiki.apache.org/confluence/display/CLOUDSTACK/IPv6+in+VPC+Rou > ter > > I basically just laid out some rough ideas. I know there has been a > lot of discussion in the past about DHCPv6, etc. My hope is that we > can at least decide on a spec, for future reference. > > > On Fri, Jan 3, 2014 at 9:53 PM, Marcus Sorensen <shadow...@gmail.com> wrote: >> It's been a long time since I've heard anything in regards to IPv6, >> let alone VPC support. Does anyone have plans for this at all? We'd >> like to support IPv6, and we have enough CS knowledge and external >> tools to hack something together, but I'd much prefer to build with >> the community and/or be forward compatible with what it deploys. >> >> I'd like to start with something simple, like perhaps optionally >> providing a /64 or larger as a parameter when creating a VPC (or a >> separate call to add an IPV6 block), and network on the vpc. Then it >> sounds like there's already a mechanism in place for tracking ipv6 >> assignments to nics, that could be leveraged to pass dhcp assignments >> to routers. >> >> Then there's the whole acl thing, that seems like at least as big of >> a project as mentioned previously. >> >> On Mon, Aug 12, 2013 at 3:47 PM, Marcus Sorensen <shadow...@gmail.com> wrote: >>> has there been any further discussion that I might have missed >>> around >>> ipv6 in VPC? >>> >>> On Thu, Mar 7, 2013 at 12:09 PM, Sheng Yang <sh...@yasker.org> wrote: >>>> Hi Dave, >>>> >>>> I am glad it fits your need. That's our target. :) >>>> >>>> --Sheng >>>> >>>> On Thu, Mar 7, 2013 at 2:14 AM, Dave Cahill <dcah...@midokura.com> wrote: >>>>> Hi Sheng, >>>>> >>>>> Thanks for the quick reply, that helps a lot. >>>>> >>>>> My main purpose was to figure out how these changes affect virtual >>>>> networking and pluggability. Having read through the IPv6 code >>>>> today, it looks like it will work very nicely with virtual networks. >>>>> >>>>> For example, when VMs are assigned an IPv6 address, the IPv6 >>>>> address is stored in the NicProfile object. So, taking DHCP as an >>>>> example, if the MidoNet plugin implements the DHCPServiceProvider >>>>> interface, it will receive the NicProfile as one of the parameters of >>>>> addDhcpEntry. >>>>> If we want to implement IPv6, we can then take the IPv6 address >>>>> from the NicProfile, and just use it as needed. >>>>> >>>>> Thanks again for taking the time to respond, and for the detailed FS. >>>>> >>>>> Dave. >>>>> >>>>> On Thu, Mar 7, 2013 at 4:57 AM, Sheng Yang <sh...@yasker.org> wrote: >>>>> >>>>>> On Wed, Mar 6, 2013 at 1:36 AM, Dave Cahill <dcah...@midokura.com> wrote: >>>>>> > Hi, >>>>>> >>>>>> Hi Dave, >>>>>> > >>>>>> > I've been catching up on IPv6 plans by reading the functional >>>>>> > specs and Jira tickets - it's great to have so much material to refer >>>>>> > to. >>>>>> > >>>>>> > I still have a few questions though, and I'm hoping someone >>>>>> > involved with the feature can enlighten me. >>>>>> > >>>>>> > *[Support for Providers other than Virtual Router]* In [3], the >>>>>> > spec says "No external device support in plan." >>>>>> > What does this mean exactly? >>>>>> >>>>>> Because CloudStack also supports using external devices as >>>>>> network controller e.g. Juniper SRX as firewall and NetScaler as >>>>>> load balancer. The words here said is just we don't support these >>>>>> devices when using IPv6. >>>>>> > >>>>>> > For example, if using Providers other than the Virtual Router, >>>>>> > does the UI still allow setting IPv6 addresses? >>>>>> > >>>>>> > If so, do we attempt to pass IPv6 addresses to the Providers no >>>>>> > matter what, or do we check whether the Provider has IPv6 support? >>>>>> >>>>>> Yes, we checked it when you try to create a IPv6 >>>>>> network(currently only support advance shared network). >>>>>> >>>>>> > >>>>>> > *[Networking Modes]* >>>>>> > Advanced Shared mode and Basic mode are mentioned in the Jira >>>>>> > ticket [1] - "Isolated Network" is mentioned briefly in [2], >>>>>> > but I wanted to check if the Advanced Isolated and VPC modes >>>>>> > are on the roadmap? >>>>>> >>>>>> There is no "basic isolated" network, so "Isolated" network is >>>>>> what we're talking about. We haven't got plan for VPC yet. >>>>>> >>>>>> And one correction: we didn't support "basic" mode for phase 1. >>>>>> We support only "advance shared network" in phase 1. The >>>>>> supported cases are described in FS. Jira ticket only provided a >>>>>> rough idea at the time. >>>>>> > >>>>>> > *[IP Address Management / IPAM]* From [1], re: handing out IPv6 >>>>>> > addresses: "One way could be that the network admin creates a >>>>>> > static route for a /48 towards a Virtual Router and then the VR >>>>>> > can hand out /64s to Instances." >>>>>> > >>>>>> > With IPv4, IPAM is handled by the CloudStack management server, >>>>>> > and the VR is told which IP address to give to the VM over >>>>>> > DHCP. Would this change with IPv6? "The VR can hand out /64s to >>>>>> > instances" sounds like the VR is handling IPAM to some extent. >>>>>> >>>>>> Well, it's not how it works now. Please refer to the FS. The >>>>>> current implementation works like before. VR get a /64 then >>>>>> handle out IPv6 addresses to VM. >>>>>> > >>>>>> > From [3], "Router advertisement should be sent by public >>>>>> > gateway in the network." - to double-check, does this mean the >>>>>> > router outside the CloudStack network should send RAs, but the VR >>>>>> > won't send RAs? >>>>>> >>>>>> Yes. Because in phase 1, we support only "advance shared >>>>>> network", in which case, VR is NOT the gateway. So we assume the >>>>>> gateway router outside CloudStack should send out RA to the VMs. >>>>>> >>>>>> But in the phase 2, VR would acting as gateway, then it would send out >>>>>> RAs. >>>>>> >>>>>> --Sheng >>>>>> > >>>>>> > Thanks, >>>>>> > Dave. >>>>>> > >>>>>> > [1] IPv6 Support main Jira ticket >>>>>> > https://issues.apache.org/jira/browse/CLOUDSTACK-452 >>>>>> > >>>>>> > [2] IPv6 Support in CloudStack FS >>>>>> > https://cwiki.apache.org/CLOUDSTACK/ipv6-support-in-cloudstack. >>>>>> > html >>>>>> > >>>>>> > [3] IPv6 Support FS >>>>>> > https://cwiki.apache.org/CLOUDSTACK/ipv6-support.html >>>>>>
