[GitHub] [cloudstack-documentation] andrijapanicsb commented on a change in pull request #224: Adding docs for SUSE / RHEL8 variants support
andrijapanicsb commented on a change in pull request #224: URL: https://github.com/apache/cloudstack-documentation/pull/224#discussion_r690124368 ## File path: source/releasenotes/compat.rst ## @@ -24,7 +24,11 @@ CloudStack Management Server. - Ubuntu 16.04 LTS, 18.04 LTS, 20.04 LTS - CentOS versions 7, 8 (note: CentOS 8 will EOL in Dec 2021) -- RHEL versions 7, 8 +- RHEL versions 7, 8 (along with binary compatible versions) +- openSUSE Leap 15 +- SUSE Linux Enterprise Server 15 Review comment: @davidjumani can you please address/reject the proposals/comments in this PR. Thx -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@cloudstack.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: IPV6 in Isolated/VPC networks
Hi Wido, (cc to Rohit and Alex) It is a good suggestion to use FRR for ipv6. The configuration is quite simple and the VMs can get SLAAC, routes, etc. Privacy extension looks not the same as what you mentioned. see https://datatracker.ietf.org/doc/html/rfc4941 You are right. To use static routing, the admins need to configure the routes in the upstream router, and add some ipv6 ranges (eg /56 for VPCs and /64 for isolated networks) and their next-hop (which will be configured in VRs) in CloudStack. CloudStack will pick up an IPv6 range and assign it to an isolated network or vpc. @Rohit, correct me if I'm wrong. I have a question, it looks stateless dhcpv6 (SLAAC from router/VR, router/dns etc via RA messages) will be the only option for now (related to your pr https://github.com/apache/cloudstack/pull/3077) . Would it be good to provide stateful dhcpv6 (which can be implemented by dnsmasq) as an option in cloudstack ? The advantages are (1) support other ipv6 cidr sizes than /64. (2) we can assign a specified Ipv6 address to a vm. vm Ipv6 addresses can be changed (4) an Ipv6 addresses can be re-used by multiple vms. The problem is, stateful dhcpv6 does not support routers,nameservers, etc. we need to figure it out (probably use radvd/frr and dnsmasq both). -Wei On Fri, 13 Aug 2021 at 12:19, Wido den Hollander wrote: > Hi, > > See my inline responses: > > Op 11-08-2021 om 14:26 schreef Rohit Yadav: > > Hi all, > > > > Thanks for your feedback and ideas, I've gone ahead with discussing them > with Alex and came up with a PoC/design which can be implemented in the > following phases: > > > >* Phase1: implement ipv6 support in isolated networks and VPC with > static routing > >* Phase2: discuss and implement support for dynamic routing (TBD) > > > > For Phase1 here's the high-level proposal: > > > >* IPv6 address management: > > * At the zone level root-admin specifies a /64 public range that > will be used for VRs, then they can add a /48, or /56 IPv6 range for guest > networks (to be used by isolated networks and VPC tiers) > > * On creation of any IPv6 enabled isolated network or VPC tier, > from the /48 or /56 block a /64 network is allocated/used > > * We assume SLAAC and autoconfiguration, no DHCPv6 in the zone > (discuss: is privacy a concern, can privacy extensions rfc4941 of slaac be > explored?) > > Privacy Extensions are only a concern for client devices which roam > between different IPv6 networks. > > If you IPv6 address of a client keeps the same suffix (MAC based) and > switches network then only the prefix (/64) will change. > > This way a network like Google, Facebook, etc could track your device > moving from network to network if they only look at the last 64-bits of > the IPv6 address. > > For servers this is not a problem as you already know in which network > they are. > > >* Network offerings: root-admin can create new network offerings > (with VPC too) that specifies a network stack option: > > * ipv4 only (default, for backward compatibility all > networks/offerings post-upgrade migrate to this option) > > * ipv4-and-ipv6 > > * ipv6-only (this can be phase 1.b) > > * A new routing option: static (phase1), dynamic (phase2, with > multiple sub-options such as ospf/bgp etc...) > > This means that the network admin will need to statically route the IPv6 > subnet to the VR's outside IPv6 address, for example, on a JunOS router: > > set routing-options rib inet6.0 static route 2001:db8:500::/48 next-hop > 2001:db8:100::50 > > I'm assuming that 2001:db8:100::50 is the address of the VR on the > outside (/64) network. In reality this will probably be a longer > address, but this is for just the example. > > >* VR changes: > > * VR gets its guest and public nics set to inet6 auto > > * For each /64 allocated to guest network and VPC tiers, radvd > is configured to do RA > > radvd is fine, but looking at phase 2 with dynamic routing you might > already want to look into FRRouting. FRR can also advertise RAs while > not doing any routing. > > interface ens4 >no ipv6 nd suppress-ra >ipv6 nd prefix 2001:db8:500::/64 >ipv6 nd rdnss 2001:db8:400::53 2001:db8:200::53 > > See: http://docs.frrouting.org/en/latest/ipv6.html > > > * Firewall: a new ipv6 zone/chain is created for ipv6 where ipv6 > firewall rules (ACLs, ingress, egress) are implemented; ACLs between VPC > tiers are managed/implemented by ipv6 firewall on VR > > Please take a look at the existing security_group.py script which > implements RFC4890 > > https://datatracker.ietf.org/doc/html/rfc4890 > > ICMPv6 is a vital part of IPv6 and certain packets should always be > allowed. > > > * It is assumed that static routes are created on the core/main > router by the admin or automated using some scripts/tools; for this > CloudStack will announce events with details of /64 networks and VR's > public IPv6 ad
RE: IPV6 in Isolated/VPC networks
Hi Wei, That's correct. The network operator would need to create static routes for the /64s, both for Isolated networks and VPCs. The next-hop for those being the outside interface of the Virtual Router, which will have an IP from the "public" /64. Cheers Alex -Original Message- From: Wei ZHOU Sent: 17 August 2021 10:20 To: dev@cloudstack.apache.org Subject: Re: IPV6 in Isolated/VPC networks Hi Wido, (cc to Rohit and Alex) It is a good suggestion to use FRR for ipv6. The configuration is quite simple and the VMs can get SLAAC, routes, etc. Privacy extension looks not the same as what you mentioned. see https://datatracker.ietf.org/doc/html/rfc4941 You are right. To use static routing, the admins need to configure the routes in the upstream router, and add some ipv6 ranges (eg /56 for VPCs and /64 for isolated networks) and their next-hop (which will be configured in VRs) in CloudStack. CloudStack will pick up an IPv6 range and assign it to an isolated network or vpc. @Rohit, correct me if I'm wrong. I have a question, it looks stateless dhcpv6 (SLAAC from router/VR, router/dns etc via RA messages) will be the only option for now (related to your pr https://github.com/apache/cloudstack/pull/3077) . Would it be good to provide stateful dhcpv6 (which can be implemented by dnsmasq) as an option in cloudstack ? The advantages are (1) support other ipv6 cidr sizes than /64. (2) we can assign a specified Ipv6 address to a vm. vm Ipv6 addresses can be changed (4) an Ipv6 addresses can be re-used by multiple vms. The problem is, stateful dhcpv6 does not support routers,nameservers, etc. we need to figure it out (probably use radvd/frr and dnsmasq both). -Wei On Fri, 13 Aug 2021 at 12:19, Wido den Hollander wrote: > Hi, > > See my inline responses: > > Op 11-08-2021 om 14:26 schreef Rohit Yadav: > > Hi all, > > > > Thanks for your feedback and ideas, I've gone ahead with discussing > > them > with Alex and came up with a PoC/design which can be implemented in > the following phases: > > > >* Phase1: implement ipv6 support in isolated networks and VPC with > static routing > >* Phase2: discuss and implement support for dynamic routing (TBD) > > > > For Phase1 here's the high-level proposal: > > > >* IPv6 address management: > > * At the zone level root-admin specifies a /64 public range that > will be used for VRs, then they can add a /48, or /56 IPv6 range for > guest networks (to be used by isolated networks and VPC tiers) > > * On creation of any IPv6 enabled isolated network or VPC tier, > from the /48 or /56 block a /64 network is allocated/used > > * We assume SLAAC and autoconfiguration, no DHCPv6 in the zone > (discuss: is privacy a concern, can privacy extensions rfc4941 of > slaac be > explored?) > > Privacy Extensions are only a concern for client devices which roam > between different IPv6 networks. > > If you IPv6 address of a client keeps the same suffix (MAC based) and > switches network then only the prefix (/64) will change. > > This way a network like Google, Facebook, etc could track your device > moving from network to network if they only look at the last 64-bits > of the IPv6 address. > > For servers this is not a problem as you already know in which network > they are. > > >* Network offerings: root-admin can create new network offerings > (with VPC too) that specifies a network stack option: > > * ipv4 only (default, for backward compatibility all > networks/offerings post-upgrade migrate to this option) > > * ipv4-and-ipv6 > > * ipv6-only (this can be phase 1.b) > > * A new routing option: static (phase1), dynamic (phase2, with > multiple sub-options such as ospf/bgp etc...) > > This means that the network admin will need to statically route the > IPv6 subnet to the VR's outside IPv6 address, for example, on a JunOS router: > > set routing-options rib inet6.0 static route 2001:db8:500::/48 > next-hop > 2001:db8:100::50 > > I'm assuming that 2001:db8:100::50 is the address of the VR on the > outside (/64) network. In reality this will probably be a longer > address, but this is for just the example. > > >* VR changes: > > * VR gets its guest and public nics set to inet6 auto > > * For each /64 allocated to guest network and VPC tiers, radvd > is configured to do RA > > radvd is fine, but looking at phase 2 with dynamic routing you might > already want to look into FRRouting. FRR can also advertise RAs while > not doing any routing. > > interface ens4 >no ipv6 nd suppress-ra >ipv6 nd prefix 2001:db8:500::/64 >ipv6 nd rdnss 2001:db8:400::53 2001:db8:200::53 > > See: http://docs.frrouting.org/en/latest/ipv6.html > > > * Firewall: a new ipv6 zone/chain is created for ipv6 where ipv6 > firewall rules (ACLs, ingress, egress) are implemented; ACLs between > VPC tiers are managed/implemented by ipv6 firewall on VR > > Plea
Re: IPV6 in Isolated/VPC networks
Hi Wei, Yes, root admins should add all the /64 and bigger /56, /48 blocks which CloudStack will use to calculate and allocate /64 block from to a IPv6 enabled isolated network or VPC tier; for every /64 allocation/assignment to such a network a static route for the /64 target with the ipv6 address of the VR as the gateway should be added to the core-router/ISP gateway. Based on what I read, the RFCs and general IPv6 addressing practice are towards deprecating stateful dhcp(v6) and promote use of /64 network for a direct-attached-network (think like a L2 network or LAN) so I think we should only support SLAAC for isolated networks and VPC tiers, and IPv6 for servers won't need privacy extensions (as Wido advised in earlier reply). If someone has a non-standard need, they may always create and use a shared network with custom-defined IPv6 ranges. Or, at least in the phase1 of the implementation I would prefer the scope to be tight and limited that is acceptable for the community. Regards. From: Alex Mattioli Sent: Tuesday, August 17, 2021 14:59 To: dev@cloudstack.apache.org Subject: RE: IPV6 in Isolated/VPC networks Hi Wei, That's correct. The network operator would need to create static routes for the /64s, both for Isolated networks and VPCs. The next-hop for those being the outside interface of the Virtual Router, which will have an IP from the "public" /64. Cheers Alex -Original Message- From: Wei ZHOU Sent: 17 August 2021 10:20 To: dev@cloudstack.apache.org Subject: Re: IPV6 in Isolated/VPC networks Hi Wido, (cc to Rohit and Alex) It is a good suggestion to use FRR for ipv6. The configuration is quite simple and the VMs can get SLAAC, routes, etc. Privacy extension looks not the same as what you mentioned. see https://datatracker.ietf.org/doc/html/rfc4941 You are right. To use static routing, the admins need to configure the routes in the upstream router, and add some ipv6 ranges (eg /56 for VPCs and /64 for isolated networks) and their next-hop (which will be configured in VRs) in CloudStack. CloudStack will pick up an IPv6 range and assign it to an isolated network or vpc. @Rohit, correct me if I'm wrong. I have a question, it looks stateless dhcpv6 (SLAAC from router/VR, router/dns etc via RA messages) will be the only option for now (related to your pr https://github.com/apache/cloudstack/pull/3077) . Would it be good to provide stateful dhcpv6 (which can be implemented by dnsmasq) as an option in cloudstack ? The advantages are (1) support other ipv6 cidr sizes than /64. (2) we can assign a specified Ipv6 address to a vm. vm Ipv6 addresses can be changed (4) an Ipv6 addresses can be re-used by multiple vms. The problem is, stateful dhcpv6 does not support routers,nameservers, etc. we need to figure it out (probably use radvd/frr and dnsmasq both). -Wei On Fri, 13 Aug 2021 at 12:19, Wido den Hollander wrote: > Hi, > > See my inline responses: > > Op 11-08-2021 om 14:26 schreef Rohit Yadav: > > Hi all, > > > > Thanks for your feedback and ideas, I've gone ahead with discussing > > them > with Alex and came up with a PoC/design which can be implemented in > the following phases: > > > >* Phase1: implement ipv6 support in isolated networks and VPC with > static routing > >* Phase2: discuss and implement support for dynamic routing (TBD) > > > > For Phase1 here's the high-level proposal: > > > >* IPv6 address management: > > * At the zone level root-admin specifies a /64 public range that > will be used for VRs, then they can add a /48, or /56 IPv6 range for > guest networks (to be used by isolated networks and VPC tiers) > > * On creation of any IPv6 enabled isolated network or VPC tier, > from the /48 or /56 block a /64 network is allocated/used > > * We assume SLAAC and autoconfiguration, no DHCPv6 in the zone > (discuss: is privacy a concern, can privacy extensions rfc4941 of > slaac be > explored?) > > Privacy Extensions are only a concern for client devices which roam > between different IPv6 networks. > > If you IPv6 address of a client keeps the same suffix (MAC based) and > switches network then only the prefix (/64) will change. > > This way a network like Google, Facebook, etc could track your device > moving from network to network if they only look at the last 64-bits > of the IPv6 address. > > For servers this is not a problem as you already know in which network > they are. > > >* Network offerings: root-admin can create new network offerings > (with VPC too) that specifies a network stack option: > > * ipv4 only (default, for backward compatibility all > networks/offerings post-upgrade migrate to this option) > > * ipv4-and-ipv6 > > * ipv6-only (this can be phase 1.b) > > * A new routing option: static (phase1), dynamic (phase2, with > multiple sub-options such as ospf/bgp etc...) > > This means that the network
RE: IPV6 in Isolated/VPC networks
+1 to keeping the scope tight on phase 1 and then expanding functionality later on. -Original Message- From: Rohit Yadav Sent: 17 August 2021 11:26 To: dev@cloudstack.apache.org Subject: Re: IPV6 in Isolated/VPC networks Hi Wei, Yes, root admins should add all the /64 and bigger /56, /48 blocks which CloudStack will use to calculate and allocate /64 block from to a IPv6 enabled isolated network or VPC tier; for every /64 allocation/assignment to such a network a static route for the /64 target with the ipv6 address of the VR as the gateway should be added to the core-router/ISP gateway. Based on what I read, the RFCs and general IPv6 addressing practice are towards deprecating stateful dhcp(v6) and promote use of /64 network for a direct-attached-network (think like a L2 network or LAN) so I think we should only support SLAAC for isolated networks and VPC tiers, and IPv6 for servers won't need privacy extensions (as Wido advised in earlier reply). If someone has a non-standard need, they may always create and use a shared network with custom-defined IPv6 ranges. Or, at least in the phase1 of the implementation I would prefer the scope to be tight and limited that is acceptable for the community. Regards. From: Alex Mattioli Sent: Tuesday, August 17, 2021 14:59 To: dev@cloudstack.apache.org Subject: RE: IPV6 in Isolated/VPC networks Hi Wei, That's correct. The network operator would need to create static routes for the /64s, both for Isolated networks and VPCs. The next-hop for those being the outside interface of the Virtual Router, which will have an IP from the "public" /64. Cheers Alex -Original Message- From: Wei ZHOU Sent: 17 August 2021 10:20 To: dev@cloudstack.apache.org Subject: Re: IPV6 in Isolated/VPC networks Hi Wido, (cc to Rohit and Alex) It is a good suggestion to use FRR for ipv6. The configuration is quite simple and the VMs can get SLAAC, routes, etc. Privacy extension looks not the same as what you mentioned. see https://datatracker.ietf.org/doc/html/rfc4941 You are right. To use static routing, the admins need to configure the routes in the upstream router, and add some ipv6 ranges (eg /56 for VPCs and /64 for isolated networks) and their next-hop (which will be configured in VRs) in CloudStack. CloudStack will pick up an IPv6 range and assign it to an isolated network or vpc. @Rohit, correct me if I'm wrong. I have a question, it looks stateless dhcpv6 (SLAAC from router/VR, router/dns etc via RA messages) will be the only option for now (related to your pr https://github.com/apache/cloudstack/pull/3077) . Would it be good to provide stateful dhcpv6 (which can be implemented by dnsmasq) as an option in cloudstack ? The advantages are (1) support other ipv6 cidr sizes than /64. (2) we can assign a specified Ipv6 address to a vm. vm Ipv6 addresses can be changed (4) an Ipv6 addresses can be re-used by multiple vms. The problem is, stateful dhcpv6 does not support routers,nameservers, etc. we need to figure it out (probably use radvd/frr and dnsmasq both). -Wei On Fri, 13 Aug 2021 at 12:19, Wido den Hollander wrote: > Hi, > > See my inline responses: > > Op 11-08-2021 om 14:26 schreef Rohit Yadav: > > Hi all, > > > > Thanks for your feedback and ideas, I've gone ahead with discussing > > them > with Alex and came up with a PoC/design which can be implemented in > the following phases: > > > >* Phase1: implement ipv6 support in isolated networks and VPC with > static routing > >* Phase2: discuss and implement support for dynamic routing (TBD) > > > > For Phase1 here's the high-level proposal: > > > >* IPv6 address management: > > * At the zone level root-admin specifies a /64 public range that > will be used for VRs, then they can add a /48, or /56 IPv6 range for > guest networks (to be used by isolated networks and VPC tiers) > > * On creation of any IPv6 enabled isolated network or VPC tier, > from the /48 or /56 block a /64 network is allocated/used > > * We assume SLAAC and autoconfiguration, no DHCPv6 in the zone > (discuss: is privacy a concern, can privacy extensions rfc4941 of > slaac be > explored?) > > Privacy Extensions are only a concern for client devices which roam > between different IPv6 networks. > > If you IPv6 address of a client keeps the same suffix (MAC based) and > switches network then only the prefix (/64) will change. > > This way a network like Google, Facebook, etc could track your device > moving from network to network if they only look at the last 64-bits > of the IPv6 address. > > For servers this is not a problem as you already know in which network > they are. > > >* Network offerings: root-admin can create new network offerings > (with VPC too) that specifies a network stack option: > > * ipv4 only (default, for backward compatibility all > networks/offerings post-upgrade migrate
Re: IPV6 in Isolated/VPC networks
Op 17-08-2021 om 11:20 schreef Wei ZHOU: Hi Wido, (cc to Rohit and Alex) It is a good suggestion to use FRR for ipv6. The configuration is quite simple and the VMs can get SLAAC, routes, etc. Privacy extension looks not the same as what you mentioned. see https://datatracker.ietf.org/doc/html/rfc4941 You are right. To use static routing, the admins need to configure the routes in the upstream router, and add some ipv6 ranges (eg /56 for VPCs and /64 for isolated networks) and their next-hop (which will be configured in VRs) in CloudStack. CloudStack will pick up an IPv6 range and assign it to an isolated network or vpc. @Rohit, correct me if I'm wrong. I have a question, it looks stateless dhcpv6 (SLAAC from router/VR, router/dns etc via RA messages) will be the only option for now (related to RA/SLAAC is NOT DHCPv6. Please don't confuse that. DHCPv6 is not involved at all when using SLAAC. your pr https://github.com/apache/cloudstack/pull/3077) . Would it be good to provide stateful dhcpv6 (which can be implemented by dnsmasq) as an option in cloudstack ? The advantages are (1) support other ipv6 cidr sizes than /64. Yes, possibly, although hardly used. A /64 for a network is the default in most cases. (2) we can assign a specified Ipv6 address to a vm. vm Ipv6 addresses can be changed Yes, correct. Although you can now also just add a secondary IPv6 address to the Instance. (4) an Ipv6 addresses can be re-used by multiple vms. Yes, that is a benefit. Although this can be achieved with secondaire IPs as well. The problem is, stateful dhcpv6 does not support routers,nameservers, etc. we need to figure it out (probably use radvd/frr and dnsmasq both). You will *always* need RAs, but you will: - Set the Other Managed flag in the RA - Do not advertise a prefix This way the client will learn the IPv6 default gateways, but obtain it's address through DHCPv6. Wido -Wei On Fri, 13 Aug 2021 at 12:19, Wido den Hollander wrote: Hi, See my inline responses: Op 11-08-2021 om 14:26 schreef Rohit Yadav: Hi all, Thanks for your feedback and ideas, I've gone ahead with discussing them with Alex and came up with a PoC/design which can be implemented in the following phases: * Phase1: implement ipv6 support in isolated networks and VPC with static routing * Phase2: discuss and implement support for dynamic routing (TBD) For Phase1 here's the high-level proposal: * IPv6 address management: * At the zone level root-admin specifies a /64 public range that will be used for VRs, then they can add a /48, or /56 IPv6 range for guest networks (to be used by isolated networks and VPC tiers) * On creation of any IPv6 enabled isolated network or VPC tier, from the /48 or /56 block a /64 network is allocated/used * We assume SLAAC and autoconfiguration, no DHCPv6 in the zone (discuss: is privacy a concern, can privacy extensions rfc4941 of slaac be explored?) Privacy Extensions are only a concern for client devices which roam between different IPv6 networks. If you IPv6 address of a client keeps the same suffix (MAC based) and switches network then only the prefix (/64) will change. This way a network like Google, Facebook, etc could track your device moving from network to network if they only look at the last 64-bits of the IPv6 address. For servers this is not a problem as you already know in which network they are. * Network offerings: root-admin can create new network offerings (with VPC too) that specifies a network stack option: * ipv4 only (default, for backward compatibility all networks/offerings post-upgrade migrate to this option) * ipv4-and-ipv6 * ipv6-only (this can be phase 1.b) * A new routing option: static (phase1), dynamic (phase2, with multiple sub-options such as ospf/bgp etc...) This means that the network admin will need to statically route the IPv6 subnet to the VR's outside IPv6 address, for example, on a JunOS router: set routing-options rib inet6.0 static route 2001:db8:500::/48 next-hop 2001:db8:100::50 I'm assuming that 2001:db8:100::50 is the address of the VR on the outside (/64) network. In reality this will probably be a longer address, but this is for just the example. * VR changes: * VR gets its guest and public nics set to inet6 auto * For each /64 allocated to guest network and VPC tiers, radvd is configured to do RA radvd is fine, but looking at phase 2 with dynamic routing you might already want to look into FRRouting. FRR can also advertise RAs while not doing any routing. interface ens4 no ipv6 nd suppress-ra ipv6 nd prefix 2001:db8:500::/64 ipv6 nd rdnss 2001:db8:400::53 2001:db8:200::53 See: http://docs.frrouting.org/en/latest/ipv6.html * Firewall: a new ipv6 zone/chain is created for ipv6 where ipv6 firewall rules (ACLs, ingress, egress) are implemented; ACLs between VPC tier
Re: IPV6 in Isolated/VPC networks
Op 16-08-2021 om 11:29 schreef Rohit Yadav: Thanks Hean, Kristaps, Wido for your feedback. I think we've some quorum and consensus on how we should proceed with IPv6 support with static routing (phase1). Based on my proof-of-concept and discussions, I believe we may target this feature as early as 4.17 and I welcome offer by Kristaps and others who may want to be involved in testing the feature as and when we'll develop it. As the next step I'll write a short design doc on cwiki including some of the new ideas/suggestions and share it on this thread for another iteration. I was thinking about this and I have nother idea for phase 2. As mentioned earlier it can be quite difficult when you have a lot of VRs running to configure OSPFv3 or BGP in the upstream routers to have this all working. Instead we could think about using ExaBGP. With ExaBGP you would have some additional tooling which picks up the subnets assigned to VRs from the messagebus. ExaBGP: https://github.com/Thomas-Mangin/exabgp This tooling then dynamically injects this route into ExaBGP where the destination of thus /48 (example) points to the VR. Inside the VR there is no need to configure OSPF or BGP. The admin doesn't need to configure static routers either. Their routers only peer via BGP with ExaBGP which injects the routes. See this blogpost to get an idea: https://vincent.bernat.ch/en/blog/2013-exabgp-highavailability "Redundancy with ExaBGP" "ExaBGP is a convenient tool to plug scripts into BGP. They can then receive and advertise routes. ExaBGP does the hard work of speaking BGP with your routers. The scripts just have to read routes from standard input or advertise them on standard output." So we would just need to feed ExaBGP all the routes which then advertises them again towards the upstream routers. ExaBGP could be running anywhere as long as it receives the messages from CloudStack and has a BGP connection with the routers. Might be worth looking into in the future for phase 2. And again: Also in shared networks it would be very nice to be able to route a subnet towards a single Instance. Native IPv6 with Docker, VPN services with IPv6 inside a VM, etc, etc. Wido Regards. From: Wido den Hollander Sent: Friday, August 13, 2021 15:48 To: dev@cloudstack.apache.org Subject: Re: IPV6 in Isolated/VPC networks Hi, See my inline responses: Op 11-08-2021 om 14:26 schreef Rohit Yadav: Hi all, Thanks for your feedback and ideas, I've gone ahead with discussing them with Alex and came up with a PoC/design which can be implemented in the following phases: * Phase1: implement ipv6 support in isolated networks and VPC with static routing * Phase2: discuss and implement support for dynamic routing (TBD) For Phase1 here's the high-level proposal: * IPv6 address management: * At the zone level root-admin specifies a /64 public range that will be used for VRs, then they can add a /48, or /56 IPv6 range for guest networks (to be used by isolated networks and VPC tiers) * On creation of any IPv6 enabled isolated network or VPC tier, from the /48 or /56 block a /64 network is allocated/used * We assume SLAAC and autoconfiguration, no DHCPv6 in the zone (discuss: is privacy a concern, can privacy extensions rfc4941 of slaac be explored?) Privacy Extensions are only a concern for client devices which roam between different IPv6 networks. If you IPv6 address of a client keeps the same suffix (MAC based) and switches network then only the prefix (/64) will change. This way a network like Google, Facebook, etc could track your device moving from network to network if they only look at the last 64-bits of the IPv6 address. For servers this is not a problem as you already know in which network they are. * Network offerings: root-admin can create new network offerings (with VPC too) that specifies a network stack option: * ipv4 only (default, for backward compatibility all networks/offerings post-upgrade migrate to this option) * ipv4-and-ipv6 * ipv6-only (this can be phase 1.b) * A new routing option: static (phase1), dynamic (phase2, with multiple sub-options such as ospf/bgp etc...) This means that the network admin will need to statically route the IPv6 subnet to the VR's outside IPv6 address, for example, on a JunOS router: set routing-options rib inet6.0 static route 2001:db8:500::/48 next-hop 2001:db8:100::50 I'm assuming that 2001:db8:100::50 is the address of the VR on the outside (/64) network. In reality this will probably be a longer address, but this is for just the example. * VR changes: * VR gets its guest and public nics set to inet6 auto * For each /64 allocated to guest network and VPC tiers, radvd is configured to do RA radvd is fine, but looking at phase 2 with dynamic routing you might already want to loo
RE: IPV6 in Isolated/VPC networks
Hi Wido, Thanks for the idea, this sounds extremely interesting. We'll investigate. Cheers Alex -Original Message- From: Wido den Hollander Sent: 17 August 2021 12:46 To: dev@cloudstack.apache.org Subject: Re: IPV6 in Isolated/VPC networks Op 16-08-2021 om 11:29 schreef Rohit Yadav: > Thanks Hean, Kristaps, Wido for your feedback. > > I think we've some quorum and consensus on how we should proceed with IPv6 > support with static routing (phase1). Based on my proof-of-concept and > discussions, I believe we may target this feature as early as 4.17 and I > welcome offer by Kristaps and others who may want to be involved in testing > the feature as and when we'll develop it. > > As the next step I'll write a short design doc on cwiki including some of the > new ideas/suggestions and share it on this thread for another iteration. > I was thinking about this and I have nother idea for phase 2. As mentioned earlier it can be quite difficult when you have a lot of VRs running to configure OSPFv3 or BGP in the upstream routers to have this all working. Instead we could think about using ExaBGP. With ExaBGP you would have some additional tooling which picks up the subnets assigned to VRs from the messagebus. ExaBGP: https://github.com/Thomas-Mangin/exabgp This tooling then dynamically injects this route into ExaBGP where the destination of thus /48 (example) points to the VR. Inside the VR there is no need to configure OSPF or BGP. The admin doesn't need to configure static routers either. Their routers only peer via BGP with ExaBGP which injects the routes. See this blogpost to get an idea: https://vincent.bernat.ch/en/blog/2013-exabgp-highavailability "Redundancy with ExaBGP" "ExaBGP is a convenient tool to plug scripts into BGP. They can then receive and advertise routes. ExaBGP does the hard work of speaking BGP with your routers. The scripts just have to read routes from standard input or advertise them on standard output." So we would just need to feed ExaBGP all the routes which then advertises them again towards the upstream routers. ExaBGP could be running anywhere as long as it receives the messages from CloudStack and has a BGP connection with the routers. Might be worth looking into in the future for phase 2. And again: Also in shared networks it would be very nice to be able to route a subnet towards a single Instance. Native IPv6 with Docker, VPN services with IPv6 inside a VM, etc, etc. Wido > > Regards. > > > From: Wido den Hollander > Sent: Friday, August 13, 2021 15:48 > To: dev@cloudstack.apache.org > Subject: Re: IPV6 in Isolated/VPC networks > > Hi, > > See my inline responses: > > Op 11-08-2021 om 14:26 schreef Rohit Yadav: >> Hi all, >> >> Thanks for your feedback and ideas, I've gone ahead with discussing them >> with Alex and came up with a PoC/design which can be implemented in the >> following phases: >> >> * Phase1: implement ipv6 support in isolated networks and VPC with >> static routing >> * Phase2: discuss and implement support for dynamic routing (TBD) >> >> For Phase1 here's the high-level proposal: >> >> * IPv6 address management: >>* At the zone level root-admin specifies a /64 public range that >> will be used for VRs, then they can add a /48, or /56 IPv6 range for guest >> networks (to be used by isolated networks and VPC tiers) >>* On creation of any IPv6 enabled isolated network or VPC tier, >> from the /48 or /56 block a /64 network is allocated/used >>* We assume SLAAC and autoconfiguration, no DHCPv6 in the zone >> (discuss: is privacy a concern, can privacy extensions rfc4941 of slaac be >> explored?) > > Privacy Extensions are only a concern for client devices which roam > between different IPv6 networks. > > If you IPv6 address of a client keeps the same suffix (MAC based) and > switches network then only the prefix (/64) will change. > > This way a network like Google, Facebook, etc could track your device > moving from network to network if they only look at the last 64-bits > of the IPv6 address. > > For servers this is not a problem as you already know in which network > they are. > >> * Network offerings: root-admin can create new network offerings (with >> VPC too) that specifies a network stack option: >>* ipv4 only (default, for backward compatibility all >> networks/offerings post-upgrade migrate to this option) >>* ipv4-and-ipv6 >>* ipv6-only (this can be phase 1.b) >>* A new routing option: static (phase1), dynamic (phase2, with >> multiple sub-options such as ospf/bgp etc...) > > This means that the network admin will need to statically route the > IPv6 subnet to the VR's outside IPv6 address, for example, on a JunOS router: > > set routing-options rib inet6.0 static route 2001:db8:500::/48 > next-hop > 2001:db8:100::50 > > I'm assuming that 200
Re: CloudMonkey v6.2.0 Test Binaries
I did basic tests on Linux using the x86-64 binary; * setup a profile * list outputs in csv, json, table, text... * sync, list API, deploy VM * tried the autocomplete (param) true|false - works! Regards. From: Pearl d'Silva Sent: Friday, August 13, 2021 17:58 To: dev@cloudstack.apache.org ; us...@cloudstack.apache.org Subject: CloudMonkey v6.2.0 Test Binaries Hi All, The test binaries for the potential CloudMonkey v6.2.0 have been placed at https://download.cloudstack.org/testing/cmk/. It would be helpful if you'll could help in a round of testing for any issues / regression. On a high-level, some of the major changes include: * Validation of arguments while setting CloudMonkey configuration * Verifying user access to CloudMonkey configuration file * Allowing sync command to be used as a verb for CloudStack API calls * Printing response with newlines if output format chosen is "text" * Providing an option to turn-off autocompletion Please do help in testing, so that if there are any issues, we can work towards getting them fixed and releasing CloudMonkey 6.2.0. I'll cut rc1 sometime next week. Thanks for the support. Thanks & Regards, Pearl Dsilva
Re: IPV6 in Isolated/VPC networks
Hi Wei, Published this month’s RFC 9099 and explains in different words/perspective what has been written by Alex, Rohit and Wido. https://www.rfc-editor.org/rfc/rfc9099.html On 2021/08/17 09:20:21, Wei ZHOU wrote: > Hi Wido, > > (cc to Rohit and Alex) > > It is a good suggestion to use FRR for ipv6. The configuration is quite > simple and the VMs can get SLAAC, routes, etc. > > Privacy extension looks not the same as what you mentioned. see > https://datatracker.ietf.org/doc/html/rfc4941 > > You are right. To use static routing, the admins need to configure the > routes in the upstream router, and add some ipv6 ranges (eg /56 for VPCs > and /64 for isolated networks) and their next-hop (which will be > configured in VRs) in CloudStack. CloudStack will pick up an IPv6 range and > assign it to an isolated network or vpc. @Rohit, correct me if I'm wrong. > > I have a question, it looks stateless dhcpv6 (SLAAC from router/VR, > router/dns etc via RA messages) will be the only option for now (related to > your pr https://github.com/apache/cloudstack/pull/3077) . Would it be good > to provide stateful dhcpv6 (which can be implemented by dnsmasq) as an > option in cloudstack ? The advantages are > (1) support other ipv6 cidr sizes than /64. > (2) we can assign a specified Ipv6 address to a vm. vm Ipv6 addresses can > be changed > (4) an Ipv6 addresses can be re-used by multiple vms. > The problem is, stateful dhcpv6 does not support routers,nameservers, etc. > we need to figure it out (probably use radvd/frr and dnsmasq both). > > -Wei > > > On Fri, 13 Aug 2021 at 12:19, Wido den Hollander wrote: > > > Hi, > > > > See my inline responses: > > > > Op 11-08-2021 om 14:26 schreef Rohit Yadav: > > > Hi all, > > > > > > Thanks for your feedback and ideas, I've gone ahead with discussing them > > with Alex and came up with a PoC/design which can be implemented in the > > following phases: > > > > > >* Phase1: implement ipv6 support in isolated networks and VPC with > > static routing > > >* Phase2: discuss and implement support for dynamic routing (TBD) > > > > > > For Phase1 here's the high-level proposal: > > > > > >* IPv6 address management: > > > * At the zone level root-admin specifies a /64 public range that > > will be used for VRs, then they can add a /48, or /56 IPv6 range for guest > > networks (to be used by isolated networks and VPC tiers) > > > * On creation of any IPv6 enabled isolated network or VPC tier, > > from the /48 or /56 block a /64 network is allocated/used > > > * We assume SLAAC and autoconfiguration, no DHCPv6 in the zone > > (discuss: is privacy a concern, can privacy extensions rfc4941 of slaac be > > explored?) > > > > Privacy Extensions are only a concern for client devices which roam > > between different IPv6 networks. > > > > If you IPv6 address of a client keeps the same suffix (MAC based) and > > switches network then only the prefix (/64) will change. > > > > This way a network like Google, Facebook, etc could track your device > > moving from network to network if they only look at the last 64-bits of > > the IPv6 address. > > > > For servers this is not a problem as you already know in which network > > they are. > > > > >* Network offerings: root-admin can create new network offerings > > (with VPC too) that specifies a network stack option: > > > * ipv4 only (default, for backward compatibility all > > networks/offerings post-upgrade migrate to this option) > > > * ipv4-and-ipv6 > > > * ipv6-only (this can be phase 1.b) > > > * A new routing option: static (phase1), dynamic (phase2, with > > multiple sub-options such as ospf/bgp etc...) > > > > This means that the network admin will need to statically route the IPv6 > > subnet to the VR's outside IPv6 address, for example, on a JunOS router: > > > > set routing-options rib inet6.0 static route 2001:db8:500::/48 next-hop > > 2001:db8:100::50 > > > > I'm assuming that 2001:db8:100::50 is the address of the VR on the > > outside (/64) network. In reality this will probably be a longer > > address, but this is for just the example. > > > > >* VR changes: > > > * VR gets its guest and public nics set to inet6 auto > > > * For each /64 allocated to guest network and VPC tiers, radvd > > is configured to do RA > > > > radvd is fine, but looking at phase 2 with dynamic routing you might > > already want to look into FRRouting. FRR can also advertise RAs while > > not doing any routing. > > > > interface ens4 > >no ipv6 nd suppress-ra > >ipv6 nd prefix 2001:db8:500::/64 > >ipv6 nd rdnss 2001:db8:400::53 2001:db8:200::53 > > > > See: http://docs.frrouting.org/en/latest/ipv6.html > > > > > * Firewall: a new ipv6 zone/chain is created for ipv6 where ipv6 > > firewall rules (ACLs, ingress, egress) are implemented; ACLs between VPC > > tiers are managed/implemented by ipv6 firewall on VR > > > > Please
Re: IPV6 in Isolated/VPC networks
Thanks Kristaps, Wido, Rohit and Alex for your replies. I am fine with not having stateful dhcpv6 and privacy extension/temporary address in phase 1. If community decides not to do eventually , it is also ok to me. We could explore how to better use secondary ipv6 addresses as Wido advised. It would be great if anyone share some user experience. -Wei On Tuesday, 17 August 2021, Kristaps Cudars wrote: > Hi Wei, > > Published this month’s RFC 9099 and explains in different > words/perspective what has been written by Alex, Rohit and Wido. > https://www.rfc-editor.org/rfc/rfc9099.html > > > On 2021/08/17 09:20:21, Wei ZHOU wrote: > > Hi Wido, > > > > (cc to Rohit and Alex) > > > > It is a good suggestion to use FRR for ipv6. The configuration is quite > > simple and the VMs can get SLAAC, routes, etc. > > > > Privacy extension looks not the same as what you mentioned. see > > https://datatracker.ietf.org/doc/html/rfc4941 > > > > You are right. To use static routing, the admins need to configure the > > routes in the upstream router, and add some ipv6 ranges (eg /56 for VPCs > > and /64 for isolated networks) and their next-hop (which will be > > configured in VRs) in CloudStack. CloudStack will pick up an IPv6 range > and > > assign it to an isolated network or vpc. @Rohit, correct me if I'm wrong. > > > > I have a question, it looks stateless dhcpv6 (SLAAC from router/VR, > > router/dns etc via RA messages) will be the only option for now (related > to > > your pr https://github.com/apache/cloudstack/pull/3077) . Would it be > good > > to provide stateful dhcpv6 (which can be implemented by dnsmasq) as an > > option in cloudstack ? The advantages are > > (1) support other ipv6 cidr sizes than /64. > > (2) we can assign a specified Ipv6 address to a vm. vm Ipv6 addresses can > > be changed > > (4) an Ipv6 addresses can be re-used by multiple vms. > > The problem is, stateful dhcpv6 does not support routers,nameservers, > etc. > > we need to figure it out (probably use radvd/frr and dnsmasq both). > > > > -Wei > > > > > > On Fri, 13 Aug 2021 at 12:19, Wido den Hollander wrote: > > > > > Hi, > > > > > > See my inline responses: > > > > > > Op 11-08-2021 om 14:26 schreef Rohit Yadav: > > > > Hi all, > > > > > > > > Thanks for your feedback and ideas, I've gone ahead with discussing > them > > > with Alex and came up with a PoC/design which can be implemented in the > > > following phases: > > > > > > > >* Phase1: implement ipv6 support in isolated networks and VPC > with > > > static routing > > > >* Phase2: discuss and implement support for dynamic routing > (TBD) > > > > > > > > For Phase1 here's the high-level proposal: > > > > > > > >* IPv6 address management: > > > > * At the zone level root-admin specifies a /64 public range > that > > > will be used for VRs, then they can add a /48, or /56 IPv6 range for > guest > > > networks (to be used by isolated networks and VPC tiers) > > > > * On creation of any IPv6 enabled isolated network or VPC > tier, > > > from the /48 or /56 block a /64 network is allocated/used > > > > * We assume SLAAC and autoconfiguration, no DHCPv6 in the > zone > > > (discuss: is privacy a concern, can privacy extensions rfc4941 of > slaac be > > > explored?) > > > > > > Privacy Extensions are only a concern for client devices which roam > > > between different IPv6 networks. > > > > > > If you IPv6 address of a client keeps the same suffix (MAC based) and > > > switches network then only the prefix (/64) will change. > > > > > > This way a network like Google, Facebook, etc could track your device > > > moving from network to network if they only look at the last 64-bits of > > > the IPv6 address. > > > > > > For servers this is not a problem as you already know in which network > > > they are. > > > > > > >* Network offerings: root-admin can create new network offerings > > > (with VPC too) that specifies a network stack option: > > > > * ipv4 only (default, for backward compatibility all > > > networks/offerings post-upgrade migrate to this option) > > > > * ipv4-and-ipv6 > > > > * ipv6-only (this can be phase 1.b) > > > > * A new routing option: static (phase1), dynamic (phase2, > with > > > multiple sub-options such as ospf/bgp etc...) > > > > > > This means that the network admin will need to statically route the > IPv6 > > > subnet to the VR's outside IPv6 address, for example, on a JunOS > router: > > > > > > set routing-options rib inet6.0 static route 2001:db8:500::/48 next-hop > > > 2001:db8:100::50 > > > > > > I'm assuming that 2001:db8:100::50 is the address of the VR on the > > > outside (/64) network. In reality this will probably be a longer > > > address, but this is for just the example. > > > > > > >* VR changes: > > > > * VR gets its guest and public nics set to inet6 auto > > > > * For each /64 allocated to guest network and VPC tiers, > radvd > > > is co
