Inline... *Will STEVENS* Lead Developer
*CloudOps* *| *Cloud Solutions Experts 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw @CloudOps_ On Wed, May 18, 2016 at 1:08 PM, Chiradeep Vittal <chirade...@gmail.com> wrote: > The mirrolist is a great idea. There's 2 issues though: > > 1. Already released ACS releases in the wild will continue to attempt to > download from download.cloud.com whenever a zone is created. These old > releases cannot understand mirror lists. But users can be advised to patch > templates.sql to point to an alternate location. The question is what is > this alternate location? I propose "templates.cloudstack.org/old" > If they patch, maybe they can pull one of the URLs from the mirror list? That way we don't have to hardcode a specific mirror into the patch procedure. > > 2. New releases of ACS. The next release of ACS has to support the mirror > list, otherwise we're kicking the can too far down the road. > I agree, we will need to get this feature into ACS ASAP. > > For (1) > * templates.cloudstack.org/old could redirect (A record) to > cloudstack.apt-get.eu . Or the project could use AWS S3 (as > download.cloud.com is currently hosted in S3). Current bill is $40-50 per > month (between 7GB-40GB downloaded per day). I like the S3 approach myself > but it needs the project to cough up some money or a benevolent sponsor. > There's quite a few organizations making money off of ACS, so this > shouldn't be a problem? > > > For (2) > * Assuming somebody steps up to write the new code, we need to find > sponsors that can host the mirrors. > > As somebody said earlier, this needs to happen much sooner than later. > > On Wed, May 18, 2016 at 1:06 AM, Paul Angus <paul.an...@shapeblue.com> > wrote: > > > Pulling a few threads together - if the ASF would create a > > download.cloudstack.org page on ASF infra with the mirrorlist json in it > > as suggested by Nux, we can maintain just that page and add/remove > mirrors > > at will. > > > > > > Kind regards, > > > > Paul Angus > > > > paul.an...@shapeblue.com > > www.shapeblue.com > > 53 Chandos Place, Covent Garden, London WC2N 4HSUK > > @shapeblue > > > > > > > > -----Original Message----- > > From: Wido den Hollander [mailto:w...@widodh.nl] > > Sent: 18 May 2016 08:37 > > To: Giles Sirett <giles.sir...@shapeblue.com>; dev@cloudstack.apache.org > > Subject: RE: Migrating CloudStack content from download.cloud.com > > > > > > > Op 17 mei 2016 om 10:34 schreef Giles Sirett < > giles.sir...@shapeblue.com > > >: > > > > > > > > > This is a MAJOR problem and needs to be figured out before the current > > download site is removed - @Raja - have Citrix committed to it being > there > > another year ? > > > > > > It is my understanding that Cloudstack relies on this download at > > > initial build/startup and then periodically if people do things like > > > build a new zone > > > > > > So, users environments aren't going to stop if its pulled, but nobody > > > would be a able to build/install cloudstack and it may cause issues as > > > and when users make config changes > > > > > > The problem is, for new builds, its hard coded : you start the Sec > > > Storage VM, CloudStack will try to download the built in template from > > > download.cloud.com > > > > > > My understanding is that its simple db change to make once an > > > environment is running (exisiting users) but *may* require a code > > > patch to fix the initial install > > > > > > #notideal > > > > > > Really, this should be maintained by ASF/ the project as its key to > > > the software working, however my understanding is it involves binaries > > > which ASF is never keen on (and may be some 3rd party binaries) - so > > > that just wont happen > > > > > > > > > > > > Apt-get is controlled by Wido et al (for which we all owe him a massive > > thanks) - but you are right, it is a 3rd party dependency and arguably a > > SPOF. But, if ASF wont host binaries, we're always going to have this > > issue. > > > > > > > Fully agree with you. I'am happy to host this, but for the project it is > a > > danger. The domain name should be controlled by the project. > > > > > So, we could patch the code to point somewhere else - but where ? - by > > > definition its got to be something maintained by a 3rd party > > > > > > > > > We had this debate some time ago when talking about the repositories. > We > > (ShapeBlue) maintain a repo for our customers, which wev'e always opened > up > > as public. However, there were concerns about having a company domain > name > > (which I understand) - so apt-get became our preferred repo. > > > > > > > > > Thoughts on how to fix this: > > > > > > 1. if ASF would allow us to create the host entry > > > download.cloudstack.org. Its under their domain, but with the site > > > being with a 3rd party it could be allowed > > > > > > > I think that would be the best. A CNAME pointing to > cloudstack.apt-get.eu > > for now might be enough. > > > > > 2. maybe Citrix could commit to pointing the current DNS entry to a > > > new location (would be simplest but may be legally complex and relies > > > on ongoing goodwill) > > > > > > 3. make a mod to allow users to choose the download location on > > > install. We can then maintain apt-get. To remove the SPOF, we could > > > then get the templates copied to a number of 3rd party locations and > > > allow people to choose which one they use > > > > > > > > > > > > > > > Kind Regards > > > Giles > > > > > > > > > giles.sir...@shapeblue.com > > > www.shapeblue.com > > > 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue > > > > > > > > > > > > -----Original Message----- > > > From: williamstev...@gmail.com [mailto:williamstev...@gmail.com] On > > Behalf Of Will Stevens > > > Sent: 16 May 2016 18:59 > > > To: dev@cloudstack.apache.org > > > Cc: Wido den Hollander <w...@widodh.nl> > > > Subject: Re: Migrating CloudStack content from download.cloud.com > > > > > > @Ian, yes, but I think there is more to it than that. We can change it > > going forward and we can move everything from there over to a new domain. > > > I believe the problem is with all of the existing ACS or CCP installs > > out there currently pointing at the download.cloud.com URL, so when it > > goes down, those installs have to be updated. > > > > > > Maybe I don't understand exactly, but that is the impression I am > > getting from this thread. Can this be confirmed? > > > > > > *Will STEVENS* > > > Lead Developer > > > > > > *CloudOps* *| *Cloud Solutions Experts > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw > > @CloudOps_ > > > > > > On Mon, May 16, 2016 at 1:38 PM, Ian Rae <i...@cloudops.com> wrote: > > > > > > > Hey guys, who controls apt-get.eu and given the (rather concerning) > > > > level of dependency on the operation of the download.cloud.com - > would > > > > it not make sense to use a domain that is under the control of party > > > > whose alignment with CloudStack will not shift. I hesitate to > > > > implicate the red tape of Apache Foundation, but perhaps the fabled > > > > CloudStack alliance organization would be a good way to handle this > > > > long term. > > > > > > > > On Mon, May 16, 2016 at 1:32 PM, Raja Pullela > > > > <raja.pull...@accelerite.com> wrote: > > > > > Paul, > > > > > > > > > > trust me, it won’t be go away overnight. We are talking to Citrix > > > > > and > > > > requested them to keep it active for another year. > > > > > > > > > > Once we have the content copied over (I will work with Wido and > > > > > verify > > > > this) to cloudstack.apt-get.eu and update the documentation (user > > > > guides update and create a wiki to show how an existing installation > > > > can fixed to point to new URLs) we don’t have a dependency on “ > > download.cloud.com” – > > > > right? sorry, if I am missing something here? Please let me know, > > > > > > > > > > best, > > > > > Raja > > > > > Senior Manager, Product Development > > > > > Accelerite, > > > > > 2055, Laurelwood Road, Santa Clara, CA 95054, USA > > > > > Phone: 1-408-216-7010, www.accelerite.com, @accelerite > > > > > > > > > > > > > > > On 5/16/16, 9:38 PM, "Paul Angus" <paul.an...@shapeblue.com> > wrote: > > > > >>Thanks Raja, > > > > >> > > > > >>May I humbly suggest that someone at Accelerite speaks to Citrix > > > > >>about > > > > how long the content will remain on download.cloud.com? > > > > >>Simply losing the content on download.cloud.com overnight will > make > > > > everyone look bad > > > > >> > > > > >>Kind regards, > > > > >> > > > > >>Paul Angus > > > > >> > > > > >>paul.an...@shapeblue.com > > > > >>www.shapeblue.com > > > > >>53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue > > > > >> > > > > >>-----Original Message----- > > > > >>From: Raja Pullela [mailto:raja.pull...@accelerite.com] > > > > >>Sent: 16 May 2016 16:44 > > > > >>To: dev@cloudstack.apache.org > > > > >>Subject: Re: Migrating CloudStack content from download.cloud.com > > > > >> > > > > >>Paul, > > > > >> > > > > >>Yes, you are right, this change impacts both CloudStack and > > > > CloudPlatform customers. Not sure, if I understand you correctly, > the > > > > move away from “download.cloud.com” is permanent. I don’t know how > > > > long the content on download.cloud.com will be up/available. > > > > >> > > > > >>Based on Wido’s availability, we can get rest of the content to > > > > cloudstack.apt-get.eu, hopefully, by this week ? BTW, the content > > on > > > > download.cloud.com is left as is and will stay as is until Citrix > > > > decides on what to do with it. > > > > >> > > > > >>Best, > > > > >>Raja > > > > >>Senior Manager, Product Development > > > > >>Accelerite, > > > > >>2055, Laurelwood Road, Santa Clara, CA 95054, USA > > > > >>Phone: 1-408-216-7010, www.accelerite.com, @accelerite > > > > >> > > > > >> > > > > >>On 5/16/16, 6:36 PM, "Paul Angus" <paul.an...@shapeblue.com> > wrote: > > > > >> > > > > >>>Raja, > > > > >>> > > > > >>>As this effects every copy of CloudStack and CloudPlatform there > > > > >>>ever > > > > was. Please may I ask how long we have to manage a migration away > > > > from download.cloud.com? > > > > >>> > > > > >>> > > > > >>> > > > > >>>Kind regards, > > > > >>> > > > > >>>Paul Angus > > > > >>> > > > > >>>Regards, > > > > >>> > > > > >>>Paul Angus > > > > >>> > > > > >>>paul.an...@shapeblue.com > > > > >>>www.shapeblue.com > > > > >>>53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue > > > > >>> > > > > >>>-----Original Message----- > > > > >>>From: Raja Pullela [mailto:raja.pull...@accelerite.com] > > > > >>>Sent: 16 May 2016 13:53 > > > > >>>To: dev@cloudstack.apache.org > > > > >>>Subject: Re: Migrating CloudStack content from download.cloud.com > > > > >>> > > > > >>>Hi Paul, > > > > >>> > > > > >>>Thank you for the additional file details and I will request wido > > > > >>>to > > > > copy these as well. > > > > >>> > > > > >>>BTW, I agree with you there is every copy of CloudStack out there > > > > >>>and > > > > reason why I have requested Wido to download all the old files as > well. > > > > Looks like we missed few and get these uploaded as well. > > > > >>> > > > > >>>Please note that Citrix owns the “download.cloud.com” address and > > > > >>>we > > > > have to get all the CloudStack related content from there. Once we > > > > confirm on the content move Citrix may choose to remove the content > > > > from the location. > > > > >>> > > > > >>>That said, I will get in touch with Wido again and do a complete > > > > >>>copy > > > > such that we have everything. > > > > >>> > > > > >>>Best, > > > > >>>Raja > > > > >>>Senior Manager, Product Development Accelerite, 2055, Laurelwood > > > > >>>Road, Santa Clara, CA 95054, USA > > > > >>>Phone: 1-408-216-7010, www.accelerite.com, @accelerite > > > > >>> > > > > >>>On 5/14/16, 7:12 AM, "Paul Angus" <paul.an...@shapeblue.com> > wrote: > > > > >>>>Raja, > > > > >>>> > > > > >>>>I've just picked up on this. > > > > >>>> > > > > >>>>There are a number of URLs which are hardcoded into the > CloudStack > > > > database for the small built-in templates. These are in just about > > > > every copy of CloudStack out there. > > > > >>>> > > > > >>>> > http://download.cloud.com/templates/builtin/f59f18fb-ae94-4f97-afd > > > > >>>>2-f8 > > > > >>>>4 > > > > >>>>755767aca.vhd.bz2 > > > > >>>> > http://download.cloud.com/releases/2.2.0/eec2209b-9875-3c8d-92be-c > > > > >>>>001b > > > > >>>>d > > > > >>>>8a0faf.qcow2.bz2 > > > > >>>> > http://download.cloud.com/templates/builtin/centos56-x86_64.vhd.bz > > > > >>>>2 > > > > >>>>http://download.cloud.com/releases/4.3/centos6_4_64bit.vhd.bz2 > > > > >>>>http://download.cloud.com/releases/2.2.0/CentOS5.3-x86_64.ova > > > > >>>> > http://download.cloud.com/templates/builtin/centos-7-x86_64.tar.gz > > > > >>>> > > > > >>>> > > > > >>>>These will need copying and the CloudStack code updating to the > > > > >>>>new > > > > location. > > > > >>>> > > > > >>>> > > > > >>>>I don't think that you can deprecate download.cloud.com or it's > > > > contents for some time, without causing a lot of issues for users. > > > > >>>> > > > > >>>> > > > > >>>> > > > > >>>> > > > > >>>> > > > > >>>> > > > > >>>> > > > > >>>> > > > > >>>> > > > > >>>>Hi, > > > > >>>> > > > > >>>> > > > > >>>> > > > > >>>>Short Term - all the content has moved to cloudstack.apt-get.eu > > > > >>>> > > > > >>>> > > > > >>>> > > > > >>>>Which sections to be modified? > > > > >>>> > > > > >>>> > > > > >>>> > > > > >>>>The Install Guide needs to be modified. Sample links/details are > > > > >>>>mentioned below for 4.6 and same needs to be updated for other > > > > >>>>Released versions of Cloudstack documentation: > > > > >>>> > > > > >>>> > http://docs.cloudstack.apache.org/projects/cloudstack-installation > > > > >>>>/en/ > > > > >>>>4 .6/management-server/#downloading-vhd-util > > > > >>>> > > > > >>>>Section - Downloading vhd-util¶ > > > > >>>> > > > > >>>> > > > > >>>> > > > > >>>>What to do modified ? > > > > >>>> > > > > >>>>modify URL from > > > > >>>>http://download.cloud.com.s3.amazonaws.com/tools/vhd-util to > > > > >>>>http://cloudstack.apt-get.eu/tools/vhd-util > > > > >>>> > > > > >>>> > > > > >>>> > > > > >>>>@Rajsekhar can you please update the sections ? > > > > >>>> > > > > >>>> > > > > >>>> > > > > >>>>Thanks > > > > >>>> > > > > >>>>Raja > > > > >>>> > > > > >>>>Senior Manager, Product Development > > > > >>>> > > > > >>>>Accelerite, > > > > >>>> > > > > >>>>2055, Laurelwood Road, Santa Clara, CA 95054, USA > > > > >>>> > > > > >>>>Phone: 1-408-216-7010, www.accelerite.com, @accelerite > > > > >>>> > > > > >>>> > > > > >>>>Kind regards, > > > > >>>> > > > > >>>>Paul Angus > > > > >>>> > > > > >>>> > > > > >>>>Regards, > > > > >>>> > > > > >>>>Paul Angus > > > > >>>> > > > > >>>>paul.an...@shapeblue.com > > > > >>>>www.shapeblue.com > > > > >>>>53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue > > > > >>> > > > > >>> > > > > >>> > > > > >>> > > > > >>>DISCLAIMER > > > > >>>========== > > > > >>>This e-mail may contain privileged and confidential information > > > > >>>which > > > > is the property of Accelerite, a Persistent Systems business. It is > > > > intended only for the use of the individual or entity to which it is > > > > addressed. If you are not the intended recipient, you are not > > > > authorized to read, retain, copy, print, distribute or use this > > > > message. If you have received this communication in error, please > > > > notify the sender and delete all copies of this message. Accelerite, > a > > > > Persistent Systems business does not accept any liability for virus > > infected mails. > > > > >> > > > > >> > > > > >> > > > > >> > > > > >>DISCLAIMER > > > > >>========== > > > > >>This e-mail may contain privileged and confidential information > > > > >>which is > > > > the property of Accelerite, a Persistent Systems business. It is > > > > intended only for the use of the individual or entity to which it is > > > > addressed. If you are not the intended recipient, you are not > > > > authorized to read, retain, copy, print, distribute or use this > > > > message. If you have received this communication in error, please > > > > notify the sender and delete all copies of this message. Accelerite, > a > > > > Persistent Systems business does not accept any liability for virus > > infected mails. > > > > > > > > > > > > > > > > > > > > > > > > > DISCLAIMER > > > > > ========== > > > > > This e-mail may contain privileged and confidential information > > > > > which is > > > > the property of Accelerite, a Persistent Systems business. It is > > > > intended only for the use of the individual or entity to which it is > > > > addressed. If you are not the intended recipient, you are not > > > > authorized to read, retain, copy, print, distribute or use this > > > > message. If you have received this communication in error, please > > > > notify the sender and delete all copies of this message. Accelerite, > a > > > > Persistent Systems business does not accept any liability for virus > > infected mails. > > > > > > > > > > > > > > > > -- > > > > Ian Rae > > > > CEO | PDG > > > > c: 514.944.4008 > > > > > > > > CloudOps | Cloud Infrastructure and Networking Solutions > > > > www.cloudops.com | 420 rue Guy | Montreal | Canada | H3J 1S6 > > > > > > >
