Re: Scalable Backup and Recovery
Even as a simple start it would be nice to have the option to back up/snapshot a VM (which means all volumes as a collection) and not just per volume. If I have 200 VMs with 3 volumes each, that's 600 snapshots and 400 attach disk actions. Now scale that out to your average production environment running 1000s of VMs. This becomes a logistical recovery nightmare. We should have some kind of routine that allows snapping multiple volumes as a collection so those same volumes can be restored as a collection. Just a thought. Thanks, -Kelcey Sent from my HTC - Reply message - From: "Darren Shepherd" To: "dev@cloudstack.apache.org" Subject: Scalable Backup and Recovery Date: Sat, Sep 28, 2013 9:22 PM Based on your use cases it sounds like what your asking for is the ability to create a selection criteria for scheduled snapshots. So as long as your VM/Volume matches that criteria it will be backed up at some given time. I think that would be useful because a user could say something like "all volumes in network 'production' should be backed up every night." So if such a thing was implemented it seems like if the storage provider implemented some capability to handle multiple snapshots at once, then it could be passed all volumes at once and it would do something intelligent. Now reading between the lines, it seems like you're looking for some use case to exploit some functionality in netapp. I'll tell you what I'd like to see implemented. Having ran netapp in the past with CS we ran into this conundrum. We would take snapshots on the filter, but in reality they were pretty useless. If you ever needed to rollback to the snapshot, your screwed. Things have changed since the snapshot, VMs were created, deleted, VM snapshots had occurred, etc. So if you rollback the metadata in CS is completely out of sync now. What I think would be a great feature is to be able to do StoragePool snapshots. Now doing the snapshot is simple, the really complex thing is how to implement the "StoragePool revert to snapshot" functionality. If somebody could do that, that would be awesome. Darren On Sat, Sep 28, 2013 at 5:41 PM, SuichII, Christopher wrote: > Well, yes, in part. By scalable I mean that if CloudStack is expected to be > able to manage such a large number of vms, it should be able to backup and > recover those vms with minimal effort. Doing things one at a time does not > necessarily scale well when you're talking about a cloud infrastructure. > >> Also certain hypervisors have various quirks which stand in the way of an >> efficient solution. > > I absolutely agree. This is where the storage subsystem API comes in. > Creating backups for some storage providers can be much faster, easier and > more efficient than hypervisor. As the storage subsystem API gains more > traction and true backup and recovery becomes available, I think we'll begin > to see people asking why things must be done one at a time. The use cases I > listed below would help us get ahead of the curve and have these features I > predict people will be asking for (and it sounds like Kelcey is asking for it > now!). > > -- > Chris Suich > chris.su...@netapp.com > NetApp Software Engineer > Data Center Platforms – Cloud Solutions > Citrix, Cisco & Red Hat > > On Sep 27, 2013, at 6:38 PM, Chiradeep Vittal > wrote: > >> Ah I see. You mean a "scalable user experience". >> >> The actual scalability of the snapshot process itself is limited by >> available disk and network bandwidth. >> Also certain hypervisors have various quirks which stand in the way of an >> efficient solution. >> >> On 9/27/13 10:27 AM, "SuichII, Christopher" wrote: >> >>> I'd like to start a discussion around the direction of scalable backup >>> and recovery in CloudStack. Currently, the only want to backup and >>> recover vms is by setting up a schedule or manually snapshotting up >>> individual vm disks or manually snapshotting vms. Unfortunately, I don't >>> believe this is a very scalable solution. What if a user wants all of >>> their vm disks to be backed up on the same schedule? What if a domain >>> administrator wants all of the vms in their domain to be backed up on the >>> same schedule or to manually backup every vm in their domain? >>> >>> Here are some use cases I see for helping to scale things up: >>> -Scheduled and manual backup of 1 to all of a user's vms and vm disks >>> -Scheduled and manual backup of 1 to all of a domain's vms and vm disks >>> (by a domain admin) >>> -Scheduled and manual backup of 1 to all vms and vm disks on primary >>> storage (by a cloud admin) - this one is tougher to find a valid use case >>> for >>> -Backup schedules attached to service offerings >>> >>> I know I previously started a discussion about backing up multiple vm >>> disks at once, but I think these use cases, broken down by user type >>> (user, domain admin and admin), should help clear things up and show the >>> utility of being able to b
Re: System VM template caching
I was able to create a work around and several community builders tested it out for me and it works. I will not submit to docs as it's a hack, but I have updated the JIRA ticket. Work around can be found at: http://cloud.kelceydamage.com/cloudfire/blog/2013/10/08/conquering-the-cloudstack-4-2-dragon-kvm/ Thanks, -Kelcey Sent from my HTC - Reply message - From: "Soheil Eizadi" To: "dev@cloudstack.apache.org" Subject: System VM template caching Date: Tue, Oct 8, 2013 9:49 PM This seems similar to a problem I had on 4.3 Master with System VM creation. If it is the same problem you can check from API command ListTemplateCommand(), from CloudMonkey and see if it returns a bogus cached value. Then you know it is the same problem. -Soheil http://mail-archives.apache.org/mod_mbox/cloudstack-dev/201309.mbox/%3c6717ec2e5a665a40a5af626d7d4fa90625e2e...@x2008mb1.infoblox.com%3E From: Kelcey Jamison Damage [kel...@backbonetechnology.com] Sent: Tuesday, October 08, 2013 12:19 PM To: Cloud Dev Subject: [ACS 4.2][Upgrade Issue] System VM template caching Hi, Several of us in the community have found that with the 4.2 upgrade, when we download and install the latest system VM template, CloudStack refuses to use this template for new system VM creation. CloudStack appears to be usin a cached or master-clone variant of the old template. This is causing may KVM+ 4.2 users to have broken clouds, A bug report has been filed: https://issues.apache.org/jira/browse/CLOUDSTACK-4826 My question is: Does anyone know where this cached template is stored? when CloudStack goes to make a new system VM, where does it look first for the template? We have observed through testing that this is no secondary storage. Thanks in advance. Kelcey Damage | Infrastructure Systems Architect Strategy | Automation | Cloud Computing | Technology Development Backbone Technology, Inc 604-331-1152 ext. 114
Re: System VM template caching
This process you mention for registering as a user VM I can't find in the upgrade guide. Do you have a link? The work around works because CloudStack defaults to re-download the system template is it is in NOT_DOWNLOADED status. How ever the database never gets updated for the life of the build. CS is designed it seems to only ever have a single unaltered template_id '3' record. And I guess the template download script just overwrites the sane GUID filename. Seems like a solution that could be handled in a better way. Either way, this is what has been working for us in the community. Sent from my HTC - Reply message - From: "Sebastien Goasguen" To: "dev@cloudstack.apache.org" Cc: "dev@cloudstack.apache.org" Subject: System VM template caching Date: Wed, Oct 9, 2013 9:30 AM Are you sure about this ? I thought we needed to register them as user vm and that the upgrade would convert them to systemVM automatically -Sebastien On 9 Oct 2013, at 17:22, "kel...@backbonetechnology.com" wrote: > I was able to create a work around and several community builders tested it > out for me and it works. > > I will not submit to docs as it's a hack, but I have updated the JIRA ticket. > > Work around can be found at: > > http://cloud.kelceydamage.com/cloudfire/blog/2013/10/08/conquering-the-cloudstack-4-2-dragon-kvm/ > > Thanks, > > -Kelcey > > Sent from my HTC > > - Reply message - > From: "Soheil Eizadi" > To: "dev@cloudstack.apache.org" > Subject: System VM template caching > Date: Tue, Oct 8, 2013 9:49 PM > > This seems similar to a problem I had on 4.3 Master with System VM creation. > If it is the same problem you can check from API command > ListTemplateCommand(), from CloudMonkey and see if it returns a bogus cached > value. Then you know it is the same problem. > -Soheil > > http://mail-archives.apache.org/mod_mbox/cloudstack-dev/201309.mbox/%3c6717ec2e5a665a40a5af626d7d4fa90625e2e...@x2008mb1.infoblox.com%3E > > > From: Kelcey Jamison Damage [kel...@backbonetechnology.com] > Sent: Tuesday, October 08, 2013 12:19 PM > To: Cloud Dev > Subject: [ACS 4.2][Upgrade Issue] System VM template caching > > Hi, > > Several of us in the community have found that with the 4.2 upgrade, when we > download and install the latest system VM template, CloudStack refuses to use > this template for new system VM creation. CloudStack appears to be usin a > cached or master-clone variant of the old template. > > This is causing may KVM+ 4.2 users to have broken clouds, A bug report has > been filed: https://issues.apache.org/jira/browse/CLOUDSTACK-4826 > > My question is: Does anyone know where this cached template is stored? when > CloudStack goes to make a new system VM, where does it look first for the > template? We have observed through testing that this is no secondary storage. > > Thanks in advance. > > Kelcey Damage | Infrastructure Systems Architect > Strategy | Automation | Cloud Computing | Technology Development > > Backbone Technology, Inc > 604-331-1152 ext. 114
Re: System VM template caching
We tested deleting the template on primary storage, and it failed to regenerate. What is the documented method for registering the new template, can you link to it? It seems many of us failed to find any documentation about updating the template period, not just in the 4.2 release doc under upgrades from 4.1 Thanks. Sent from my HTC - Reply message - From: "Ahmad Emneina" To: "dev@cloudstack.apache.org" Subject: System VM template caching Date: Wed, Oct 9, 2013 9:48 AM there might be a more sound way than swapping the template on secondary storage and hacking the db. I figure one should be able to register the template, via the documented route... wait for download to succeed, upgrade the binary bits. then when the system vm's fail to launch. delete the cached template on primary storage. That should be enough to trigger a new system vm propagated to the primary storage. I find it hard to believe this passed QA... On Wed, Oct 9, 2013 at 9:39 AM, kel...@backbonetechnology.com < kel...@backbonetechnology.com> wrote: > This process you mention for registering as a user VM I can't find in the > upgrade guide. Do you have a link? > > The work around works because CloudStack defaults to re-download the > system template is it is in NOT_DOWNLOADED status. How ever the database > never gets updated for the life of the build. > > CS is designed it seems to only ever have a single unaltered template_id > '3' record. And I guess the template download script just overwrites the > sane GUID filename. > > Seems like a solution that could be handled in a better way. > > Either way, this is what has been working for us in the community. > > Sent from my HTC > > - Reply message - > From: "Sebastien Goasguen" > To: "dev@cloudstack.apache.org" > Cc: "dev@cloudstack.apache.org" > Subject: System VM template caching > Date: Wed, Oct 9, 2013 9:30 AM > > Are you sure about this ? I thought we needed to register them as user vm > and that the upgrade would convert them to systemVM automatically > > -Sebastien > > On 9 Oct 2013, at 17:22, "kel...@backbonetechnology.com"< > kel...@backbonetechnology.com> wrote: > > > I was able to create a work around and several community builders tested > it out for me and it works. > > > > I will not submit to docs as it's a hack, but I have updated the JIRA > ticket. > > > > Work around can be found at: > > > > > http://cloud.kelceydamage.com/cloudfire/blog/2013/10/08/conquering-the-cloudstack-4-2-dragon-kvm/ > > > > Thanks, > > > > -Kelcey > > > > Sent from my HTC > > > > - Reply message - > > From: "Soheil Eizadi" > > To: "dev@cloudstack.apache.org" > > Subject: System VM template caching > > Date: Tue, Oct 8, 2013 9:49 PM > > > > This seems similar to a problem I had on 4.3 Master with System VM > creation. If it is the same problem you can check from API command > ListTemplateCommand(), from CloudMonkey and see if it returns a bogus > cached value. Then you know it is the same problem. > > -Soheil > > > > > http://mail-archives.apache.org/mod_mbox/cloudstack-dev/201309.mbox/%3c6717ec2e5a665a40a5af626d7d4fa90625e2e...@x2008mb1.infoblox.com%3E > > > > > > From: Kelcey Jamison Damage [kel...@backbonetechnology.com] > > Sent: Tuesday, October 08, 2013 12:19 PM > > To: Cloud Dev > > Subject: [ACS 4.2][Upgrade Issue] System VM template caching > > > > Hi, > > > > Several of us in the community have found that with the 4.2 upgrade, > when we download and install the latest system VM template, CloudStack > refuses to use this template for new system VM creation. CloudStack appears > to be usin a cached or master-clone variant of the old template. > > > > This is causing may KVM+ 4.2 users to have broken clouds, A bug report > has been filed: https://issues.apache.org/jira/browse/CLOUDSTACK-4826 > > > > My question is: Does anyone know where this cached template is stored? > when CloudStack goes to make a new system VM, where does it look first for > the template? We have observed through testing that this is no secondary > storage. > > > > Thanks in advance. > > > > Kelcey Damage | Infrastructure Systems Architect > > Strategy | Automation | Cloud Computing | Technology Development > > > > Backbone Technology, Inc > > 604-331-1152 ext. 114 >
Re: 4.0.x to 4.2 upgrade
4.1 had some major schema modifications around networking... At least for KVM. Sent from my HTC - Reply message - From: "Paul Angus" To: "dev@cloudstack.apache.org" Subject: 4.0.x to 4.2 upgrade Date: Sun, Oct 13, 2013 1:31 AM Hey, I’ve come across a few people trying to upgrade from 4.0.x to 4.2 having problems. It’s not an upgrade that’s mentioned in the release notes and the ../setup/db directory contains sql scripts to update the db from 4.0 to 4.1 and 4.1 to 4.2 – so I guess it has to be a two-step upgrade? Can someone confirm the upgrade process from 4.0.x to 4.2 Regards Paul Angus Senior Consultant / Cloud Architect S: +44 20 3603 0540 | M: +447711418784 | T: CloudyAngus paul.an...@shapeblue.com | www.shapeblue.com | Twitter:@shapeblue ShapeBlue Ltd, 53 Chandos Place, Covent Garden, London, WC2N 4HS Apache CloudStack Bootcamp training courses 02/03 October, London 13/14 November, London 27/28 November, Bangalore 08/09 January 2014, London This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
Re: [GSoC] Congratulations and Wrap up
Thank you to all the students and mentors for bringing new and interesting features into cloudstack. With all the hard work and dedication, we ended up with some great additions. See you in Amsterdam. Sent from my HTC - Reply message - From: "sebgoa" To: "dev@cloudstack.apache.org" Subject: [GSoC] Congratulations and Wrap up Date: Thu, Oct 17, 2013 7:19 AM Hi, This week end is the Google Summer of Code summit. This year we had the chance to get five students in the program: Dharmesh, Meng, Ian, Nguyen and Shiva and I'd be remiss not to acknowledge their hard work. I would like to congratulate all of them on working hard this past summer (program finished late september), and challenging themselves to learn the open source way through our project at the Apache Software Foundation. Thanks also to the mentors: Kelcey, Hugo and Abhi who took time out of their busy schedule and $dayjob to help these students. -Ian Duffy made committer during the GSoC period, he developed a new LDAP/AD plugin and did an outstanding job doing his development in a complete software development pipeline with 90% unit test coverage, automatic building of rpms through jenkins etc. Check it out: http://youtu.be/-3LG8wP7Zac http://cloudstack.apache.org/docs/en-US/Apache_CloudStack/4.2.0/html/Admin_Guide/LDAPserver-for-user-authentication.html Lately Ian has worked with me on a Google compute engine (GCE) interface but this was not part of his charter for GSoC, just a bonus: https://pypi.python.org/pypi/gcloud/0.0.1 -Nguyen worked on extending our GRE SDN controller to support Xen and KVM hypervisor. All his code is in the sdnextensions branch, and once Hugo is done reviewing it (or any other volunteer), we will merge it for 4.3 https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=tree;h=refs/heads/sdnextensions;hb=sdnextensions Nguyen has done a terrific job tracking his progress in JIRA and dividing work in sub-tasks: https://issues.apache.org/jira/browse/CLOUDSTACK-1779 Both Nguyen and Ian will be at the CCC conference in Amsterdam, Nguyen will talk about 'Autoscaling without netscaler' a patch he submitted recently, and Ian will talk about gcloud. -Meng worked on integration of whirr in CloudStack. The goal was to be able to provision Hadoop clusters in an elastic manner and use Yarn to expose a map-reduce API compatible with AWS EMR. This was a very ambitious project that required learning whirr, working across projects at the ASF and learning how to create new APIs. Her final patch is on review board: https://reviews.apache.org/r/14347/ This will need additional work and more architectural thinking before we can use it, but is a solid base to think through how to expose an AWS EMR api with CloudStack. -Shiva worked on a new UI framework using bootstrap and angular.js, he progressed well and his code is now in master under tools/ngui , he also has a final patch in my inbox. Bad practice but review board is finicky with large patches :( This UI is very interesting because it can operate as standalone UI, meaning that you can run locally to manage your cloud remotely. Unfortunately Dharmesh was busy and did not progress well enough to pass the mid-term mark, he did try to tackle the big task of moving us from com.cloud to org.apache: https://issues.apache.org/jira/browse/CLOUDSTACK-212 something that we still need to do and requires some planning/coordination and should be done very early in the release cycle (like now…) Early on they all worked on a great document that is built on jenkins but not released as part of our docs, it served the purpose of getting everyone through the "bonding period" and have all students learn how to submit a patch: http://jenkins.buildacloud.org/view/All/job/docs-4.3-gsoc-guide/ I will try to wrap it up to have a final version of our GSoC 2013 for Apache CloudStack. Again congratulations to all, I am thrilled to have had you in the project and I hope you will continue participating in the community, we will hopefully do this again next year with another set of students. Cheers, -Sebastien
Re: regions
I was under the impression that from the UI you could select the current region and view/manipulate resources. I was planning on using this to deploy a region for local dev work at a client's office, but maintain admin authority. Simply allowing tasks to be processed locally to them. Perhaps I miss-interpreted the use of regions. Does this mean I need to set the remote office as a dedicated cluster and have all that management traffic over the site to site link? Thanks. Sent from my HTC - Reply message - From: "Marcus Sorensen" To: "dev@cloudstack.apache.org" Subject: regions Date: Fri, Oct 18, 2013 8:13 AM Thanks for the reply. On Fri, Oct 18, 2013 at 8:46 AM, Kishan Kavala wrote: > >> -Original Message- >> From: Marcus Sorensen [mailto:shadow...@gmail.com] >> Sent: Friday, 18 October 2013 4:28 AM >> To: dev@cloudstack.apache.org >> Subject: regions >> >> I'm looking for more information on use cases for regions. I've been through: >> >> https://cwiki.apache.org/confluence/display/CLOUDSTACK/AWS- >> Style+Regions+Functional+Spec >> > [KK] Requirements doc has some info on use cases [1] >> And I've set up two management servers as separate regions. From what I can >> tell, I'm not sure why I'd want to use it. 1) accounts need to be replicated >> manually, and > [KK] Account sync need not be manual. Currently it is expected that sync is > done outside CS. Either though integrating with directory service or using a > portal layer above. > Alex Ough started a discussion [2] to use messaging framework to sync account > data Ok, that's good to know. To me this is pretty much the one thing from a management perspective that would differentiate regions vs standalone clusters. If a cloud operator has the means to manage external accounts on their own then they can also easily deploy multiple standalone management clusters and treat them as endpoints with the same amount of effort. If they don't, then regions become really useful if the feature handles account sync. > > 2)I can't make an API call to one MS to deploy in another region > [KK] AWS also doesn't support making API call to another region. You need > select an end_point before making an API call. Sure, but I don't generally think of CloudStack in terms of what AWS does or doesn't do. It never really occurs to me that a feature might do or not do something because it's what AWS does. I think we do that a bit too much. At any rate, I was just looking for reasons from a management perspective to use regions over standalone management clusters; if I have to choose an endpoint then at least in this regard it doesn't matter if that endpoint is a standalone cloudstack install or a region. > >> (or at least I don't see a documented way to do this). Between those two >> limitations, it means I could also set up two standalone management servers >> and they'd function the same, aside from the slight UI change of having >> another >> region listed in the UI. I realize there are GSLB and portable IP features, >> but >> they're not mentioned in the functional spec.I'm just looking for the >> differences from a management perspective and I don't see much. > [KK] GSLB spec [3]. EIP spec [4] Thanks. I also see that in some of the docs there are unimplemented things. I realize that it's a work in progress. Portable IPs look to be a feature within a region, and not cross-region, so this feature would apply to standalone mgmt servers, each being their own standalone region, correct? In a nutshell, if a cloud operator is managing their own accounts outside of CS, and has no intentions of buying a netscaler, there is currently no reason to link multiple sites into regions. Just treat each standalone site as its own endpoint. Is this a correct assessment? Also, it seems fairly straightforward to link two sites together into regions later as the feature gains more functionality, correct? > > [1] https://cwiki.apache.org/confluence/display/CLOUDSTACK/AWS-Style+Regions > [2] > http://mail-archives.apache.org/mod_mbox/cloudstack-dev/201310.mbox/browser > [3] > https://cwiki.apache.org/confluence/display/CLOUDSTACK/GSLB+%28Global+Server+Load+Balancing%29+Functional+specification+and+Design+Document > [4] https://cwiki.apache.org/confluence/display/CLOUDSTACK/portable+public+IP >
Re: 4.2.0 Upgrade and System Templates (KVM - Ubuntu 12.04)
Hi, Try this, it was written for your situation! http://cloud.kelceydamage.com/cloudfire/blog/2013/10/08/conquering-the-cloudstack-4-2-dragon-kvm/ -Kelcey Sent from my HTC - Reply message - From: "Marcus Sorensen" To: Subject: 4.2.0 Upgrade and System Templates (KVM - Ubuntu 12.04) Date: Sat, Oct 26, 2013 4:22 PM I'm not sure. Somebody on the list has asked about this before, so you may be able to find answers in the history. I've never actually done it because I could never get answers about how it was supposed to work. I did do some digging and found that cloudstack always looks for the newest system type template of a certain name and uses that. But I wasn't sure how the script went about triggering a redeploy of the root disk, it just seemed to reboot the VMS. Personally, I've always just replaced the template file by hand, swapping out the old file with the the new on secondary and primary storage, then set the global variable that recreates system VMs when you restart them. I wouldn't recommend doing it that way unless you don't care if it gets messed up (Dev environment). When I ran through upgrade scenarios in testing the release, I was always using the newer template with 4.1 thus didn't need to do that step. On Oct 26, 2013 3:08 PM, "Marty Sweet" wrote: > Hi Marcus, > > That is so irritating, when registering the new template using the > interface should the routing box be checked? > I say this because on past system templates they appear as routing in the > database although it is not specifically stated in the docs (as I assume it > wasn't an option in 3.x). > > Also, how would I go about downloading this now? Seeing as my SecStorage VM > is offline? > This script seems to have little effect: > > > /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt > -m /var/export/secondary -u > > http://d21ifhcun6b1t2.cloudfront.net/templates/4.2/systemvmtemplate-2013-06-12-master-kvm.qcow2.bz2 > -h kvm -s mykey -F -o localhost -r root -d mypassword > > cloudstack-sysvmadm -d 127.0.0.1 -u cloud -p -a > Running item 12 fails with and shows a nice list of options for the > command: > /usr/bin/cloudstack-sysvmadm: line 21: /etc/rc.d/init.d/functions: No such > file or directory > > Thanks for your help so far! > Marty > > > On Sat, Oct 26, 2013 at 9:51 PM, Marcus Sorensen >wrote: > > > Yes, you do need to upgrade your system VMS, and you should also have a > new > > systemvm.iso that was bundled in the cloudstack-common deb file that > would > > have been installed as an upgrade on your KVM hosts. I also feel that the > > documentation of system vm upgrade is lacking. The only place I know if > is > > in the release notes: > > > > > http://cloudstack.apache.org/docs/en-US/Apache_CloudStack/4.2.0/html-single/Release_Notes/index.html > > , > > see 3.1 "Upgrade Instructions", item 12. It references a script > > "cloudstack-sysvmadm", but the upgrade of the system vm template should > be > > done beforehand. Now look at the section just below, 3.2. This > > documentation is obviously messed up because it first says "this applies > > only to VMware", and then it promptly gives system vm upgrade > instructions > > for XenServer, KVM, and VMWare hosts. It's unclear why this system vm > > upgrade would only apply to zones which had VMware hosts, and why these > > instructions aren't also on the 4.1.x to 4.2.x instructions. At any rate, > > the system vm instuctions there for KVM should apply. Register the > template > > (optionally, check the data base to ensure the template is set as system > > type), then restart the system vms per the item 12 script. If your KVM > > hosts relaunch the system vms per the new template and they have the new > > systemvm.iso, they should work. > > > > > > On Oct 26, 2013 2:19 PM, "Marty Sweet" wrote: > > > > > Hi Guys, > > > > > > I have just upgraded to 4.2.0 from 4.1.1 and am having some issues with > > the > > > SystemVMs. > > > I understand that we are meant to upgrade to the new system image? > Using > > > the script in the 'Prepare systemvm' documentation I did this with no > > > avail, editing the database to suit what I think would work has also > not > > > worked. > > > > > > Restoring a backup, I now have my original 4.1.1 acton systemvm > > templates. > > > What steps should I take to launch a systemVM successfully? > > > > > > The upgrade documentation is pretty lacking in this respect, and just > > says > > > restart the systemvms, with no reference to upgrading the image. > > > > > > I also note that the new systemvms don't seem to be mounting the NFS > and > > > are instead using /usr/share/cloudstack-common/vms/systemvm.iso. > > > > > > Opening a VNC session to the VM, shows the following messages: > > > Cannot assign requested address: make_sock: could not bind address > > > dnsmasq: unknown interface eth0 > > > dnsmasq apache2 ... failed! > > > > > > My MD5 sum for the CD boot file is below and is consistant across a
Re: 4.2.0 Upgrade and System Templates (KVM - Ubuntu 12.04)
Not yet. This is a 'hack'. Officially the process is to download update templates as step 1, before the stuff in the release notes. Sent from my HTC - Reply message - From: "Marty Sweet" To: "dev@cloudstack.apache.org" Subject: 4.2.0 Upgrade and System Templates (KVM - Ubuntu 12.04) Date: Sun, Oct 27, 2013 3:40 AM Thanks Kelcey, that worked a treat! Do you know of any plans to update the official documentation? Marty On Sun, Oct 27, 2013 at 1:16 AM, kel...@backbonetechnology.com < kel...@backbonetechnology.com> wrote: > Hi, > > Try this, it was written for your situation! > > > http://cloud.kelceydamage.com/cloudfire/blog/2013/10/08/conquering-the-cloudstack-4-2-dragon-kvm/ > > -Kelcey > > Sent from my HTC > > - Reply message - > From: "Marcus Sorensen" > To: > Subject: 4.2.0 Upgrade and System Templates (KVM - Ubuntu 12.04) > Date: Sat, Oct 26, 2013 4:22 PM > > I'm not sure. Somebody on the list has asked about this before, so you may > be able to find answers in the history. I've never actually done it because > I could never get answers about how it was supposed to work. I did do some > digging and found that cloudstack always looks for the newest system type > template of a certain name and uses that. But I wasn't sure how the script > went about triggering a redeploy of the root disk, it just seemed to reboot > the VMS. > > Personally, I've always just replaced the template file by hand, swapping > out the old file with the the new on secondary and primary storage, then > set the global variable that recreates system VMs when you restart them. I > wouldn't recommend doing it that way unless you don't care if it gets > messed up (Dev environment). > > When I ran through upgrade scenarios in testing the release, I was always > using the newer template with 4.1 thus didn't need to do that step. > On Oct 26, 2013 3:08 PM, "Marty Sweet" wrote: > > > Hi Marcus, > > > > That is so irritating, when registering the new template using the > > interface should the routing box be checked? > > I say this because on past system templates they appear as routing in the > > database although it is not specifically stated in the docs (as I assume > it > > wasn't an option in 3.x). > > > > Also, how would I go about downloading this now? Seeing as my SecStorage > VM > > is offline? > > This script seems to have little effect: > > > > > > > /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt > > -m /var/export/secondary -u > > > > > http://d21ifhcun6b1t2.cloudfront.net/templates/4.2/systemvmtemplate-2013-06-12-master-kvm.qcow2.bz2 > > -h kvm -s mykey -F -o localhost -r root -d mypassword > > > > cloudstack-sysvmadm -d 127.0.0.1 -u cloud -p -a > > Running item 12 fails with and shows a nice list of options for the > > command: > > /usr/bin/cloudstack-sysvmadm: line 21: /etc/rc.d/init.d/functions: No > such > > file or directory > > > > Thanks for your help so far! > > Marty > > > > > > On Sat, Oct 26, 2013 at 9:51 PM, Marcus Sorensen > >wrote: > > > > > Yes, you do need to upgrade your system VMS, and you should also have a > > new > > > systemvm.iso that was bundled in the cloudstack-common deb file that > > would > > > have been installed as an upgrade on your KVM hosts. I also feel that > the > > > documentation of system vm upgrade is lacking. The only place I know if > > is > > > in the release notes: > > > > > > > > > http://cloudstack.apache.org/docs/en-US/Apache_CloudStack/4.2.0/html-single/Release_Notes/index.html > > > , > > > see 3.1 "Upgrade Instructions", item 12. It references a script > > > "cloudstack-sysvmadm", but the upgrade of the system vm template should > > be > > > done beforehand. Now look at the section just below, 3.2. This > > > documentation is obviously messed up because it first says "this > applies > > > only to VMware", and then it promptly gives system vm upgrade > > instructions > > > for XenServer, KVM, and VMWare hosts. It's unclear why this system vm > > > upgrade would only apply to zones which had VMware hosts, and why these > > > instructions aren't also on the 4.1.x to 4.2.x instructions. At any > rate, > > > the system vm instuctions there for KVM should apply. Register the > > template > > > (optionally, check the data base to ensure the template is set as > syste
Re: Not able to view the console of the VM. Getting Timeout error
Is your public Ip routable? If not, then you may require either a socks proxy or connecting your client right to the public network. The CPVM will not render a console over NAT. Sent from my HTC - Reply message - From: "jitendra shelar" To: Subject: Not able to view the console of the VM. Getting Timeout error Date: Tue, Nov 5, 2013 8:08 PM Hi Marty/Kelven, Thanks for your reply. My Management Server is 10.44.66.151. My Host is 10.44.65.143. I am able to ping between the Management Server and Host. PFA snaps and mangement server logs. I am not able to connect from my local browser to CPVM. Please help me. Thanks, Jitendra On Wed, Nov 6, 2013 at 3:19 AM, Kelven Yang wrote: This is the flow, Local browser -> Management Server -> token to Console Proxy VM -> client redirection with token -> Console Proxy VM -> Management Server for authentication -> Console Proxy to create session -> Browser To trouble shooting the problem 1) check assigned public IP address of console proxy VM, ping it from the machine that browser is running 2) check management server log 3) check if ³host² global configuration value is correctly set and console proxy agent connection status Kelven On 11/4/13, 8:45 AM, "jitendra shelar" wrote: >Hi, > >I wanted to know the flow of the request to connect to a particular >provisioned VM. >Is it the following? > >My local browser -> Console Proxy VM -> Host ->Provisioned VM > >I have done setup of the cloudstack but on clicking on view console, >I am getting timeout error. > >Some pointers would really be helpful. > >Thanks, >Jitendra
Re: Not able to view the console of the VM. Getting Timeout error
Yes your public Ip is non route-able (rfc 1918). So either connect you client directly to the private network, or use a socks proxy. Also I'm assuming those are cidr 24 networks. Sent from my HTC - Reply message - From: "jitendra shelar" To: Subject: Not able to view the console of the VM. Getting Timeout error Date: Tue, Nov 5, 2013 10:56 PM Hi Kelcey/Marty/Kelven, I was not able to send the snaps before. I have zipped them now. Thanks, Jitendra On Wed, Nov 6, 2013 at 4:28 AM, kel...@backbonetechnology.com wrote: Is your public Ip routable? If not, then you may require either a socks proxy or connecting your client right to the public network. The CPVM will not render a console over NAT. Sent from my HTC - Reply message - From: "jitendra shelar" To: Subject: Not able to view the console of the VM. Getting Timeout error Date: Tue, Nov 5, 2013 8:08 PM Hi Marty/Kelven, Thanks for your reply. My Management Server is 10.44.66.151. My Host is 10.44.65.143. I am able to ping between the Management Server and Host. PFA snaps and mangement server logs. I am not able to connect from my local browser to CPVM. Please help me. Thanks, Jitendra On Wed, Nov 6, 2013 at 3:19 AM, Kelven Yang wrote: This is the flow, Local browser -> Management Server -> token to Console Proxy VM -> client redirection with token -> Console Proxy VM -> Management Server for authentication -> Console Proxy to create session -> Browser To trouble shooting the problem 1) check assigned public IP address of console proxy VM, ping it from the machine that browser is running 2) check management server log 3) check if ³host² global configuration value is correctly set and console proxy agent connection status Kelven On 11/4/13, 8:45 AM, "jitendra shelar" wrote: >Hi, > >I wanted to know the flow of the request to connect to a particular >provisioned VM. >Is it the following? > >My local browser -> Console Proxy VM -> Host ->Provisioned VM > >I have done setup of the cloudstack but on clicking on view console, >I am getting timeout error. > >Some pointers would really be helpful. > >Thanks, >Jitendra
Re: [VOTE] List CloudStack related books on the website
+1 of course! Sent from my HTC - Reply message - From: "Noah Slater" To: "dev@cloudstack.apache.org" , "market...@cloudstack.apache.org" Subject: [VOTE] List CloudStack related books on the website Date: Mon, May 27, 2013 7:00 AM -0 I think we should be listing books, but I would prefer it to be on the wiki. (Note that your vote email should include +0 and -0, which are weak approval/disapproval.) On 27 May 2013 09:27, Sebastien Goasguen wrote: > Hi, > > After a relatively long discussion on the marketing@ list about the > "Packt Book" [1] I would like to call a vote. > > Proposal: > > I propose to list CloudStack related books on our website [2]. The page > listing these books would contain the following disclaimer: > > "This listing does not represent official endorsement by the Apache > CloudStack project. The Apache CloudStack project does not recommend one > book versus another nor does it guarantee the quality of the books." > > Inclusion of a book in the listing would be done via a vote on the > marketing@ list. > > > As a quick summary, alternatives to this proposal were to: > 1-not do anything > 2-list the books on the wiki > > A few of us have already expressed their opinions and discussed the > possibilities. Check [1]. > > Vote will be open for 96 hours (To accommodate Memorial day in the USA). > > Reply with: > > [ ] +1 approve > [ ] +0 no opinion > [ ] -1 disapprove (and reason why) > > PS: If edits of the disclaimer are needed but that they do not change the > meaning of it, the disclaimer will be modified but the vote will not be > restarted. > > [1] http://markmail.org/thread/r4qdmbonmx6yq2uv > [2] http://cloudstack.apache.org > > -Sebastien -- NS
Re: IRC Meeting Tomorrow? (Wednesday, May 29)
I can't make it, I have a meeting with Websense Tomorrow from UTC 14:00 - 18:00 Maybe next week. Sent from my HTC - Reply message - From: "Abhinandan Prateek" To: "dev@cloudstack.apache.org" Subject: IRC Meeting Tomorrow? (Wednesday, May 29) Date: Tue, May 28, 2013 10:52 PM Can it be shifted to somewhere between 2:00 - 16:00 UTC ? On 29/05/13 10:58 AM, "Animesh Chaturvedi" wrote: >I will attend > >> -Original Message- >> From: Joe Brockmeier [mailto:j...@zonker.net] >> Sent: Tuesday, May 28, 2013 7:44 AM >> To: dev@cloudstack.apache.org >> Subject: IRC Meeting Tomorrow? (Wednesday, May 29) >> >> Hey all, >> >> The regular scheduled time for the weekly IRC meeting is 17:00 UTC >> tomorrow. We've had really low turnouts for the last month or so. >> Instead of sending a reminder, I wanted to ping the list and see whether >> we had enough interest to sustain the meeting time. >> >> Thoughts, suggestions, comments, flames? >> >> Best, >> >> jzb >> -- >> Joe Brockmeier >> j...@zonker.net >> Twitter: @jzb >> http://www.dissociatedpress.net/
Re: migrateVirtualMachine
Migration is only achievable in a cluster. It is hypervisor dependant. Migration is used only for CAP management and HA. In order to cross the hypervisor barrier you need to automate the 'create template > export template > convirt template > import template > provision' process. Sent from my HTC - Reply message - From: "La Motta, David" To: "" Subject: migrateVirtualMachine Date: Fri, Jun 7, 2013 8:20 AM Anybody know a bit more in depth what that API call really does? From the API docs it "Attempts Migration of a VM to a different host or Root volume of the vm to a different storage pool". Does this mean if the VM is on XenServer and I want to move it to vShpere, it does VHD to OVA (VMDK?) conversion under the covers? Just trying to understand this a bit better. Thanks. David La Motta Technical Marketing Engineer Citrix Solutions NetApp 919.476.5042 dlamo...@netapp.com
Re: [GSoC] Accounts vs User
This might help if your interested in how Ops/Users view these concepts: https://cwiki.apache.org/confluence/display/CLOUDSTACK/2012/09/13/A-CS+Post+Install+Architecture Sent from my HTC - Reply message - From: "Mathias Mullins" To: "dev@cloudstack.apache.org" Subject: [GSoC] Accounts vs User Date: Sun, Jun 9, 2013 8:46 PM Ian, I'm going to add one more example that may clears it up for a lot of people I work with. Accounts are the organization / company / group , etc... Users are the people in that organization / company / group 1. With active Directory it is usually more logical to map the OU to the Cloudstack domain and Account. Depends on how deep the OU is. If you Are talking about /Org/mmullins -then Org is best mapped to a account as my mmullins user is going to live in the Org Account. If you are a multi-layered like many organizations, /Company/Org/Dept/mmullins - Then there's two possibilities. Private Cloud - Company is the Cloud is Org = Domain, Dept = Account, mmullins = user Public Cloud - Company is the customer so it maps differently. Company = Domain, Org = Sub-Domain, Dept = Account, mmullins = user You have to anticipate both possible usage patterns. 2. Yes that sounds very worth-wild in my opinion. There are several real-world use cases that I could implement that against. Keep it up! Matt On 6/9/13 3:54 PM, "Ian Duffy" wrote: >Thanks for the replies! > >> We need to figure out the most >> efficient way to map ldap users to >> domains/accounts/users. > >So would it be logical to: > >1) Create a mapping between organizational units and cloudstack domains? > >2) Create accounts that are independent to cloudstack but require that >a user(or users) are imported from the domain/OU in which the account >exists?
Re: Chip and David elected ASF Members
Congratulations! Sent from my HTC - Reply message - From: "Joe Brockmeier" To: Subject: Chip and David elected ASF Members Date: Wed, Jun 12, 2013 6:00 AM On Wed, Jun 12, 2013, at 03:31 AM, Sebastien Goasguen wrote: > Congratulations to Chip and David for being elected members of the ASF: > > https://blogs.apache.org/foundation/entry/the_apache_software_foundation_welcomes2 Awesome work, folks! Congrats! Best, jzb -- Joe Brockmeier j...@zonker.net Twitter: @jzb http://www.dissociatedpress.net/
Re: Advanced Network Question
No sir, 2 separate architectures. You can download your templates or transfer then to the new zone however. Sent from my HTC - Reply message - From: "Maurice L." To: Subject: Advanced Network Question Date: Fri, Jun 21, 2013 6:44 PM Hello, Is it possible to convert a basic network to an advanced network without jeopardizing the current instances? - Maurice
Re: apache libcloud
I'll take a look, I've been waiting for the improved ACS support. Sent from my HTC - Reply message - From: "Sebastien Goasguen" To: "dev@cloudstack.apache.org" Subject: apache libcloud Date: Wed, Jun 26, 2013 1:43 PM Hi folks, libcloud 0.13 RC is up for vote. There is much better support for CloudStack in it. Check it out: http://buildacloud.org/blog/268-cloudstack-support-in-apache-libcloud.html -sebastien
Re: Dev/Test Environment
Hate to be that guy, but it's the other way around.. Xen does provide paravirtualization, KVM does not, but libvirt does with Qemu. Virtualbox is PV only not HAV http://en.m.wikipedia.org/wiki/Paravirtualization :) Sent from my HTC - Reply message - From: "Marcus Sorensen" To: "dev@cloudstack.apache.org" Subject: Dev/Test Environment Date: Thu, May 2, 2013 1:22 PM Virtualbox doesn't support nested virtualization (unless they just barely added it), which is why we have to use fusion (or KVM if you run linux) for the KVM version of devcloud. Vbox works for the Xen devcloud though because it doesn't require the Xen guests to have paravirtualization, whereas cloudstack KVM guests do. On Thu, May 2, 2013 at 2:17 PM, Joe Brockmeier wrote: > On Thu, May 2, 2013, at 03:16 PM, Joe Brockmeier wrote: > > On Thu, May 2, 2013, at 01:44 PM, Soheil Eizadi wrote: > > > The wiki is organized around VirtualBox, assuming it works, I think it > is > > > a better choice than Fusion since it free and easy to setup. I don't > have > > > much experience with VirtualBox, does it not support nested > hypervisors? > > > > If you've gotten something working in Fusion, it would be worth putting > > that on the wiki for folks who aren't a fan of VirtualBox. (Or folks who > > already use Fusion and don't want to run two desktop virt. products...) > > Didn't complete my thought - would be worth putting on the wiki *on a > new page* so that it's not conflicting with the "standard" DevCloud > info. > > Best, > > jzb > -- > Joe Brockmeier > j...@zonker.net > Twitter: @jzb > http://www.dissociatedpress.net/ >
Re: [Doc] Validation Issue in Release Notes
Hi we had a similar issue at the conf with a nested list not contained in a listitem. It's worth a shot. I am not yet near reliable internet to look myself. -Kelcey Sent from my HTC - Reply message - From: "Sebastien Goasguen" To: Subject: [Doc] Validation Issue in Release Notes Date: Mon, Nov 25, 2013 2:58 AM On Nov 25, 2013, at 5:40 AM, Abhinandan Prateek wrote: > There are some issues with the 4.2/master docs. 4.2 is a priority. > > Anyone who fixes the build gets a special mention in the release notes ! > Now can we have someone fix this. > I can't even locate: en-US/Revision_History.xml > -abhi > > On 23/11/13 4:20 pm, "Radhika Puthiyetath" > wrote: > >> Hi, >> >> Sorry for cross-posting. >> >> While validating the Release Notes by using publican, there is a >> validity issue which I am not able to resolve. >> >> The command used is: >> >> >> Publican build -format=test -langs=en-us -config=publican.cfg. >> >> >> The error I am getting is the following: >> >> Release_Notes.xml:3509: validity error : Element listitem content does >> not follow the DTD, expecting >> (calloutlist | glosslist | bibliolist | itemizedlist | orderedlist | >> segmentedlist | simplelist | v >> ariablelist | caution | important | note | tip | warning | literallayout >> | programlisting | programl >> istingco | screen | screenco | screenshot | synopsis | cmdsynopsis | >> funcsynopsis | classsynopsis | >> fieldsynopsis | constructorsynopsis | destructorsynopsis | methodsynopsis >> | formalpara | para | simp >> ara | address | blockquote | graphic | graphicco | mediaobject | >> mediaobjectco | informalequation | >> informalexample | informalfigure | informaltable | equation | example | >> figure | table | msgset | pr >> ocedure | sidebar | qandaset | task | anchor | bridgehead | remark | >> highlights | abstract | authorb >> lurb | epigraph | indexterm | beginpage)+, got (para programlisting CDATA) >> >> The issue is that the CDATA cannot be located in the file. If it is >> removed, we can successfully build the file. The issue persists on both >> Master and 4.2 >> >> Thanks in advance >> >> -Radhika >
