Let me illustrate this with an example: * User creates a VM whose root disk is placed on managed storage
* Storage plug-in creates a volume on its SAN * VMware server resource creates a datastore based on the newly created SAN volume (let me stress that this datastore was created by the VMware server resource - not manually by an admin as would be the case for non-managed storage) * Inside the datastore are placed the VMDK file (root disk) along with VM config files like VMX, NVRAM, etc. * User detaches the root volume (the VMDK file and VM config files remain in the datastore) * User attaches another root volume to the VM (the VMDK file is stored in a datastore different from the datastore where the VM config files reside, which is fine for now) * User deletes and expunges the original root disk (this leads to the datastore the VMDK file is on being removed...as a side effect, you will also lose your VM config files), SAN volume is deleted, CloudStack volume is marked as deleted in the database On Fri, Mar 28, 2014 at 5:05 PM, Mike Tutkowski < mike.tutkow...@solidfire.com> wrote: > So, do you guys see my concern with VMware, though? > > VMware is different from XenServer and KVM in that its VM config files are > stored in the datastore along side the root volume (in CloudStack 4.3, for > example). > > If you switch the VM to use a VMDK file in a different datastore, the > config files will remain in the original datastore (unless we transfer them > ourselves to the new datastore). > > If they remain in the original datastore and that disk is deleted later, > the datastore that contains that disk will be removed (along with the VM > config files that are new being used in conjunction with a disk in another > datastore). > > > On Fri, Mar 28, 2014 at 4:58 PM, Alena Prokharchyk < > alena.prokharc...@citrix.com> wrote: > >> >> >> On 3/28/14, 3:50 PM, "Marcus" <shadow...@gmail.com> wrote: >> >> > I see this feature as mainly just shuffling around object properties >> >in the database. I don't expect any major issues to arise with any >> >storage if an inactive "root" disk is marked as a "data" disk in the >> >DB, for example. In the end, when you start a VM you're always going >> >to have a root disk in the vm instance object, and volumes that are >> >attached/detached are going to be passed as data disks (If I >> >understand correctly). It doesn't really matter to the storage drivers >> >if the volume object was previously of type root or data. >> >> Correct. That¹s what I reflected in the spec. But I¹m going to test it on >> all major supported hypervisors - KVM/Xen/VmWare - anyway, just to be 100% >> sure nothing breaks. >> >> >> >> > >> >On Fri, Mar 28, 2014 at 12:48 PM, Alena Prokharchyk >> ><alena.prokharc...@citrix.com> wrote: >> >> I will look into it more, Mike. vmWare indeed can be different. >> >> >> >> -Alena. >> >> >> >> From: Mike Tutkowski >> >><mike.tutkow...@solidfire.com<mailto:mike.tutkow...@solidfire.com>> >> >> Date: Friday, March 28, 2014 at 11:39 AM >> >> To: Alena Prokharchyk >> >><alena.prokharc...@citrix.com<mailto:alena.prokharc...@citrix.com>> >> >> Cc: "dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>" >> >><dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>> >> >> Subject: Re: [PROPOSAL] ROOT volume detach - feature for CS 4.5 >> >> >> >> VMware is also different because when you shut a VMware VM down from >> >>CloudStack, the VM still exists in vCenter Server (whereas for XenServer >> >>and KVM, the VM is gone). >> >> >> >> Since the life of a datastore that was created for managed storage is >> >>tied to the life of the CloudStack volume it stores, when the CloudStack >> >>volume is deleted, the datastore goes away, as well. >> >> >> >> If the datastore in question was automatically created to store a root >> >>disk (alongside VM config files) and you switch the VM to another root >> >>disk (which has to necessarily be in another datastore), you won't see a >> >>problem until the original root volume is expunged by CloudStack. At >> >>this point, its datastore will go away along with your VM config files. >> >> >> >> >> >> On Fri, Mar 28, 2014 at 12:31 PM, Mike Tutkowski >> >><mike.tutkow...@solidfire.com<mailto:mike.tutkow...@solidfire.com>> >> >>wrote: >> >> Well, the reason I brought it up was mainly due to VMware. >> >> >> >> Let's use that as an example: >> >> >> >> I initiate the process of spinning up a VM based on managed storage. >> >> A volume is dynamically created on a SAN. >> >> VmwareStorageProcessor dynamically creates a datastore to consume the >> >>newly created SAN volume. >> >> All VMware VM files (ex. VMX, NVRAM) are placed in the datastore >> >>alongside the VMDK file that represents the root volume. >> >> >> >> Now, let's say we want to detach this root volume and give the VM a new >> >>root volume. >> >> >> >> The new root volume will necessarily be on a different datastore than >> >>the datastore of the previous root volume (because a datastore created >> >>to consume managed storage will have at most one VMDK file*). >> >> >> >> Is it going to be a problem that the VM's files (ex. VMX, NVRAM) are on >> >>one datastore, but its root disk is on another? >> >> >> >> I don't think it's really a problem until you go to delete the original >> >>root volume from CloudStack. At that point, its datastore will be >> >>removed (including, of course, your VM's VMX, NVRAM, etc. files). >> >> >> >> This is not really a problem on XenServer because XenServer does not >> >>store VM config files in the SR, so I think we're OK there. >> >> >> >> We should also be OK for KVM. >> >> >> >> * Technically it can have many if those other VMDK files are delta >> >>snapshots, but they still - together - represent a single disk. >> >> >> >> >> >> On Fri, Mar 28, 2014 at 10:36 AM, Alena Prokharchyk >> >><alena.prokharc...@citrix.com<mailto:alena.prokharc...@citrix.com>> >> >>wrote: >> >> Mike, thank you for the explanation on managed storage.. As far as I >> >>understand from your email, the main difference is instead of creating >> >>an SR on the PS, CloudStack will recognize pre-existing volume created >> >>outside of the CS. Am I correct? >> >> >> >> If so, I don't think there would be any difference. When root volume >> >>detach happens, no storage attributes - path, clusterId - are being >> >>changed. And we would apply the same set of checks to the root volume >> >>attach, as for a dataDisk attach. >> >> >> >> -Alena. >> >> >> >> From: Mike Tutkowski >> >><mike.tutkow...@solidfire.com<mailto:mike.tutkow...@solidfire.com>> >> >> Date: Thursday, March 27, 2014 at 9:40 PM >> >> To: Alena Prokharchyk >> >><alena.prokharc...@citrix.com<mailto:alena.prokharc...@citrix.com>> >> >> Cc: "dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>" >> >><dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>> >> >> Subject: Re: [PROPOSAL] ROOT volume detach - feature for CS 4.5 >> >> >> >> Hi Alena, >> >> >> >> I was wondering if you've taken "managed" storage into consideration >> >>for this? >> >> >> >> If you're unfamiliar with it, managed storage is named as such because >> >>CloudStack manages it on behalf of the admin (ex. dynamically creating >> >>SRs as needed). >> >> >> >> For example, when I add primary storage to CloudStack that is based on >> >>the SolidFire SAN, I use the SolidFire plug-in, which is an example of >> >>managed storage. >> >> >> >> In this case, the primary storage represents a SAN as opposed to a >> >>preallocated volume. >> >> >> >> When the time comes to, say, attach a data disk to a VM for the first >> >>time, the SolidFire plug-in goes off to its SAN and dynamically creates >> >>a new volume on it (with the appropriate size and IOPS requirements). >> >> >> >> CloudStack has logic that recognizes managed storage. >> >> >> >> For example, for XenServer, its logic has been augmented to >> >>automatically create an SR based on the iSCSI target that was created on >> >>the SAN and to create a VDI within it that is attached to the VM in >> >>question. >> >> >> >> The big takeaway is that each CloudStack volume here will be associated >> >>with a unique volume on a SAN and consumed as an SR (XenServer) or >> >>datastore (ESX) (KVM handles this differently). In this situation, there >> >>is a 1:1 mapping between a SAN volume and an SR. No other VDIs are >> >>stored on the SR except for the one representing this one CloudStack >> >>volume. >> >> >> >> That being the case, I was wondering what you thought of this with >> >>regards to your root-volume-detach feature? >> >> >> >> If we don't want to look into this for 4.5, it might be best to simply >> >>fail to detach a root volume from a VM if the volume is based on managed >> >>storage or to fail to attach a bootable volume to a VM if it is based on >> >>managed storage. >> >> >> >> Talk to you later, >> >> Mike >> >> >> >> >> >> On Tue, Mar 25, 2014 at 1:24 PM, Alena Prokharchyk >> >><alena.prokharc...@citrix.com<mailto:alena.prokharc...@citrix.com>> >> >>wrote: >> >> Mike, >> >> >> >> Volume has a template_id referencing vm_template table. Vm_template has >> >> bootable flag, so we will derive information from there. >> >> And sure, this information will not change if the root disk is >> detached. >> >> >> >> On 3/25/14, 12:18 PM, "Mike Tutkowski" >> >><mike.tutkow...@solidfire.com<mailto:mike.tutkow...@solidfire.com>> >> >> wrote: >> >> >> >>>Hi Alena, >> >>> >> >>>I was wondering how we plan to keep track of the new "bootable" >> >>>property? >> >>>When we create a VM, would we just mark its root disk as bootable and >> >>>then >> >>>that property becomes immutable (for the upgrade case, all root disks >> >>>would >> >>>be marked as bootable)? >> >>> >> >>>I'm thinking we'd want to keep track of bootable disks even when there >> >>>are >> >>>detached and turned into data disks. Is that what you had in mind? >> >>> >> >>>Thanks! >> >>>Mike >> >>> >> >>> >> >>>On Tue, Mar 25, 2014 at 12:20 PM, Alena Prokharchyk < >> >>>alena.prokharc...@citrix.com<mailto:alena.prokharc...@citrix.com>> >> >>>wrote: >> >>> >> >>>> Here is the link to the corresponding FS (placed in "4.5 Design >> >>>>documents" >> >>>> section) >> >>>> >> >>>> >> >>>> >> https://cwiki.apache.org/confluence/display/CLOUDSTACK/ROOT+volume+deta >> >>>>ch >> >>>> >> >>>> -Alena. >> >>>> >> >>>> From: Alena Prokharchyk >> >>>><alena.prokharc...@citrix.com<mailto:alena.prokharc...@citrix.com >> ><mail >> >>>>to: >> >>>> alena.prokharc...@citrix.com<mailto:alena.prokharc...@citrix.com>>> >> >>>> Date: Monday, March 24, 2014 at 11:37 AM >> >>>> To: >> >>>>"dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org><mailto: >> dev >> >>>>@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>>" < >> >>>> >> >>>>dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org><mailto: >> dev@ >> >>>>cloudstack.apache.org<mailto:dev@cloudstack.apache.org>>> >> >>>> Subject: [PROPOSAL] ROOT volume detach - feature for CS 4.5 >> >>>> >> >>>> I would like to propose a new feature for CS 4.5 - "ROOT volume >> >>>>detach" >> >>>>- >> >>>> that enables support for following use cases: >> >>>> >> >>>> 1) Replace current ROOT volume with the new one for existing vm. >> >>>> 2) Case when ROOT volume of vm1 gets corrupted, and you want to >> attach >> >>>>it >> >>>> to vm2 to run the recovery utils on it. With current CS >> implemntation, >> >>>>you >> >>>> have to perform several steps - create snapshot of vm1's volume, >> >>>>create >> >>>> volume from snapshot, attach volume to the vm2. New implementation >> >>>>will >> >>>> merge it all to one step. >> >>>> >> >>>> >> >>>> With the planned implementation, once the ROOT volume is detached, it >> >>>>can >> >>>> be attached to any existing vm (with respect to Admin/Domain/Physical >> >>>> resources limitations), either as a DataDisk or a Root disk. >> >>>> >> >>>> Amazon EC2 already has this functionality in place, so I think CS >> >>>>would >> >>>> only benefit from having it. Storage experts (Edison, others) please >> >>>>raise >> >>>> your concerns if you have any, or if you see any potential problems >> >>>>with >> >>>> the planned implementation. And if anyone can think of other use >> cases >> >>>>this >> >>>> feature can possible solve, I would appreciate this input as well. >> >>>> >> >>>> >> >>>> Feature limitations: >> >>>> >> >>>> * ROOT volume can be detached only when vm is in Stopped state >> >>>> * CS will fail to start the vm not having a ROOT volume >> >>>> >> >>>> I will send out the link to the FS once I start getting feedback on >> >>>>the >> >>>> proposal. >> >>>> >> >>>> -Alena. >> >>>> >> >>> >> >>> >> >>> >> >>>-- >> >>>*Mike Tutkowski* >> >>>*Senior CloudStack Developer, SolidFire Inc.* >> >>>e: mike.tutkow...@solidfire.com<mailto:mike.tutkow...@solidfire.com> >> >>>o: 303.746.7302<tel:303.746.7302> >> >>>Advancing the way the world uses the >> >>>cloud<http://solidfire.com/solution/overview/?video=play> >> >>>*(tm)* >> >> >> >> >> >> >> >> >> >> -- >> >> Mike Tutkowski >> >> Senior CloudStack Developer, SolidFire Inc. >> >> e: mike.tutkow...@solidfire.com<mailto:mike.tutkow...@solidfire.com> >> >> o: 303.746.7302<tel:303.746.7302> >> >> Advancing the way the world uses the >> >>cloud<http://solidfire.com/solution/overview/?video=play>(tm) >> >> >> >> >> >> >> >> -- >> >> Mike Tutkowski >> >> Senior CloudStack Developer, SolidFire Inc. >> >> e: mike.tutkow...@solidfire.com<mailto:mike.tutkow...@solidfire.com> >> >> o: 303.746.7302<tel:303.746.7302> >> >> Advancing the way the world uses the >> >>cloud<http://solidfire.com/solution/overview/?video=play>(tm) >> >> >> >> >> >> >> >> -- >> >> Mike Tutkowski >> >> Senior CloudStack Developer, SolidFire Inc. >> >> e: mike.tutkow...@solidfire.com<mailto:mike.tutkow...@solidfire.com> >> >> o: 303.746.7302 >> >> Advancing the way the world uses the >> >>cloud<http://solidfire.com/solution/overview/?video=play>(tm) >> >> > > > -- > *Mike Tutkowski* > *Senior CloudStack Developer, SolidFire Inc.* > e: mike.tutkow...@solidfire.com > o: 303.746.7302 > Advancing the way the world uses the > cloud<http://solidfire.com/solution/overview/?video=play> > *(tm)* > -- *Mike Tutkowski* *Senior CloudStack Developer, SolidFire Inc.* e: mike.tutkow...@solidfire.com o: 303.746.7302 Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play> *(tm)*
