Correct, Will. That Global Settings would only be for managed storage. Non-managed (traditional) volume snapshots are completely un-impacted by this feature.
If we need to sometimes keep the snapshots on the SAN and sometimes push them to secondary storage, we'll need a more robust solution than Global Settings, though. On Mon, Feb 8, 2016 at 12:11 PM, Will Stevens <wstev...@cloudops.com> wrote: > Sorry. I missed a bit of context when I responded. The global setting > would be only for the managed storage case, currently being called Storage > Snapshots, and is only to determine if a copy is pushed to secondary > storage right? The global setting would not change the behavior of the > Volume Snapshots right? > > I was referring to the need for Volume Snapshots and Storage Snapshots to > exist together. Disregard my comment. I caught up on context after I > posted. My bad... > > *Will STEVENS* > Lead Developer > > *CloudOps* *| *Cloud Solutions Experts > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 > w cloudops.com *|* tw @CloudOps_ > > On Mon, Feb 8, 2016 at 2:05 PM, Mike Tutkowski < > mike.tutkow...@solidfire.com > > wrote: > > > Hey Will, > > > > Who's picking the behavior? Is it the cloud provider or the end user? > > > > Thanks > > > > On Mon, Feb 8, 2016 at 11:52 AM, Will Stevens <wstev...@cloudops.com> > > wrote: > > > > > I don't think a global setting is a good option because we need both > > > functionality to be available at the same time and for different use > > cases > > > to be able to pick which they choose. > > > > > > *Will STEVENS* > > > Lead Developer > > > > > > *CloudOps* *| *Cloud Solutions Experts > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 > > > w cloudops.com *|* tw @CloudOps_ > > > > > > On Mon, Feb 8, 2016 at 1:48 PM, Mike Tutkowski < > > > mike.tutkow...@solidfire.com > > > > wrote: > > > > > > > Now that I re-read your e-mail, it dawned on me: The end-user doesn't > > > care > > > > where the snapshot is. > > > > > > > > If that's true, then we should perhaps control this via Global > Settings > > > or > > > > something. > > > > > > > > On Mon, Feb 8, 2016 at 11:46 AM, Mike Tutkowski < > > > > mike.tutkow...@solidfire.com> wrote: > > > > > > > > > It's not ideal - true, but it does allow us to be backward > > compatible. > > > > > > > > > > If you have other ideas, though, about how to maintain backward > > > > > compatibility, I'm definitely open to hear them. > > > > > > > > > > Thanks! > > > > > > > > > > On Mon, Feb 8, 2016 at 11:42 AM, Syed Mushtaq < > > syed1.mush...@gmail.com > > > > > > > > > wrote: > > > > > > > > > >> Hi Mike, > > > > >> > > > > >> Adding a flag to createSnapshot was the first and the most obvious > > > thing > > > > >> that came to our minds. The problem that I had with this was that: > > > > >> > > > > >> 1) I feel it is exposing something to the end user that is > internal > > to > > > > the > > > > >> cloud. > > > > >> > > > > >> 2) We have to follow two different ways of restore/deletion in the > > > same > > > > >> code path depending on where the Snapshot resides which I find > kind > > > of a > > > > >> bad design. > > > > >> > > > > >> But if exposing a archive flag to the end user is acceptable then > we > > > can > > > > >> definitely use this instead of adding the StorageSnapshot API > > > > >> > > > > >> Thanks, > > > > >> -Syed > > > > >> > > > > >> > > > > >> On Mon, Feb 8, 2016 at 1:26 PM, Mike Tutkowski < > > > > >> mike.tutkow...@solidfire.com > > > > >> > wrote: > > > > >> > > > > >> > Hi Pierre-Luc, > > > > >> > > > > > >> > My recommendation would be this: > > > > >> > > > > > >> > Add an "archive" flag to the current volume-snapshot API. Its > > > default > > > > >> would > > > > >> > be "false" because that would be backward compatible with how > 4.6 > > > has > > > > >> > volume snapshots implemented (i.e. they stay on the SAN in 4.6, > > 4.7, > > > > and > > > > >> > 4.8). > > > > >> > > > > > >> > If you set archive=true, then we would perform a background > > > migration > > > > of > > > > >> > the snapshot from the SAN to the secondary storage (then delete > > the > > > > SAN > > > > >> > snapshot). > > > > >> > > > > > >> > That archive parameter would only be valid for managed storage. > > > > >> > > > > > >> > Sound reasonable? > > > > >> > > > > > >> > Also, a VM snapshot that includes disks provided by managed > > storage > > > > >> should > > > > >> > work. > > > > >> > > > > > >> > Talk to you later, > > > > >> > Mike > > > > >> > > > > > >> > On Mon, Feb 8, 2016 at 9:22 AM, Pierre-Luc Dion < > > pd...@cloudops.com > > > > > > > > >> > wrote: > > > > >> > > > > > >> > > Mike, > > > > >> > > > > > > >> > > In terms of API's, would you prefer introducing a parameter to > > the > > > > >> > existing > > > > >> > > VolumeSnapshot, example: extract={true|false} with a > default > > > > value > > > > >> of > > > > >> > > true which would extract snapshot into the secondary storage, > > > which > > > > >> is > > > > >> > the > > > > >> > > current default behavior. Then for SAN snapshot that remain on > > the > > > > >> SAN we > > > > >> > > would just set "extract=false" ? as oppose to create a new > > > > >> > > StorageSnapshot API ? > > > > >> > > > > > > >> > > > > > > >> > > Paul, > > > > >> > > > > > > >> > > From what I'm seeing so far, we can't do a VM-snapshot when > > using > > > > >> managed > > > > >> > > storage for VM having more than one Volume. For the reason > that > > > > >> snapshot > > > > >> > > are performed outside of the hypervisor awareness and > > > > asynchronously. > > > > >> If > > > > >> > > someone have a way to address that, it would make thinks much > > more > > > > >> > > attractive. > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > On Mon, Feb 8, 2016 at 10:57 AM, Ian Rae <i...@cloudops.com> > > > wrote: > > > > >> > > > > > > >> > > > I think a service provider backup scenario is more likely to > > > take > > > > >> > > advantage > > > > >> > > > of SAN snapshot. There are a few reasons for this. > Traditional > > > > >> backups > > > > >> > > > involve access to the file system, and there is an > expectation > > > > that > > > > >> > this > > > > >> > > > can be done with reasonably short time frames without > > negatively > > > > >> > > impacting > > > > >> > > > VM performance, and that the backup orchestrator can apply > > > various > > > > >> > logic > > > > >> > > > and or transformations to the data (compress, encrypt, > deltas > > > > >> etc...). > > > > >> > > > While it is true that one could apply a backup process to a > > > cloud > > > > >> > > snapshot, > > > > >> > > > this would be slow and inefficient requiring the data to be > > > moved > > > > >> > several > > > > >> > > > times and there are some major bottlenecks with cloud > > snapshots. > > > > >> With a > > > > >> > > > cloud snapshot - there seems to be no reasonable expectation > > of > > > > >> being > > > > >> > > able > > > > >> > > > to do differential snapshots (I think this depends on the > > > > >> hypervisor) > > > > >> > and > > > > >> > > > if you do differential snapshots this will make file backups > > > from > > > > >> those > > > > >> > > > snapshots even more complicated to orchestrate. > > > > >> > > > > > > > >> > > > Suspect there needs to be a different thread on how to > better > > > > enable > > > > >> > > > backups, as a service. As per Paul's suggestion, but it is a > > > > related > > > > >> > > > workflow so relevant to this discussion. > > > > >> > > > > > > > >> > > > Ian > > > > >> > > > > > > > >> > > > On Monday, February 8, 2016, Mike Tutkowski < > > > > >> > > mike.tutkow...@solidfire.com> > > > > >> > > > wrote: > > > > >> > > > > > > > >> > > > > To me it sounds like number two and number three are > > different > > > > >> uses > > > > >> > for > > > > >> > > > the > > > > >> > > > > same "thing"(which is totally fine). > > > > >> > > > > > > > > >> > > > > As for taking a fast SAN snapshot and exporting it > > > > >> asynchronously, do > > > > >> > > we > > > > >> > > > > see the SSVM as performing the export? > > > > >> > > > > > > > > >> > > > > To be backwards compatible with what we have in 4.6 and > > later > > > > for > > > > >> > > volume > > > > >> > > > > snapshots for managed storage, I think it might be easier > if > > > we > > > > >> pass > > > > >> > > in a > > > > >> > > > > flag that says whether or not to archive the SAN snapshot > > > > (which, > > > > >> I > > > > >> > > > think, > > > > >> > > > > is something that you suggested, as well, Pierre-Luc). > > > > >> > > > > > > > > >> > > > > On Monday, February 8, 2016, Pierre-Luc Dion < > > > > pd...@cloudops.com > > > > >> > > > > <javascript:;>> wrote: > > > > >> > > > > > > > > >> > > > > > Hi Mike, > > > > >> > > > > > > > > > >> > > > > > The reason behind the creation of a SAN snapshot which > is > > > > >> exported > > > > >> > > into > > > > >> > > > > > secondary storage, is because creating a copy of the > .VHD > > > > >> directly > > > > >> > > > would > > > > >> > > > > > impact uptime of the VM as creating that copy take lots > of > > > > time. > > > > >> > Has > > > > >> > > > > oppose > > > > >> > > > > > to a SAN snapshot that is close to instantaneous which > can > > > > >> > afterward > > > > >> > > be > > > > >> > > > > > clone into Secondary Storage asynchronously. > > > > >> > > > > > > > > > >> > > > > > I would suspect an extracted VolumeSnapshot taken from a > > SAN > > > > >> > snapshot > > > > >> > > > > could > > > > >> > > > > > have is SAN snapshot deleted to avoid duplica and space > > > > >> consumption > > > > >> > > on > > > > >> > > > > the > > > > >> > > > > > Primary Storage side. > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > I see 3 definitions in our current discussion regarding > > the > > > > term > > > > >> > > > snapshot > > > > >> > > > > > (these are not official terminology but by own > > > interpretation > > > > of > > > > >> > > them): > > > > >> > > > > > > > > > >> > > > > > 1. *Snapshot* (AKA: Storage Snapshot / Mike's definition > > of > > > a > > > > >> > > > snapshot): > > > > >> > > > > > it's a volume snapshot at the storage level, point in > time > > > of > > > > >> your > > > > >> > > > data. > > > > >> > > > > it > > > > >> > > > > > reside on the primary storage. Useful and efficient for > > > > software > > > > >> > side > > > > >> > > > > > incident. > > > > >> > > > > > 2. *Cloud Snapshot *( AKA: CloudStack VolumeSnapshot/ > > cloud > > > > >> backup > > > > >> > > > aws-S3 > > > > >> > > > > > style ): Point in time copy of the Virtual Disk that > > reside > > > > on a > > > > >> > > > > different > > > > >> > > > > > storage array then the original Volume. Facilitate data > > > > >> migration > > > > >> > > > between > > > > >> > > > > > clusters and, in case of primary storage incident, > Volume > > > > >> snapshots > > > > >> > > are > > > > >> > > > > not > > > > >> > > > > > impacted and can be reuse. > > > > >> > > > > > 3. *Backup*: Archival of your Virtual-machines data that > > > also > > > > >> > > validate > > > > >> > > > > data > > > > >> > > > > > integrity, provide a storage efficient archiving method > > and > > > an > > > > >> > > > > independent > > > > >> > > > > > way to restore your data in case of an major > > infrastructure > > > > >> > disaster. > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > Regards, > > > > >> > > > > > > > > > >> > > > > > PL > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > On Fri, Feb 5, 2016 at 1:34 PM, Mike Tutkowski < > > > > >> > > > > > mike.tutkow...@solidfire.com <javascript:;> > > <javascript:;> > > > > >> > > > > > > wrote: > > > > >> > > > > > > > > > >> > > > > > > So, let's see if I currently follow the requirements: > > > > >> > > > > > > > > > > >> > > > > > > * Augment volume snapshots for managed storage to > > > > >> conditionally > > > > >> > > > export > > > > >> > > > > > data > > > > >> > > > > > > to NFS. The current process of taking a snapshot on > the > > > SAN > > > > is > > > > >> > > fine, > > > > >> > > > > but > > > > >> > > > > > > we'd like the option to export the data to NFS, as > well. > > > > >> > > > > > > > > > > >> > > > > > > Questions: > > > > >> > > > > > > > > > > >> > > > > > > Once the data has been exported to NFS, do we keep the > > SAN > > > > >> > snapshot > > > > >> > > > or > > > > >> > > > > > > delete it? > > > > >> > > > > > > > > > > >> > > > > > > If we are deleting the SAN snapshot, then why don't we > > > just > > > > >> copy > > > > >> > > the > > > > >> > > > > VHD > > > > >> > > > > > > from primary to secondary the way we do today for > > > > non-managed > > > > >> > (i.e. > > > > >> > > > > > > traditional) storage? Why create a SAN snapshot in > that > > > > >> scenario? > > > > >> > > > > Perhaps > > > > >> > > > > > > to have the SSVM mount and perform the VHD copy to > > > secondary > > > > >> > > storage > > > > >> > > > > > > instead of a XenServer host? > > > > >> > > > > > > > > > > >> > > > > > > Thanks for the clarification. > > > > >> > > > > > > > > > > >> > > > > > > By the way, to me a backup is when you copy data from > > one > > > > >> storage > > > > >> > > > > system > > > > >> > > > > > to > > > > >> > > > > > > another (regardless of features, if any, to restore > the > > > data > > > > >> in > > > > >> > the > > > > >> > > > > > > future). A snapshot is a point-in-time view of the > data > > > of a > > > > >> > volume > > > > >> > > > and > > > > >> > > > > > > it's stored on the same storage system as the volume. > > > > >> > > > > > > > > > > >> > > > > > > On Fri, Feb 5, 2016 at 11:09 AM, Pierre-Luc Dion < > > > > >> > > pd...@cloudops.com > > > > >> > > > > <javascript:;> > > > > >> > > > > > <javascript:;>> > > > > >> > > > > > > wrote: > > > > >> > > > > > > > > > > >> > > > > > > > That's fun to see that discussion happening. I 100% > > > agree > > > > >> with > > > > >> > > > Paul's > > > > >> > > > > > > > points of view. VolumeSnapshot are not a backup, > but I > > > do > > > > >> > > consider > > > > >> > > > > them > > > > >> > > > > > > as > > > > >> > > > > > > > a safety vest against Primary Storage failure, > because > > > > >> failure > > > > >> > > > append > > > > >> > > > > > > :-( . > > > > >> > > > > > > > > > > > >> > > > > > > > The current proposal around snapshots that reside on > > the > > > > >> > primary > > > > >> > > > > > storage > > > > >> > > > > > > or > > > > >> > > > > > > > snapshots that end in the Secondary Storage is not > to > > > > >> address > > > > >> > > any > > > > >> > > > > kind > > > > >> > > > > > > of > > > > >> > > > > > > > backups requirement because a snapshot is not a > > backup, > > > > >> event > > > > >> > an > > > > >> > > > > > > extracted > > > > >> > > > > > > > VM snapshot. > > > > >> > > > > > > > > > > > >> > > > > > > > The main idea, and again this is for managed > storage; > > > > >> > > > > > > > > > > > >> > > > > > > > 1. StorageSnapshotAPI: Provide storage side snapshot > > > > >> capability > > > > >> > > for > > > > >> > > > > > fast > > > > >> > > > > > > > response time that support rollback to previous > > > timestamp, > > > > >> > create > > > > >> > > > new > > > > >> > > > > > > > volume and maybe create template. > > > > >> > > > > > > > not required to be a new API if the work is > > already > > > > >> done, I > > > > >> > > > think > > > > >> > > > > > > this > > > > >> > > > > > > > is a different behaviors than the user expectation > of > > a > > > > >> > > > > > volume-snapshot. > > > > >> > > > > > > > 2. VolumeSnapshotAPI: Provide current cloudstack > > > behavior > > > > >> that > > > > >> > > > create > > > > >> > > > > > an > > > > >> > > > > > > > extraction of a volume into SecondaryStorage which > can > > > be > > > > >> reuse > > > > >> > > to > > > > >> > > > > > > create a > > > > >> > > > > > > > new volume into another Primary Storage. This type > of > > > > >> snapshot > > > > >> > > is a > > > > >> > > > > > slow > > > > >> > > > > > > > job since yes it would have to copy the full volume > > size > > > > on > > > > >> the > > > > >> > > > > > Secondary > > > > >> > > > > > > > storage. > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > PL > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > On Fri, Feb 5, 2016 at 12:45 PM, Syed Mushtaq < > > > > >> > > > > syed1.mush...@gmail.com <javascript:;> > > > > >> > > > > > <javascript:;>> > > > > >> > > > > > > > wrote: > > > > >> > > > > > > > > > > > >> > > > > > > > > I think I share you view on the 'Ideal world'. > > Backup > > > > (via > > > > >> > > Volume > > > > >> > > > > > > > > Snapshots) is a huge bottleneck in Cloudstack. > This > > is > > > > >> > > amplified > > > > >> > > > > > > > especially > > > > >> > > > > > > > > when you have a object storage as your secondary > > > storage > > > > >> > > because > > > > >> > > > it > > > > >> > > > > > > > > requires two copies (one to an NFS staging area > and > > > from > > > > >> > there > > > > >> > > to > > > > >> > > > > > > object > > > > >> > > > > > > > > storage). And not to mention that all these copies > > are > > > > >> > > consuming > > > > >> > > > > > > > hypervisor > > > > >> > > > > > > > > resources. Xenserver's Dom0 is also a huge > > bottleneck > > > as > > > > >> all > > > > >> > > the > > > > >> > > > > > > Network > > > > >> > > > > > > > > and I/O flow through it. So our intention of > > proposing > > > > the > > > > >> > > > "Storage > > > > >> > > > > > > > > Snapshots" is to give a better way of achiving > > > snapshots > > > > >> > while > > > > >> > > > > still > > > > >> > > > > > > > > keeping the original definition of volume > snpashots > > > (ie > > > > >> > upload > > > > >> > > to > > > > >> > > > > sec > > > > >> > > > > > > > > storage). > > > > >> > > > > > > > > > > > > >> > > > > > > > > But as Erik pointed out volume snapshots are not > > > > backups. > > > > >> > They > > > > >> > > > > don't > > > > >> > > > > > > work > > > > >> > > > > > > > > form multi-disk LVM volume groups and dynamic > > disks. I > > > > am > > > > >> all > > > > >> > > in > > > > >> > > > > for > > > > >> > > > > > a > > > > >> > > > > > > > > better backup solution which handles these use > cases > > > and > > > > >> > > utilizes > > > > >> > > > > the > > > > >> > > > > > > > > storage's advanced features. > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > On Fri, Feb 5, 2016 at 12:29 PM, Paul Angus < > > > > >> > > > > > paul.an...@shapeblue.com <javascript:;> <javascript:;>> > > > > >> > > > > > > > > wrote: > > > > >> > > > > > > > > > > > > >> > > > > > > > > > In the beginning... there were CloudStack > > snapshots > > > > and > > > > >> > they > > > > >> > > > were > > > > >> > > > > > > > > actually > > > > >> > > > > > > > > > volume snapshots not hypervisor point-in-time > > > > snapshots. > > > > >> > > > > > > > > > Then VM snapshots were created (which are > > > > point-in-time > > > > >> > > > > hypervisor > > > > >> > > > > > > > > > snapshots) and we started referring to the > > original > > > > >> > snapshots > > > > >> > > > as > > > > >> > > > > > > volume > > > > >> > > > > > > > > > snapshots. > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > CloudStack does not offer 'backups', but many > > people > > > > use > > > > >> > > volume > > > > >> > > > > > > > snapshots > > > > >> > > > > > > > > > as backups. However you can't in-place restore > > > volume > > > > >> > > snapshots > > > > >> > > > > and > > > > >> > > > > > > if > > > > >> > > > > > > > > you > > > > >> > > > > > > > > > have a VM with multiple volumes, the volume > > > snapshots > > > > >> must > > > > >> > be > > > > >> > > > > done > > > > >> > > > > > in > > > > >> > > > > > > > > > series, meaning that the state across all of the > > > > >> volumes is > > > > >> > > > > > unlikely > > > > >> > > > > > > to > > > > >> > > > > > > > > be > > > > >> > > > > > > > > > consistent. > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > 'Actual Backups' would enable all of the restore > > > > options > > > > >> > > which > > > > >> > > > > > users > > > > >> > > > > > > > > might > > > > >> > > > > > > > > > expect as well options as to where they might be > > > > >> stored. In > > > > >> > > my > > > > >> > > > > > ideal > > > > >> > > > > > > > > world > > > > >> > > > > > > > > > they would also be able to leverage back-end > > > hardware > > > > >> (such > > > > >> > > as > > > > >> > > > > > > > Solidfire, > > > > >> > > > > > > > > > NetApp etc :) ) and software such as Veeam, > > > Commvault > > > > >> etc > > > > >> > to > > > > >> > > > > > > accelerate > > > > >> > > > > > > > > the > > > > >> > > > > > > > > > process. > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > [image: ShapeBlue] <http://www.shapeblue.com> > > > > >> > > > > > > > > > Paul Angus > > > > >> > > > > > > > > > VP Technology , ShapeBlue > > > > >> > > > > > > > > > d: *+44 203 617 0528 | s: +44 203 603 0540* > > > > >> > > > > > > > > > > > > > >> > > <+44%20203%20617%200528%20%7C%20s:%20+44%20203%20603%200540> > | > > > > >> > > > > m: > > > > >> > > > > > > > > > *+44 7711 418784* <+44%207711%20418784> > > > > >> > > > > > > > > > e: *paul.an...@shapeblue.com <javascript:;> > > > > >> > <javascript:;> > > > > >> > > | > > > > >> > > > > t: @cloudyangus* > > > > >> > > > > > > > > > <paul.an...@shapeblue.com <javascript:;> > > > > >> > > > > <javascript:;>%20%7C%20t:%20@cloudyangus> > > > > >> > > > > > | w: > > > > >> > > > > > > > > > *www.shapeblue.com* <http://www.shapeblue.com> > > > > >> > > > > > > > > > a: 53 Chandos Place, Covent Garden London WC2N > > 4HS > > > UK > > > > >> > > > > > > > > > Shape Blue Ltd is a company incorporated in > > England > > > & > > > > >> > Wales. > > > > >> > > > > > > ShapeBlue > > > > >> > > > > > > > > > Services India LLP is a company incorporated in > > > India > > > > >> and > > > > >> > is > > > > >> > > > > > operated > > > > >> > > > > > > > > under > > > > >> > > > > > > > > > license from Shape Blue Ltd. Shape Blue Brasil > > > > >> Consultoria > > > > >> > > Ltda > > > > >> > > > > is > > > > >> > > > > > a > > > > >> > > > > > > > > > company incorporated in Brasil and is operated > > under > > > > >> > license > > > > >> > > > from > > > > >> > > > > > > Shape > > > > >> > > > > > > > > > Blue Ltd. ShapeBlue SA Pty Ltd is a company > > > registered > > > > >> by > > > > >> > The > > > > >> > > > > > > Republic > > > > >> > > > > > > > of > > > > >> > > > > > > > > > South Africa and is traded under license from > > Shape > > > > Blue > > > > >> > Ltd. > > > > >> > > > > > > ShapeBlue > > > > >> > > > > > > > > is > > > > >> > > > > > > > > > a registered trademark. > > > > >> > > > > > > > > > This email and any attachments to it may be > > > > confidential > > > > >> > and > > > > >> > > > are > > > > >> > > > > > > > intended > > > > >> > > > > > > > > > solely for the use of the individual to whom it > is > > > > >> > addressed. > > > > >> > > > Any > > > > >> > > > > > > views > > > > >> > > > > > > > > or > > > > >> > > > > > > > > > opinions expressed are solely those of the > author > > > and > > > > do > > > > >> > not > > > > >> > > > > > > > necessarily > > > > >> > > > > > > > > > represent those of Shape Blue Ltd or related > > > > companies. > > > > >> If > > > > >> > > you > > > > >> > > > > are > > > > >> > > > > > > not > > > > >> > > > > > > > > the > > > > >> > > > > > > > > > intended recipient of this email, you must > neither > > > > take > > > > >> any > > > > >> > > > > action > > > > >> > > > > > > > based > > > > >> > > > > > > > > > upon its contents, nor copy or show it to > anyone. > > > > Please > > > > >> > > > contact > > > > >> > > > > > the > > > > >> > > > > > > > > sender > > > > >> > > > > > > > > > if you believe you have received this email in > > > error. > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > -----Original Message----- > > > > >> > > > > > > > > > From: Syed Mushtaq [mailto: > > syed1.mush...@gmail.com > > > > >> > > > > <javascript:;> <javascript:;>] > > > > >> > > > > > > > > > Sent: Friday, February 5, 2016 4:58 PM > > > > >> > > > > > > > > > To: dev@cloudstack.apache.org <javascript:;> > > > > >> > <javascript:;> > > > > >> > > > > > > > > > Subject: Re: [Propose][New Feature] Storage > > > Snapshots > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > Paul, > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > When you say actual backups, how would it be > > > different > > > > >> from > > > > >> > > the > > > > >> > > > > > > Volume > > > > >> > > > > > > > > > Snapshots that exist currently. My understanding > > is > > > > that > > > > >> > > > Backups > > > > >> > > > > > end > > > > >> > > > > > > up > > > > >> > > > > > > > > in > > > > >> > > > > > > > > > Sec Storage whereas Snapshots are just a > > > point-in-time > > > > >> > state > > > > >> > > of > > > > >> > > > > > your > > > > >> > > > > > > > > volume > > > > >> > > > > > > > > > which can be restored back correct? > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > -Syed > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > On Fri, Feb 5, 2016 at 11:23 AM, Paul Angus < > > > > >> > > > > > > paul.an...@shapeblue.com <javascript:;> > <javascript:;>> > > > > >> > > > > > > > > > wrote: > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > Hi Syed, > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > As I understand it, the SolidFire plugin will > > > export > > > > >> the > > > > >> > > > > snapshot > > > > >> > > > > > > to > > > > >> > > > > > > > > > > secondary storage if the user requests a > > template > > > > from > > > > >> > the > > > > >> > > > > > snapshot > > > > >> > > > > > > > or > > > > >> > > > > > > > > > > wants to download the snapshot from the cloud. > > > This > > > > >> is a > > > > >> > > > good, > > > > >> > > > > > > > > > > pragmatic approach and yes Mike the SolidFire > > > > storage > > > > >> is > > > > >> > > > super > > > > >> > > > > > > > > > > reliable and snapshots on SolidFire arrays > take > > up > > > > >> next > > > > >> > to > > > > >> > > no > > > > >> > > > > > > space. > > > > >> > > > > > > > > > > BUT I think that we are talking about a more > > > general > > > > >> > > purpose > > > > >> > > > > API, > > > > >> > > > > > > and > > > > >> > > > > > > > > > > other storage systems may not be as awesome as > > > > Mike's. > > > > >> > > That's > > > > >> > > > > my > > > > >> > > > > > > > > > > concern. Also, the time to transfer for say > 1TB > > to > > > > >> move > > > > >> > > from > > > > >> > > > > > > primary > > > > >> > > > > > > > > > > to sec storage and then create a VM template > out > > > of > > > > it > > > > >> > may > > > > >> > > be > > > > >> > > > > too > > > > >> > > > > > > > long > > > > >> > > > > > > > > > for users. > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > @Mike I don’t think 'we' use the term volume > > > > snapshot > > > > >> for > > > > >> > > > > backup, > > > > >> > > > > > > > it's > > > > >> > > > > > > > > > > just that users want to do backups and a > volume > > > > >> snapshot > > > > >> > is > > > > >> > > > the > > > > >> > > > > > > only > > > > >> > > > > > > > > > > type of snapshot that copies the disk > elsewhere > > > and > > > > >> can > > > > >> > be > > > > >> > > > > > > scheduled. > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > I'm 'pondering' the implications of enabling > > > actual > > > > >> > backups > > > > >> > > > > > > (through > > > > >> > > > > > > > > > > recognised backup providers) and the user > > > > requirements > > > > >> > > around > > > > >> > > > > > them > > > > >> > > > > > > > > > > (particularly restoration use cases) as a > > separate > > > > >> thread > > > > >> > > of > > > > >> > > > > > work. > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > [image: ShapeBlue] <http://www.shapeblue.com> > > > Paul > > > > >> Angus > > > > >> > > VP > > > > >> > > > > > > > Technology > > > > >> > > > > > > > > > > , ShapeBlue > > > > >> > > > > > > > > > > d: *+44 203 617 0528 | s: +44 203 603 0540* > > > > >> > > > > > > > > > > > > > > >> > > <+44%20203%20617%200528%20%7C%20s:%20+44%20203%20603%200540> > > > > >> > > > | > > > > >> > > > > m: > > > > >> > > > > > > > > > > *+44 7711 418784* <+44%207711%20418784> > > > > >> > > > > > > > > > > e: *paul.an...@shapeblue.com <javascript:;> > > > > >> > > <javascript:;> | > > > > >> > > > > t: @cloudyangus* > > > > >> > > > > > > > > > > <paul.an...@shapeblue.com <javascript:;> > > > > >> <javascript:;> > > > > >> > > > > > %20%7C%20t:%20@cloudyangus> | w: > > > > >> > > > > > > > > > > *www.shapeblue.com* <http://www.shapeblue.com > > > > > > >> > > > > > > > > > > a: 53 Chandos Place, Covent Garden London WC2N > > 4HS > > > > UK > > > > >> > Shape > > > > >> > > > > Blue > > > > >> > > > > > > Ltd > > > > >> > > > > > > > > > > is a company incorporated in England & Wales. > > > > >> ShapeBlue > > > > >> > > > > Services > > > > >> > > > > > > > India > > > > >> > > > > > > > > > > LLP is a company incorporated in India and is > > > > operated > > > > >> > > under > > > > >> > > > > > > license > > > > >> > > > > > > > > > > from Shape Blue Ltd. Shape Blue Brasil > > Consultoria > > > > >> Ltda > > > > >> > is > > > > >> > > a > > > > >> > > > > > > company > > > > >> > > > > > > > > > > incorporated in Brasil and is operated under > > > license > > > > >> from > > > > >> > > > Shape > > > > >> > > > > > > Blue > > > > >> > > > > > > > > > > Ltd. ShapeBlue SA Pty Ltd is a company > > registered > > > by > > > > >> The > > > > >> > > > > Republic > > > > >> > > > > > > of > > > > >> > > > > > > > > > > South Africa and is traded under license from > > > Shape > > > > >> Blue > > > > >> > > Ltd. > > > > >> > > > > > > > > > > ShapeBlue is a registered trademark. > > > > >> > > > > > > > > > > This email and any attachments to it may be > > > > >> confidential > > > > >> > > and > > > > >> > > > > are > > > > >> > > > > > > > > > > intended solely for the use of the individual > to > > > > whom > > > > >> it > > > > >> > is > > > > >> > > > > > > > addressed. > > > > >> > > > > > > > > > > Any views or opinions expressed are solely > those > > > of > > > > >> the > > > > >> > > > author > > > > >> > > > > > and > > > > >> > > > > > > do > > > > >> > > > > > > > > > > not necessarily represent those of Shape Blue > > Ltd > > > or > > > > >> > > related > > > > >> > > > > > > > > > > companies. If you are not the intended > recipient > > > of > > > > >> this > > > > >> > > > email, > > > > >> > > > > > you > > > > >> > > > > > > > > > > must neither take any action based upon its > > > > contents, > > > > >> nor > > > > >> > > > copy > > > > >> > > > > or > > > > >> > > > > > > > show > > > > >> > > > > > > > > > > it to anyone. Please contact the sender if you > > > > believe > > > > >> > you > > > > >> > > > have > > > > >> > > > > > > > > received > > > > >> > > > > > > > > > this email in error. > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > -----Original Message----- > > > > >> > > > > > > > > > > From: Syed Mushtaq [mailto: > > > syed1.mush...@gmail.com > > > > >> > > > > <javascript:;> > > > > >> > > > > > <javascript:;>] > > > > >> > > > > > > > > > > Sent: 05 February 2016 15:31 > > > > >> > > > > > > > > > > To: dev@cloudstack.apache.org <javascript:;> > > > > >> > > <javascript:;> > > > > >> > > > > > > > > > > Subject: Re: [Propose][New Feature] Storage > > > > Snapshots > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > I think the terminology confusion comes from > AWS > > > > where > > > > >> > they > > > > >> > > > do > > > > >> > > > > > EBS > > > > >> > > > > > > > > > > snapshots backed up to S3 and CloudStack sort > of > > > > >> followed > > > > >> > > > that. > > > > >> > > > > > And > > > > >> > > > > > > > as > > > > >> > > > > > > > > > > an end user who is oblivious to the internals > of > > > my > > > > >> > > provider, > > > > >> > > > > my > > > > >> > > > > > > > > > > expectation would be something similar to what > > AWS > > > > as > > > > >> > that > > > > >> > > is > > > > >> > > > > my > > > > >> > > > > > > > > > > biggest reference point. > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > To your point Mike, I agree that a Primary > > Storage > > > > >> > failure > > > > >> > > on > > > > >> > > > > > > > > > > SolidFire is unlikely, there are other > > motivations > > > > >> for us > > > > >> > > to > > > > >> > > > > push > > > > >> > > > > > > > data > > > > >> > > > > > > > > > > to secondary storage. Primary storage (atleast > > for > > > > us) > > > > >> > > costs > > > > >> > > > > > > around 3 > > > > >> > > > > > > > > > > times as much as secondary storage and > snapshots > > > on > > > > >> > primary > > > > >> > > > > > storage > > > > >> > > > > > > > > > > are rarely used (especially for some of our > > > > customers > > > > >> who > > > > >> > > do > > > > >> > > > > > daily > > > > >> > > > > > > > > > backups). > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > On Fri, Feb 5, 2016 at 10:18 AM, Mike > Tutkowski > > < > > > > >> > > > > > > > > > > mike.tutkow...@solidfire.com <javascript:;> > > > > >> > > <javascript:;>> > > > > >> > > > > wrote: > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > Some of the weirdness is around terminology. > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > For most systems I've worked on, a snapshot > > and > > > a > > > > >> > backup > > > > >> > > > are > > > > >> > > > > > two > > > > >> > > > > > > > > > > > completely different things (but CloudStack > > has > > > > >> > > > traditionally > > > > >> > > > > > > used > > > > >> > > > > > > > > > > > the term "volume snapshot" to mean backup). > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > I will put in a SolidFire "plug" here and > say, > > > > >> though, > > > > >> > > that > > > > >> > > > > if > > > > >> > > > > > > your > > > > >> > > > > > > > > > > > primary storage is running on SolidFire that > > it > > > is > > > > >> > > unlikely > > > > >> > > > > > > you'll > > > > >> > > > > > > > > > > > encounter an issue where your primary > storage > > > goes > > > > >> > > offline > > > > >> > > > > (and > > > > >> > > > > > > > > > > > you'll even maintain your performance > > guarantees > > > > >> during > > > > >> > > > > failure > > > > >> > > > > > > > > > > > scenarios and upgrades, as well). That being > > the > > > > >> case, > > > > >> > it > > > > >> > > > is > > > > >> > > > > > less > > > > >> > > > > > > > > > > > useful to require a backup to Swift (but > it's > > > > >> perfectly > > > > >> > > OK > > > > >> > > > if > > > > >> > > > > > > > that's > > > > >> > > > > > > > > > > > what we want to do > > > > >> > > > > > > > > > > here). > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > On Fri, Feb 5, 2016 at 8:07 AM, Syed Mushtaq > > > > >> > > > > > > > > > > > <syed1.mush...@gmail.com <javascript:;> > > > > >> > <javascript:;>> > > > > >> > > > > > > > > > > > wrote: > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > Paul, > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > I believe with the current implementation > of > > > > >> > Snapshots > > > > >> > > on > > > > >> > > > > > > managed > > > > >> > > > > > > > > > > > > storage > > > > >> > > > > > > > > > > > > (SolidFire) the snapshots are never > exported > > > to > > > > >> the > > > > >> > > > > secondary > > > > >> > > > > > > > > > storage. > > > > >> > > > > > > > > > > > > While this solves the problem of having > > > > snapshots > > > > >> > > taking > > > > >> > > > > > > forever > > > > >> > > > > > > > > > > > > to get to sec storage, this leaves us > with a > > > > >> > > > > > > > > > > > huge > > > > >> > > > > > > > > > > > > liability if our primary storage goes > down. > > We > > > > see > > > > >> > > > > snapshots > > > > >> > > > > > as > > > > >> > > > > > > > > > > > > our recovery path because we store them in > > > Swift > > > > >> > which > > > > >> > > is > > > > >> > > > > > > > reliable > > > > >> > > > > > > > > > > > > and resilient to failures. > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > With Storage snpashots our goal is to have > > > > Volume > > > > >> > > > snapshots > > > > >> > > > > > > > always > > > > >> > > > > > > > > > > > > backed up to secondary storage and Storage > > > > >> Snapshots > > > > >> > > stay > > > > >> > > > > on > > > > >> > > > > > > the > > > > >> > > > > > > > > > > > > primary > > > > >> > > > > > > > > > > > storage. > > > > >> > > > > > > > > > > > > A provider could potentially mix both > these > > > and > > > > >> solve > > > > >> > > the > > > > >> > > > > > > problem > > > > >> > > > > > > > > > > > > that you mentioned where you want to meet > > > user's > > > > >> > > > > expectation > > > > >> > > > > > > of a > > > > >> > > > > > > > > > > > > snapshot (ie backup to sec storage) while > > > having > > > > >> an > > > > >> > > > ability > > > > >> > > > > > to > > > > >> > > > > > > > > > > > > utilize faster sanpshots (i.e. on the > > device) > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > Hope this clarifies things. > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > Thanks, > > > > >> > > > > > > > > > > > > -Syed > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > On Fri, Feb 5, 2016 at 9:04 AM, Paul Angus > > > > >> > > > > > > > > > > > > <paul.an...@shapeblue.com <javascript:;> > > > > >> > > <javascript:;>> > > > > >> > > > > > > > > > > > > wrote: > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > HI guys, > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > Could someone point me to the Jira bug > of > > FS > > > > for > > > > >> > the > > > > >> > > > > > > > > > > > > > SAN-snapshot > > > > >> > > > > > > > > > > > feature > > > > >> > > > > > > > > > > > > > in 4.6 which is mentioned. > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > From my discussions with users and > > operators > > > > >> around > > > > >> > > > > > snapshots > > > > >> > > > > > > > > > > > > > I'd make > > > > >> > > > > > > > > > > > > the > > > > >> > > > > > > > > > > > > > following observations: > > > > >> > > > > > > > > > > > > > a. 'users' use snapshots as backups > (both > > > > >> long-term > > > > >> > > and > > > > >> > > > > > short > > > > >> > > > > > > > > > > > > > term) > > > > >> > > > > > > > > > > > with > > > > >> > > > > > > > > > > > > > the expectation that they can use them > for > > > > >> recovery > > > > >> > > if > > > > >> > > > > > > > required. > > > > >> > > > > > > > > > > > > > b. operators fall back to snapshots if > > > > something > > > > >> > has > > > > >> > > > gone > > > > >> > > > > > > wrong > > > > >> > > > > > > > > > > > > > with primary storage. > > > > >> > > > > > > > > > > > > > c. users sometimes want to be able to > > export > > > > >> > > snapshots > > > > >> > > > as > > > > >> > > > > > > well > > > > >> > > > > > > > > > > > > > as > > > > >> > > > > > > > > > > > create > > > > >> > > > > > > > > > > > > > new VMs from their snapshots > > > > >> > > > > > > > > > > > > > d. snapshots are a currently a massive > > pain > > > > for > > > > >> > > > > operators, > > > > >> > > > > > I > > > > >> > > > > > > > > > > > > > know at > > > > >> > > > > > > > > > > > > least > > > > >> > > > > > > > > > > > > > one public cloud who have snapshots > which > > > > take 2 > > > > >> > days > > > > >> > > > to > > > > >> > > > > > > > > complete. > > > > >> > > > > > > > > > > > > > e. snapshots (as they are) can't be used > > for > > > > >> > multiple > > > > >> > > > LVM > > > > >> > > > > > > > disks. > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > I think the process Mike has used in the > > > > >> SolidFire > > > > >> > > > plugin > > > > >> > > > > > > (only > > > > >> > > > > > > > > > > > > > moving > > > > >> > > > > > > > > > > > > the > > > > >> > > > > > > > > > > > > > disk image to secondary storage when you > > > > >> absolutely > > > > >> > > > have > > > > >> > > > > > to) > > > > >> > > > > > > is > > > > >> > > > > > > > > > > > > > a very > > > > >> > > > > > > > > > > > > good > > > > >> > > > > > > > > > > > > > and pragmatic solution. I wonder what > > > problems > > > > >> an > > > > >> > > > > operator > > > > >> > > > > > > > might > > > > >> > > > > > > > > > > > > experience > > > > >> > > > > > > > > > > > > > if they have an issue with a given > primary > > > > >> storage > > > > >> > > pool > > > > >> > > > > in > > > > >> > > > > > a > > > > >> > > > > > > > > > cluster. > > > > >> > > > > > > > > > > > (I > > > > >> > > > > > > > > > > > > > know that that is REALLY unlikely in the > > > > >> SolidFire > > > > >> > > case > > > > >> > > > > > Mike > > > > >> > > > > > > :) > > > > >> > > > > > > > > > > > > > ) And > > > > >> > > > > > > > > > > > if > > > > >> > > > > > > > > > > > > > the transfer from primary to secondary > is > > > > slow, > > > > >> the > > > > >> > > > time > > > > >> > > > > to > > > > >> > > > > > > > > > > > > > being able > > > > >> > > > > > > > > > > > to > > > > >> > > > > > > > > > > > > > create a template or export the volume > > will > > > be > > > > >> > slow. > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > So for me the issue is around making > sure > > > that > > > > >> the > > > > >> > > end > > > > >> > > > > > users > > > > >> > > > > > > > > > > > expectations > > > > >> > > > > > > > > > > > > > are met (while improving the > > > speed/efficiency > > > > of > > > > >> > the > > > > >> > > > back > > > > >> > > > > > > end) > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > [image: ShapeBlue] < > > > http://www.shapeblue.com> > > > > >> Paul > > > > >> > > > Angus > > > > >> > > > > > VP > > > > >> > > > > > > > > > > > > > Technology , ShapeBlue > > > > >> > > > > > > > > > > > > > d: *+44 203 617 0528 | s: +44 203 603 > > 0540* > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > <+44%20203%20617%200528%20%7C%20s:%20+44%20203%20603%200540> > > > > >> > > > > > > | > > > > >> > > > > > > > m: > > > > >> > > > > > > > > > > > > > *+44 7711 418784* <+44%207711%20418784> > > > > >> > > > > > > > > > > > > > e: *paul.an...@shapeblue.com > > <javascript:;> > > > > >> > > > > <javascript:;> | t: > > > > >> > > > > > @cloudyangus* > > > > >> > > > > > > > > > > > > > <paul.an...@shapeblue.com > <javascript:;> > > > > >> > > > <javascript:;> > > > > >> > > > > > %20%7C%20t:%20@cloudyangus> | w: > > > > >> > > > > > > > > > > > > > *www.shapeblue.com* < > > > http://www.shapeblue.com > > > > > > > > > >> > > > > > > > > > > > > > a: 53 Chandos Place, Covent Garden > London > > > WC2N > > > > >> 4HS > > > > >> > UK > > > > >> > > > > Shape > > > > >> > > > > > > > Blue > > > > >> > > > > > > > > > > > > > Ltd is a company incorporated in > England & > > > > >> Wales. > > > > >> > > > > ShapeBlue > > > > >> > > > > > > > > > > > > > Services India LLP is a company > > incorporated > > > > in > > > > >> > India > > > > >> > > > and > > > > >> > > > > > is > > > > >> > > > > > > > > > > > > > operated > > > > >> > > > > > > > > > > > > under > > > > >> > > > > > > > > > > > > > license from Shape Blue Ltd. Shape Blue > > > Brasil > > > > >> > > > > Consultoria > > > > >> > > > > > > Ltda > > > > >> > > > > > > > > > > > > > is a company incorporated in Brasil and > is > > > > >> operated > > > > >> > > > under > > > > >> > > > > > > > > > > > > > license from Shape Blue Ltd. ShapeBlue > SA > > > Pty > > > > >> Ltd > > > > >> > is > > > > >> > > a > > > > >> > > > > > > company > > > > >> > > > > > > > > > > > > > registered by The Republic > > > > >> > > > > > > > > > > > of > > > > >> > > > > > > > > > > > > > South Africa and is traded under license > > > from > > > > >> Shape > > > > >> > > > Blue > > > > >> > > > > > Ltd. > > > > >> > > > > > > > > > > > > > ShapeBlue > > > > >> > > > > > > > > > > > > is > > > > >> > > > > > > > > > > > > > a registered trademark. > > > > >> > > > > > > > > > > > > > This email and any attachments to it may > > be > > > > >> > > > confidential > > > > >> > > > > > and > > > > >> > > > > > > > are > > > > >> > > > > > > > > > > > intended > > > > >> > > > > > > > > > > > > > solely for the use of the individual to > > whom > > > > it > > > > >> is > > > > >> > > > > > addressed. > > > > >> > > > > > > > > > > > > > Any views > > > > >> > > > > > > > > > > > > or > > > > >> > > > > > > > > > > > > > opinions expressed are solely those of > the > > > > >> author > > > > >> > and > > > > >> > > > do > > > > >> > > > > > not > > > > >> > > > > > > > > > > > necessarily > > > > >> > > > > > > > > > > > > > represent those of Shape Blue Ltd or > > related > > > > >> > > companies. > > > > >> > > > > If > > > > >> > > > > > > you > > > > >> > > > > > > > > > > > > > are not > > > > >> > > > > > > > > > > > > the > > > > >> > > > > > > > > > > > > > intended recipient of this email, you > must > > > > >> neither > > > > >> > > take > > > > >> > > > > any > > > > >> > > > > > > > > > > > > > action > > > > >> > > > > > > > > > > > based > > > > >> > > > > > > > > > > > > > upon its contents, nor copy or show it > to > > > > >> anyone. > > > > >> > > > Please > > > > >> > > > > > > > contact > > > > >> > > > > > > > > > > > > > the > > > > >> > > > > > > > > > > > > sender > > > > >> > > > > > > > > > > > > > if you believe you have received this > > email > > > in > > > > >> > error. > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > -----Original Message----- > > > > >> > > > > > > > > > > > > > From: Pierre-Luc Dion [mailto: > > > > >> pd...@cloudops.com > > > > >> > > > > <javascript:;> > > > > >> > > > > > <javascript:;>] > > > > >> > > > > > > > > > > > > > Sent: Friday, February 5, 2016 12:56 PM > > > > >> > > > > > > > > > > > > > To: dev@cloudstack.apache.org > > > <javascript:;> > > > > >> > > > > <javascript:;> > > > > >> > > > > > > > > > > > > > Subject: Re: [Propose][New Feature] > > Storage > > > > >> > Snapshots > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > Hi Mike, > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > The idea of introducing a new API: > > > > >> StorageSnapshot > > > > >> > > for > > > > >> > > > > > > managed > > > > >> > > > > > > > > > > > > > storage > > > > >> > > > > > > > > > > > is > > > > >> > > > > > > > > > > > > > because the VolumeSnapshot default, or > > > > expected, > > > > >> > > > behavior > > > > >> > > > > > is > > > > >> > > > > > > to > > > > >> > > > > > > > > > > > > > archive snapshots into the Secondary > > > Storage. > > > > >> So a > > > > >> > > > > > > > > > > > > > StorageSnapshot API would be > > > > >> > > > > > > > > > > > > for > > > > >> > > > > > > > > > > > > > snapshot that remain on the managed > > storage > > > > >> > > appliance. > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > Quickly looking at the API doc and I > don't > > > > see a > > > > >> > > strong > > > > >> > > > > > > > > > > > > > requirement for volume snapshots to be > > moved > > > > >> into > > > > >> > > > > secondary > > > > >> > > > > > > > > > > > > > storage. So, maybe StorageSnapshot API > is > > > not > > > > >> > useful, > > > > >> > > > but > > > > >> > > > > > > both > > > > >> > > > > > > > > > > > > > use cases are required. A snapshot that > > > remain > > > > >> on > > > > >> > the > > > > >> > > > > > managed > > > > >> > > > > > > > > > > > > > storage, and another type of > > > > >> > > > > > > > > > > > snapshot > > > > >> > > > > > > > > > > > > > that end up into the secondary storage. > > > Since > > > > >> > you've > > > > >> > > > > done a > > > > >> > > > > > > lot > > > > >> > > > > > > > > > > > > > of > > > > >> > > > > > > > > > > > work, > > > > >> > > > > > > > > > > > > > might easier to just add a parameter to > > the > > > > >> current > > > > >> > > > > > snapshot > > > > >> > > > > > > > API > > > > >> > > > > > > > > > > > > > that > > > > >> > > > > > > > > > > > > would > > > > >> > > > > > > > > > > > > > trigger an extraction of the storage > > > snapshot > > > > >> into > > > > >> > > the > > > > >> > > > > > > > secondary > > > > >> > > > > > > > > > > > storage? > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > PL > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > On Thu, Feb 4, 2016 at 9:02 PM, Mike > > > > Tutkowski < > > > > >> > > > > > > > > > > > > > mike.tutkow...@solidfire.com > > <javascript:;> > > > > >> > > > > <javascript:;> > > > > >> > > > > > > > > > > > > > > wrote: > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > I think that all sounds reasonable > then > > - > > > > >> thanks! > > > > >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > On Thu, Feb 4, 2016 at 6:52 PM, Syed > > > > Mushtaq < > > > > >> > > > > > > > > > > > syed1.mush...@gmail.com <javascript:;> > > > > >> <javascript:;>> > > > > >> > > > > > > > > > > > > > > wrote: > > > > >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> You are correct Mike in terms of the > > > > >> > requirements. > > > > >> > > > One > > > > >> > > > > > of > > > > >> > > > > > > > our > > > > >> > > > > > > > > > > > earlier > > > > >> > > > > > > > > > > > > > >> iterations on this was to have an > > > argument > > > > to > > > > >> > the > > > > >> > > > > create > > > > >> > > > > > > > > > > > > > >> snapshot > > > > >> > > > > > > > > > > > API > > > > >> > > > > > > > > > > > > > >> which decides whether to backup the > > > volume > > > > to > > > > >> > sec > > > > >> > > > > > storage > > > > >> > > > > > > > but > > > > >> > > > > > > > > > > > > > >> we realized it would make management > of > > > > >> > snapshots > > > > >> > > > > quite > > > > >> > > > > > > > messy > > > > >> > > > > > > > > > > > > > >> so we proposed a new api instead. > > > > >> > > > > > > > > > > > > > >> > > > > >> > > > > > > > > > > > > > >> On Thu, Feb 4, 2016, 8:29 PM Mike > > > Tutkowski > > > > >> > > > > > > > > > > > > > >> <mike.tutkow...@solidfire.com > > > > <javascript:;> > > > > >> > > > > <javascript:;>> > > > > >> > > > > > > > > > > > > > >> wrote: > > > > >> > > > > > > > > > > > > > >> > > > > >> > > > > > > > > > > > > > >>> Hi, > > > > >> > > > > > > > > > > > > > >>> > > > > >> > > > > > > > > > > > > > >>> Just to make sure I understand all > the > > > > >> > > requirements > > > > >> > > > > > here: > > > > >> > > > > > > > > > > > > > >>> > > > > >> > > > > > > > > > > > > > >>> 1) This relates only to managed > > storage > > > > (1:1 > > > > >> > > > mapping > > > > >> > > > > > > > between > > > > >> > > > > > > > > > > > > > >>> a virtual disk and a backend SAN > > > volume). > > > > >> > > > > > > > > > > > > > >>> > > > > >> > > > > > > > > > > > > > >>> 2) We want to take the current > > > (introduced > > > > >> in > > > > >> > > 4.6) > > > > >> > > > > > > > > > > > > > >>> functionality, which creates a > > snapshot > > > on > > > > >> the > > > > >> > > SAN, > > > > >> > > > > and > > > > >> > > > > > > > > > > > > > >>> extend it via a config option (or > > > > >> > > > > > > > > > > > > > >>> something) to not only take the SAN > > > > >> snapshot, > > > > >> > but > > > > >> > > > to > > > > >> > > > > > copy > > > > >> > > > > > > > > > > > > > >>> the underlying VHD (XenServer only) > to > > > > NFS. > > > > >> > > > > > > > > > > > > > >>> > > > > >> > > > > > > > > > > > > > >>> 3) The SAN snapshot is always taken. > > > It's > > > > >> the > > > > >> > > > backup > > > > >> > > > > to > > > > >> > > > > > > NFS > > > > >> > > > > > > > > > > > > > >>> that is optional. > > > > >> > > > > > > > > > > > > > >>> > > > > >> > > > > > > > > > > > > > >>> 4) Templates can be created from the > > > > >> snapshot > > > > >> > > > that's > > > > >> > > > > on > > > > >> > > > > > > the > > > > >> > > > > > > > > > > > > > >>> SAN (already works). > > > > >> > > > > > > > > > > > > > >>> > > > > >> > > > > > > > > > > > > > >>> 5) CloudStack volumes can be created > > > from > > > > >> the > > > > >> > > > > snapshot > > > > >> > > > > > > > > > > > > > >>> that's on > > > > >> > > > > > > > > > > > the > > > > >> > > > > > > > > > > > > > >>> SAN (already works as long as the > new > > > > >> > CloudStack > > > > >> > > > > volume > > > > >> > > > > > > > ends > > > > >> > > > > > > > > > > > > > >>> up on the same primary storage). > > > > >> > > > > > > > > > > > > > >>> > > > > >> > > > > > > > > > > > > > >>> Would we have a need for a storage > > > > snapshot > > > > >> API > > > > >> > > > then > > > > >> > > > > or > > > > >> > > > > > > > > > > > > > >>> would that just be the standard > volume > > > > >> snapshot > > > > >> > > > > without > > > > >> > > > > > > the > > > > >> > > > > > > > > > > > > > >>> backup to > > > > >> > > > > > > > > > > NFS? > > > > >> > > > > > > > > > > > > > >>> > > > > >> > > > > > > > > > > > > > >>> Thanks! > > > > >> > > > > > > > > > > > > > >>> Mike > > > > >> > > > > > > > > > > > > > >>> > > > > >> > > > > > > > > > > > > > >>> On Thu, Feb 4, 2016 at 5:24 PM, Syed > > > > Mushtaq > > > > >> > > > > > > > > > > > > > >>> <syed1.mush...@gmail.com > > <javascript:;> > > > > >> > > > > <javascript:;>> > > > > >> > > > > > > > > > > > > > >>> wrote: > > > > >> > > > > > > > > > > > > > >>> > > > > >> > > > > > > > > > > > > > >>>> Is it possible to have both > > > > functionalities > > > > >> > > > > (snapshot > > > > >> > > > > > on > > > > >> > > > > > > > > > > > > > >>>> SAN & Sec > > > > >> > > > > > > > > > > > > > >>>> Storage) coexist? Because Ideally, > we > > > > would > > > > >> > like > > > > >> > > > to > > > > >> > > > > > have > > > > >> > > > > > > > > both. > > > > >> > > > > > > > > > > > > > >>>> For example, some of our customers > > want > > > > to > > > > >> > > > implement > > > > >> > > > > > > their > > > > >> > > > > > > > > > > > > > >>>> own backup strategies and do > > encryption > > > > to > > > > >> > their > > > > >> > > > > > backups > > > > >> > > > > > > > > > > > > > >>>> which is a perfect use case for > > Storage > > > > >> > Snapshot > > > > >> > > > > while > > > > >> > > > > > > our > > > > >> > > > > > > > > > > > > > >>>> other customers will still keep > using > > > the > > > > >> > > standard > > > > >> > > > > > > volume > > > > >> > > > > > > > > > > snapshot. > > > > >> > > > > > > > > > > > > > >>>> > > > > >> > > > > > > > > > > > > > >>>> To keep things backward compatible, > > we > > > > can > > > > >> > add a > > > > >> > > > > > setting > > > > >> > > > > > > > > > > > > > >>>> which > > > > >> > > > > > > > > > > > says > > > > >> > > > > > > > > > > > > > >>>> to not upload on secondary storage, > > > > >> because, > > > > >> > > after > > > > >> > > > > > all, > > > > >> > > > > > > > you > > > > >> > > > > > > > > > > > > > >>>> would take a SAN snapshot first > when > > > > doing > > > > >> a > > > > >> > > > Volume > > > > >> > > > > > > > > Snapshot. > > > > >> > > > > > > > > > > > > > >>>> You could stop the process there > and > > > not > > > > do > > > > >> > the > > > > >> > > > > > upload. > > > > >> > > > > > > > > > > > > > >>>> > > > > >> > > > > > > > > > > > > > >>>> What do you think about this > > approach? > > > > >> > > > > > > > > > > > > > >>>> > > > > >> > > > > > > > > > > > > > >>>> Thanks, > > > > >> > > > > > > > > > > > > > >>>> -Syed > > > > >> > > > > > > > > > > > > > >>>> > > > > >> > > > > > > > > > > > > > >>>> On Thu, Feb 4, 2016 at 4:25 PM, > Mike > > > > >> > Tutkowski < > > > > >> > > > > > > > > > > > > > >>>> mike.tutkow...@solidfire.com > > > > >> <javascript:;> > > > > >> > > > > <javascript:;>> wrote: > > > > >> > > > > > > > > > > > > > >>>> > > > > >> > > > > > > > > > > > > > >>>>> So, this is just me thinking out > > load > > > > >> here, > > > > >> > but > > > > >> > > > if > > > > >> > > > > a > > > > >> > > > > > > > given > > > > >> > > > > > > > > > > > > > >>>>> CloudStack cloud doesn't actually > > need > > > > to > > > > >> > > provide > > > > >> > > > > > both > > > > >> > > > > > > > the > > > > >> > > > > > > > > > > > ability > > > > >> > > > > > > > > > > > > > >>>>> to take a SAN snapshot and export > it > > > to > > > > >> NFS > > > > >> > (if > > > > >> > > > > just > > > > >> > > > > > > > > > > > > > >>>>> taking a SAN snapshot is OK), then > > we > > > > >> might > > > > >> > be > > > > >> > > > able > > > > >> > > > > > to > > > > >> > > > > > > > get > > > > >> > > > > > > > > > > > > > >>>>> away with no new > > > > >> > > > > > > > > > > > API > > > > >> > > > > > > > > > > > > > >>>>> calls and simply implement a new > > > custom > > > > >> > > snapshot > > > > >> > > > > > > strategy > > > > >> > > > > > > > > > > > > > >>>>> and > > > > >> > > > > > > > > > > > data > > > > >> > > > > > > > > > > > > > >>>>> motion strategy to handle the case > > > where > > > > >> the > > > > >> > > > > > CloudStack > > > > >> > > > > > > > > > > > > > >>>>> cloud > > > > >> > > > > > > > > > > > does > > > > >> > > > > > > > > > > > > > >>>>> want both a SAN snapshot and > > > > >> exported-to-NFS > > > > >> > > > > backup. > > > > >> > > > > > > > > > > > > > >>>>> > > > > >> > > > > > > > > > > > > > >>>>> In other words, the "default" > > behavior > > > > >> would > > > > >> > be > > > > >> > > > to > > > > >> > > > > > use > > > > >> > > > > > > > the > > > > >> > > > > > > > > > > > > > >>>>> snapshot strategy and data motion > > > > strategy > > > > >> > that > > > > >> > > > we > > > > >> > > > > > > > already > > > > >> > > > > > > > > > > > > > >>>>> have (the one that only takes a > SAN > > > > >> > snapshot). > > > > >> > > > > > > > > > > > > > >>>>> > > > > >> > > > > > > > > > > > > > >>>>> If your CloudStack cloud, however, > > > wants > > > > >> to > > > > >> > > take > > > > >> > > > a > > > > >> > > > > > SAN > > > > >> > > > > > > > > > > > > > >>>>> snapshot and have the data > exported > > to > > > > >> NFS, > > > > >> > > then > > > > >> > > > we > > > > >> > > > > > > could > > > > >> > > > > > > > > > > > > > >>>>> have you manipulate a Swing config > > > file > > > > to > > > > >> > make > > > > >> > > > use > > > > >> > > > > > of > > > > >> > > > > > > a > > > > >> > > > > > > > > > > > > > >>>>> new snapshot strategy and data > > motion > > > > >> > strategy > > > > >> > > > that > > > > >> > > > > > > > > > > > > > >>>>> performs both of these > > > > >> > > > > > > > > > > > > > activities. > > > > >> > > > > > > > > > > > > > >>>>> > > > > >> > > > > > > > > > > > > > >>>>> This way, the old behavior is > still > > > the > > > > >> > default > > > > >> > > > for > > > > >> > > > > > > > users, > > > > >> > > > > > > > > > > > > > >>>>> but CloudStack admins can change > > this > > > > >> > behavior > > > > >> > > > via > > > > >> > > > > > > > > > > configuration. > > > > >> > > > > > > > > > > > > > >>>>> > > > > >> > > > > > > > > > > > > > >>>>> Thoughts? > > > > >> > > > > > > > > > > > > > >>>>> > > > > >> > > > > > > > > > > > > > >>>>> On Thu, Feb 4, 2016 at 11:55 AM, > > Mike > > > > >> > > Tutkowski < > > > > >> > > > > > > > > > > > > > >>>>> mike.tutkow...@solidfire.com > > > > >> <javascript:;> > > > > >> > > > > <javascript:;>> wrote: > > > > >> > > > > > > > > > > > > > >>>>> > > > > >> > > > > > > > > > > > > > >>>>>> Right...I think we will need to > > come > > > up > > > > >> > with a > > > > >> > > > > > viable > > > > >> > > > > > > > > > > > > > >>>>>> upgrade path or some reasonable > way > > > for > > > > >> them > > > > >> > > to > > > > >> > > > > move > > > > >> > > > > > > > from > > > > >> > > > > > > > > > > > > > >>>>>> the old way to the new way (and > > some > > > > >> obvious > > > > >> > > way > > > > >> > > > > > that > > > > >> > > > > > > > > > > > > > >>>>>> they will know they need > > > > >> > > > > > > > > > > > to > > > > >> > > > > > > > > > > > > > do this). > > > > >> > > > > > > > > > > > > > >>>>>> > > > > >> > > > > > > > > > > > > > >>>>>> On Thu, Feb 4, 2016 at 11:45 AM, > > Syed > > > > >> > Mushtaq > > > > >> > > < > > > > >> > > > > > > > > > > > > > >>>>>> syed1.mush...@gmail.com > > > <javascript:;> > > > > >> > > > > <javascript:;>> wrote: > > > > >> > > > > > > > > > > > > > >>>>>> > > > > >> > > > > > > > > > > > > > >>>>>>> I'm not really sure about the > > > upgrade > > > > >> path > > > > >> > > > > however, > > > > >> > > > > > > > > > > > > > >>>>>>> customers who are using 4.6 and > > are > > > > on a > > > > >> > > > managed > > > > >> > > > > > > > storage > > > > >> > > > > > > > > > > > > > >>>>>>> would no longer have the same > > > > >> functionality > > > > >> > > > with > > > > >> > > > > > > Volume > > > > >> > > > > > > > > > > Snapshots. > > > > >> > > > > > > > > > > > > > >>>>>>> > > > > >> > > > > > > > > > > > > > >>>>>>> On Thu, Feb 4, 2016 at 1:43 PM, > > Syed > > > > >> > Mushtaq > > > > >> > > < > > > > >> > > > > > > > > > > > > > >>>>>>> syed1.mush...@gmail.com > > > > <javascript:;> > > > > >> > > > > <javascript:;>> wrote: > > > > >> > > > > > > > > > > > > > >>>>>>> > > > > >> > > > > > > > > > > > > > >>>>>>>> So if I understand correctly, > > > > currently > > > > >> > > > taking a > > > > >> > > > > > > > Volume > > > > >> > > > > > > > > > > > > > >>>>>>>> Snapshots of a volume on a > > managed > > > > >> storage > > > > >> > > > keeps > > > > >> > > > > > it > > > > >> > > > > > > on > > > > >> > > > > > > > > > > > > > >>>>>>>> the storage array. As a part of > > > this > > > > >> > > feature, > > > > >> > > > we > > > > >> > > > > > can > > > > >> > > > > > > > > > > > > > >>>>>>>> make sure > > > > >> > > > > > > > > > > > that > > > > >> > > > > > > > > > > > > > >>>>>>>> Volume Snapshots on managed > > storage > > > > are > > > > >> > > > uploaded > > > > >> > > > > > to > > > > >> > > > > > > > the > > > > >> > > > > > > > > > > > > > >>>>>>>> secondary storage. This would > > make > > > > the > > > > >> > > Volume > > > > >> > > > > > > Snapshot > > > > >> > > > > > > > > > > > > > >>>>>>>> feature behave the same > > regardless > > > of > > > > >> the > > > > >> > > > > storage > > > > >> > > > > > > > > > > > > > >>>>>>>> (managed or > > > > >> > > > > > > > > > > > > > >>>>>>>> non-managed) And, for utilizing > > the > > > > >> > > efficient > > > > >> > > > > > > backend > > > > >> > > > > > > > > > > > > > >>>>>>>> storage > > > > >> > > > > > > > > > > > > > capabilities, we can use the new storage > > > > >> snapshots > > > > >> > > API. > > > > >> > > > > > > > > > > > > > >>>>>>>> > > > > >> > > > > > > > > > > > > > >>>>>>>> On Thu, Feb 4, 2016 at 1:36 PM, > > > Mike > > > > >> > > > Tutkowski < > > > > >> > > > > > > > > > > > > > >>>>>>>> mike.tutkow...@solidfire.com > > > > >> > <javascript:;> > > > > >> > > > > <javascript:;>> > > > > >> > > > > > wrote: > > > > >> > > > > > > > > > > > > > >>>>>>>> > > > > >> > > > > > > > > > > > > > >>>>>>>>> Hi everyone, > > > > >> > > > > > > > > > > > > > >>>>>>>>> > > > > >> > > > > > > > > > > > > > >>>>>>>>> Whatever we do here, we need > to > > > > have a > > > > >> > plan > > > > >> > > > to > > > > >> > > > > > deal > > > > >> > > > > > > > > > > > > > >>>>>>>>> with the fact that we already > > > have a > > > > >> > > feature > > > > >> > > > > (in > > > > >> > > > > > > 4.6 > > > > >> > > > > > > > > > > > > > >>>>>>>>> and > > > > >> > > > > > > > > > > > > > >>>>>>>>> later) that allows you to use > > the > > > > >> > existing > > > > >> > > > > > > > > > > > > > >>>>>>>>> volume-snapshot APIs to > create a > > > > >> volume > > > > >> > > > > snapshot > > > > >> > > > > > > (for > > > > >> > > > > > > > > > > > > > >>>>>>>>> managed > > > > >> > > > > > > > > > > > > > >>>>>>>>> storage) that resides on a > > backend > > > > SAN > > > > >> > > > (using a > > > > >> > > > > > > > custom > > > > >> > > > > > > > > > > > > > >>>>>>>>> snapshot strategy and a custom > > > data > > > > >> > motion > > > > >> > > > > > > strategy). > > > > >> > > > > > > > > > > > > > >>>>>>>>> > > > > >> > > > > > > > > > > > > > >>>>>>>>> If these new APIs go in, then > > how > > > > >> should > > > > >> > > the > > > > >> > > > > > > original > > > > >> > > > > > > > > > > > > > >>>>>>>>> implementation (present in 4.6 > > and > > > > >> later) > > > > >> > > be > > > > >> > > > > > > changed? > > > > >> > > > > > > > > > > > > > >>>>>>>>> If it > > > > >> > > > > > > > > > > > is > > > > >> > > > > > > > > > > > > > >>>>>>>>> changed, how do we support > > > customers > > > > >> who > > > > >> > > are > > > > >> > > > > > > already > > > > >> > > > > > > > > > > > > > >>>>>>>>> using > > > > >> > > > > > > > > > > > the > > > > >> > > > > > > > > > > > > > >>>>>>>>> original volume-snapshot API > to > > > take > > > > >> > > > snapshots > > > > >> > > > > > on a > > > > >> > > > > > > > > > > > > > >>>>>>>>> backend > > > > >> > > > > > > > > > > > > SAN? > > > > >> > > > > > > > > > > > > > >>>>>>>>> > > > > >> > > > > > > > > > > > > > >>>>>>>>> Thanks, > > > > >> > > > > > > > > > > > > > >>>>>>>>> Mike > > > > >> > > > > > > > > > > > > > >>>>>>>>> > > > > >> > > > > > > > > > > > > > >>>>>>>>> On Thu, Feb 4, 2016 at 11:27 > AM, > > > > Will > > > > >> > > > Stevens < > > > > >> > > > > > > > > > > > > > >>>>>>>>> wstev...@cloudops.com > > > > <javascript:;> > > > > >> > > > > <javascript:;>> wrote: > > > > >> > > > > > > > > > > > > > >>>>>>>>> > > > > >> > > > > > > > > > > > > > >>>>>>>>>> Will you be able to create a > > > > Template > > > > >> > > from a > > > > >> > > > > > > > > > > > StorageSnapshot? > > > > >> > > > > > > > > > > > > > >>>>>>>>>> If yes, will the template be > > > stored > > > > >> in > > > > >> > the > > > > >> > > > > > > secondary > > > > >> > > > > > > > > > > > > > >>>>>>>>>> storage like normal templates > > or > > > > will > > > > >> > that > > > > >> > > > be > > > > >> > > > > > > > handled > > > > >> > > > > > > > > > > > > > >>>>>>>>>> somehow on the > > > > >> > > > > > > > > > > > > > vendor side? > > > > >> > > > > > > > > > > > > > >>>>>>>>>> > > > > >> > > > > > > > > > > > > > >>>>>>>>>> *Will STEVENS* > > > > >> > > > > > > > > > > > > > >>>>>>>>>> Lead Developer > > > > >> > > > > > > > > > > > > > >>>>>>>>>> > > > > >> > > > > > > > > > > > > > >>>>>>>>>> *CloudOps* *| *Cloud > Solutions > > > > >> Experts > > > > >> > > > > > > > > > > > > > >>>>>>>>>> 420 rue Guy *|* Montreal *|* > > > Quebec > > > > >> *|* > > > > >> > > H3J > > > > >> > > > > 1S6 > > > > >> > > > > > w > > > > >> > > > > > > > > > > > > > >>>>>>>>>> cloudops.com *|* tw > @CloudOps_ > > > > >> > > > > > > > > > > > > > >>>>>>>>>> > > > > >> > > > > > > > > > > > > > >>>>>>>>>> On Thu, Feb 4, 2016 at 1:22 > PM, > > > > Syed > > > > >> > > > Mushtaq < > > > > >> > > > > > > > > > > > > > >>>>>>>>>> syed1.mush...@gmail.com > > > > >> <javascript:;> > > > > >> > > > > <javascript:;>> wrote: > > > > >> > > > > > > > > > > > > > >>>>>>>>>> > > > > >> > > > > > > > > > > > > > >>>>>>>>>>> Thanks Will!!! > > > > >> > > > > > > > > > > > > > >>>>>>>>>>> > > > > >> > > > > > > > > > > > > > >>>>>>>>>>> On Thu, Feb 4, 2016 at 1:19 > > PM, > > > > Will > > > > >> > > > Stevens > > > > >> > > > > < > > > > >> > > > > > > > > > > > > > >>>>>>>>>>> wstev...@cloudops.com > > > > >> <javascript:;> > > > > >> > > > > <javascript:;>> wrote: > > > > >> > > > > > > > > > > > > > >>>>>>>>>>> > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>> I explicitly linked the > > Design > > > > >> Spec in > > > > >> > > the > > > > >> > > > > > Jira > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>> ticket because it was not > > clear > > > > in > > > > >> the > > > > >> > > > > > 'mention' > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>> section because it shows > as a > > > > page > > > > >> > 'you > > > > >> > > do > > > > >> > > > > not > > > > >> > > > > > > > have > > > > >> > > > > > > > > > > permission to'. > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>> > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>> *Will STEVENS* > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>> Lead Developer > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>> > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>> *CloudOps* *| *Cloud > > Solutions > > > > >> Experts > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>> 420 rue Guy *|* Montreal > *|* > > > > Quebec > > > > >> > *|* > > > > >> > > > H3J > > > > >> > > > > > 1S6 > > > > >> > > > > > > w > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>> cloudops.com *|* tw > > @CloudOps_ > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>> > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>> On Thu, Feb 4, 2016 at 1:02 > > PM, > > > > >> Syed > > > > >> > > Ahmed > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>> <sah...@cloudops.com > > > > >> <javascript:;> > > > > >> > > > > <javascript:;> > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>> > wrote: > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>> > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>>> > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>>> Design Spec: > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>>> > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>>> > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>>> > > > > >> > > > > > > > > > > > > > > > >> > > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Sto > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>>> rageSnapshot++API > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>>> > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>>> Jira Ticket > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>>> > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>>> > > > > >> > > > > > > > https://issues.apache.org/jira/browse/CLOUDSTACK-9 > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>>> 27 > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>>> 8 > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>>> > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>>> > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>>> Hi All, > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>>> > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>>> We plan to propose a new > set > > > of > > > > >> APIs > > > > >> > to > > > > >> > > > do > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>>> snapshots on managed > storage > > > > >> backends > > > > >> > > > like > > > > >> > > > > > > > > SolidFire. > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>>> Snapshots on current > managed > > > > >> storage > > > > >> > > stay > > > > >> > > > > on > > > > >> > > > > > > the > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>>> device which is contrary > to > > > what > > > > >> > > > CloudStack > > > > >> > > > > > > calls > > > > >> > > > > > > > > > > snpshots. > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>>> But taking snapshots on > > > storage > > > > >> and > > > > >> > > > keeping > > > > >> > > > > > it > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>>> there has its own > advantages > > > > >> > > > > > > > > > > > and > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>>> we would ideally like to > > have > > > > both > > > > >> > ways > > > > >> > > > of > > > > >> > > > > > > doing > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>>> snapshots. This proposal > > adds > > > 4 > > > > >> new > > > > >> > > APIs > > > > >> > > > to > > > > >> > > > > > > > create > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>>> snapshots on backend > > storage. > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>>> > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>>> What do you guys think of > > this > > > > >> > > feature? I > > > > >> > > > > > would > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>>> love to have some > feedback. > > I > > > am > > > > >> > > working > > > > >> > > > on > > > > >> > > > > > > > making > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>>> the design > > > > >> > > > > > > > > > > > spec > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>>> more concrete but wanted > to > > > > have a > > > > >> > high > > > > >> > > > > level > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>>> feedback first before > > starting > > > > to > > > > >> > work > > > > >> > > on > > > > >> > > > > it. > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>>> > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>>> Thanks, > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>>> -Syed > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>>> > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>>> > > > > >> > > > > > > > > > > > > > >>>>>>>>>>>> > > > > >> > > > > > > > > > > > > > >>>>>>>>>>> > > > > >> > > > > > > > > > > > > > >>>>>>>>>> > > > > >> > > > > > > > > > > > > > >>>>>>>>> > > > > >> > > > > > > > > > > > > > >>>>>>>>> > > > > >> > > > > > > > > > > > > > >>>>>>>>> -- > > > > >> > > > > > > > > > > > > > >>>>>>>>> *Mike Tutkowski* > > > > >> > > > > > > > > > > > > > >>>>>>>>> *Senior CloudStack Developer, > > > > >> SolidFire > > > > >> > > Inc.* > > > > >> > > > > > > > > > > > > > >>>>>>>>> e: > mike.tutkow...@solidfire.com > > > > >> > > > <javascript:;> > > > > >> > > > > <javascript:;> > > > > >> > > > > > > > > > > > > > >>>>>>>>> o: 303.746.7302 > > > > >> > > > > > > > > > > > > > >>>>>>>>> Advancing the way the world > uses > > > the > > > > >> > cloud > > > > >> > > > > > > > > > > > > > >>>>>>>>> < > > > > >> > > > > > > http://solidfire.com/solution/overview/?video=play > > > > >> > > > > > > > >*™ > > > > >> > > > > > > > > > > > > > >>>>>>>>> * > > > > >> > > > > > > > > > > > > > >>>>>>>>> > > > > >> > > > > > > > > > > > > > >>>>>>>> > > > > >> > > > > > > > > > > > > > >>>>>>>> > > > > >> > > > > > > > > > > > > > >>>>>>> > > > > >> > > > > > > > > > > > > > >>>>>> > > > > >> > > > > > > > > > > > > > >>>>>> > > > > >> > > > > > > > > > > > > > >>>>>> -- > > > > >> > > > > > > > > > > > > > >>>>>> *Mike Tutkowski* > > > > >> > > > > > > > > > > > > > >>>>>> *Senior CloudStack Developer, > > > SolidFire > > > > >> > Inc.* > > > > >> > > > > > > > > > > > > > >>>>>> e: mike.tutkow...@solidfire.com > > > > >> > > <javascript:;> > > > > >> > > > > <javascript:;> > > > > >> > > > > > > > > > > > > > >>>>>> o: 303.746.7302 > > > > >> > > > > > > > > > > > > > >>>>>> Advancing the way the world uses > > the > > > > >> cloud > > > > >> > > > > > > > > > > > > > >>>>>> < > > > > >> > > > > http://solidfire.com/solution/overview/?video=play > > > > >> > > > > > > >*™* > > > > >> > > > > > > > > > > > > > >>>>>> > > > > >> > > > > > > > > > > > > > >>>>> > > > > >> > > > > > > > > > > > > > >>>>> > > > > >> > > > > > > > > > > > > > >>>>> > > > > >> > > > > > > > > > > > > > >>>>> -- > > > > >> > > > > > > > > > > > > > >>>>> *Mike Tutkowski* > > > > >> > > > > > > > > > > > > > >>>>> *Senior CloudStack Developer, > > > SolidFire > > > > >> Inc.* > > > > >> > > > > > > > > > > > > > >>>>> e: mike.tutkow...@solidfire.com > > > > >> > <javascript:;> > > > > >> > > > > <javascript:;> > > > > >> > > > > > > > > > > > > > >>>>> o: 303.746.7302 > > > > >> > > > > > > > > > > > > > >>>>> Advancing the way the world uses > the > > > > cloud > > > > >> > > > > > > > > > > > > > >>>>> < > > > > >> > > > > http://solidfire.com/solution/overview/?video=play > > > > >> > > > > > > >*™* > > > > >> > > > > > > > > > > > > > >>>>> > > > > >> > > > > > > > > > > > > > >>>> > > > > >> > > > > > > > > > > > > > >>>> > > > > >> > > > > > > > > > > > > > >>> > > > > >> > > > > > > > > > > > > > >>> > > > > >> > > > > > > > > > > > > > >>> -- > > > > >> > > > > > > > > > > > > > >>> *Mike Tutkowski* > > > > >> > > > > > > > > > > > > > >>> *Senior CloudStack Developer, > > SolidFire > > > > >> Inc.* > > > > >> > > > > > > > > > > > > > >>> e: mike.tutkow...@solidfire.com > > > > >> <javascript:;> > > > > >> > > > > <javascript:;> > > > > >> > > > > > > > > > > > > > >>> o: 303.746.7302 > > > > >> > > > > > > > > > > > > > >>> Advancing the way the world uses the > > > cloud > > > > >> > > > > > > > > > > > > > >>> < > > > > >> > > > http://solidfire.com/solution/overview/?video=play > > > > >> > > > > > >*™* > > > > >> > > > > > > > > > > > > > >>> > > > > >> > > > > > > > > > > > > > >> > > > > >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > -- > > > > >> > > > > > > > > > > > > > > *Mike Tutkowski* > > > > >> > > > > > > > > > > > > > > *Senior CloudStack Developer, > SolidFire > > > > Inc.* > > > > >> > > > > > > > > > > > > > > e: mike.tutkow...@solidfire.com > > > > >> <javascript:;> > > > > >> > > > > <javascript:;> > > > > >> > > > > > > > > > > > > > > o: 303.746.7302 > > > > >> > > > > > > > > > > > > > > Advancing the way the world uses the > > cloud > > > > >> > > > > > > > > > > > > > > < > > > > >> > > http://solidfire.com/solution/overview/?video=play > > > > >> > > > > >*™* > > > > >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > Find out more about ShapeBlue and our > > range > > > of > > > > >> > > > CloudStack > > > > >> > > > > > > > > > > > > > related > > > > >> > > > > > > > > > > > > services: > > > > >> > > > > > > > > > > > > > IaaS Cloud Design & Build > > > > >> > > > > > > > > > > > > > < > > > > >> > http://shapeblue.com/iaas-cloud-design-and-build//> > > > > >> > > | > > > > >> > > > > > > > CSForge – > > > > >> > > > > > > > > > > > > > rapid IaaS deployment framework < > > > > >> > > > > > > http://shapeblue.com/csforge/ > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > > > CloudStack Consulting > > > > >> > > > > > > > > > > > > > < > > > http://shapeblue.com/cloudstack-consultancy/ > > > > > > > > > >> | > > > > >> > > > > > > > > > > > > CloudStack > > > > >> > > > > > > > > > > > > > Software Engineering > > > > >> > > > > > > > > > > > > > < > > > > >> > > http://shapeblue.com/cloudstack-software-engineering/ > > > > >> > > > > > > > > >> > > > > > > > > > > > > > CloudStack Infrastructure Support > > > > >> > > > > > > > > > > > > > < > > > > >> > > > http://shapeblue.com/cloudstack-infrastructure-support/ > > > > >> > > > > > > > > > >> > > > > > | > > > > >> > > > > > > > > > > > > > CloudStack Bootcamp Training Courses > > > > >> > > > > > > > > > > > > > < > > http://shapeblue.com/cloudstack-training/> > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > -- > > > > >> > > > > > > > > > > > *Mike Tutkowski* > > > > >> > > > > > > > > > > > *Senior CloudStack Developer, SolidFire > Inc.* > > > > >> > > > > > > > > > > > e: mike.tutkow...@solidfire.com > > <javascript:;> > > > > >> > > > > <javascript:;> > > > > >> > > > > > > > > > > > o: 303.746.7302 > > > > >> > > > > > > > > > > > Advancing the way the world uses the cloud > > > > >> > > > > > > > > > > > < > > > > http://solidfire.com/solution/overview/?video=play > > > > >> > >*™* > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > Find out more about ShapeBlue and our range of > > > > >> CloudStack > > > > >> > > > > related > > > > >> > > > > > > > > > services: > > > > >> > > > > > > > > > > IaaS Cloud Design & Build > > > > >> > > > > > > > > > > < > > > http://shapeblue.com/iaas-cloud-design-and-build// > > > > > > > > > >> | > > > > >> > > > > CSForge – > > > > >> > > > > > > > rapid > > > > >> > > > > > > > > > > IaaS deployment framework < > > > > >> http://shapeblue.com/csforge/ > > > > >> > > > > > > >> > > > > > > CloudStack > > > > >> > > > > > > > > > > Consulting < > > > > >> http://shapeblue.com/cloudstack-consultancy/ > > > > >> > > > > > > >> > > | > > > > >> > > > > > > > CloudStack > > > > >> > > > > > > > > > > Software Engineering > > > > >> > > > > > > > > > > < > > > > >> http://shapeblue.com/cloudstack-software-engineering/> > > > > >> > > > > > > > > > > CloudStack Infrastructure Support > > > > >> > > > > > > > > > > < > > > > >> http://shapeblue.com/cloudstack-infrastructure-support/ > > > > >> > > > > > > >> > > | > > > > >> > > > > > > > CloudStack > > > > >> > > > > > > > > > > Bootcamp Training Courses < > > > > >> > > > > > > http://shapeblue.com/cloudstack-training/ > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > Find out more about ShapeBlue and our range of > > > > >> CloudStack > > > > >> > > > related > > > > >> > > > > > > > > services: > > > > >> > > > > > > > > > IaaS Cloud Design & Build > > > > >> > > > > > > > > > < > > http://shapeblue.com/iaas-cloud-design-and-build// > > > > > > > > | > > > > >> > > > CSForge – > > > > >> > > > > > > rapid > > > > >> > > > > > > > > > IaaS deployment framework < > > > > >> http://shapeblue.com/csforge/> > > > > >> > > > > > > > > > CloudStack Consulting < > > > > >> > > > > > http://shapeblue.com/cloudstack-consultancy/> > > > > >> > > > > > > | > > > > >> > > > > > > > > CloudStack > > > > >> > > > > > > > > > Software Engineering > > > > >> > > > > > > > > > < > > > > http://shapeblue.com/cloudstack-software-engineering/> > > > > >> > > > > > > > > > CloudStack Infrastructure Support > > > > >> > > > > > > > > > < > > > > >> http://shapeblue.com/cloudstack-infrastructure-support/> > > > > >> > | > > > > >> > > > > > > CloudStack > > > > >> > > > > > > > > > Bootcamp Training Courses < > > > > >> > > > > > http://shapeblue.com/cloudstack-training/ > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > -- > > > > >> > > > > > > *Mike Tutkowski* > > > > >> > > > > > > *Senior CloudStack Developer, SolidFire Inc.* > > > > >> > > > > > > e: mike.tutkow...@solidfire.com <javascript:;> > > > > <javascript:;> > > > > >> > > > > > > o: 303.746.7302 > > > > >> > > > > > > Advancing the way the world uses the cloud > > > > >> > > > > > > <http://solidfire.com/solution/overview/?video=play > >*™* > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > -- > > > > >> > > > > *Mike Tutkowski* > > > > >> > > > > *Senior CloudStack Developer, SolidFire Inc.* > > > > >> > > > > e: mike.tutkow...@solidfire.com <javascript:;> > > > > >> > > > > o: 303.746.7302 > > > > >> > > > > Advancing the way the world uses the cloud > > > > >> > > > > <http://solidfire.com/solution/overview/?video=play>*™* > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > -- > > > > >> > > > Ian Rae > > > > >> > > > CEO | PDG > > > > >> > > > c: 514.944.4008 > > > > >> > > > > > > > >> > > > CloudOps | Cloud Infrastructure and Networking Solutions > > > > >> > > > www.cloudops.com | 420 rue Guy | Montreal | Canada | H3J > 1S6 > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > > >> > > > > > >> > -- > > > > >> > *Mike Tutkowski* > > > > >> > *Senior CloudStack Developer, SolidFire Inc.* > > > > >> > e: mike.tutkow...@solidfire.com > > > > >> > o: 303.746.7302 > > > > >> > Advancing the way the world uses the cloud > > > > >> > <http://solidfire.com/solution/overview/?video=play>*™* > > > > >> > > > > > >> > > > > > > > > > > > > > > > > > > > > -- > > > > > *Mike Tutkowski* > > > > > *Senior CloudStack Developer, SolidFire Inc.* > > > > > e: mike.tutkow...@solidfire.com > > > > > o: 303.746.7302 > > > > > Advancing the way the world uses the cloud > > > > > <http://solidfire.com/solution/overview/?video=play>*™* > > > > > > > > > > > > > > > > > > > > > -- > > > > *Mike Tutkowski* > > > > *Senior CloudStack Developer, SolidFire Inc.* > > > > e: mike.tutkow...@solidfire.com > > > > o: 303.746.7302 > > > > Advancing the way the world uses the cloud > > > > <http://solidfire.com/solution/overview/?video=play>*™* > > > > > > > > > > > > > > > -- > > *Mike Tutkowski* > > *Senior CloudStack Developer, SolidFire Inc.* > > e: mike.tutkow...@solidfire.com > > o: 303.746.7302 > > Advancing the way the world uses the cloud > > <http://solidfire.com/solution/overview/?video=play>*™* > > > -- *Mike Tutkowski* *Senior CloudStack Developer, SolidFire Inc.* e: mike.tutkow...@solidfire.com o: 303.746.7302 Advancing the way the world uses the cloud <http://solidfire.com/solution/overview/?video=play>*™*