It's not ideal - true, but it does allow us to be backward compatible.
If you have other ideas, though, about how to maintain backward
compatibility, I'm definitely open to hear them.
Thanks!
On Mon, Feb 8, 2016 at 11:42 AM, Syed Mushtaq <syed1.mush...@gmail.com>
wrote:
> Hi Mike,
>
> Adding a flag to createSnapshot was the first and the most obvious thing
> that came to our minds. The problem that I had with this was that:
>
> 1) I feel it is exposing something to the end user that is internal to the
> cloud.
>
> 2) We have to follow two different ways of restore/deletion in the same
> code path depending on where the Snapshot resides which I find kind of a
> bad design.
>
> But if exposing a archive flag to the end user is acceptable then we can
> definitely use this instead of adding the StorageSnapshot API
>
> Thanks,
> -Syed
>
>
> On Mon, Feb 8, 2016 at 1:26 PM, Mike Tutkowski <
> mike.tutkow...@solidfire.com
> > wrote:
>
> > Hi Pierre-Luc,
> >
> > My recommendation would be this:
> >
> > Add an "archive" flag to the current volume-snapshot API. Its default
> would
> > be "false" because that would be backward compatible with how 4.6 has
> > volume snapshots implemented (i.e. they stay on the SAN in 4.6, 4.7, and
> > 4.8).
> >
> > If you set archive=true, then we would perform a background migration of
> > the snapshot from the SAN to the secondary storage (then delete the SAN
> > snapshot).
> >
> > That archive parameter would only be valid for managed storage.
> >
> > Sound reasonable?
> >
> > Also, a VM snapshot that includes disks provided by managed storage
> should
> > work.
> >
> > Talk to you later,
> > Mike
> >
> > On Mon, Feb 8, 2016 at 9:22 AM, Pierre-Luc Dion <pd...@cloudops.com>
> > wrote:
> >
> > > Mike,
> > >
> > > In terms of API's, would you prefer introducing a parameter to the
> > existing
> > > VolumeSnapshot, example: extract={true|false} with a default value
> of
> > > true which would extract snapshot into the secondary storage, which is
> > the
> > > current default behavior. Then for SAN snapshot that remain on the SAN
> we
> > > would just set "extract=false" ? as oppose to create a new
> > > StorageSnapshot API ?
> > >
> > >
> > > Paul,
> > >
> > > From what I'm seeing so far, we can't do a VM-snapshot when using
> managed
> > > storage for VM having more than one Volume. For the reason that
> snapshot
> > > are performed outside of the hypervisor awareness and asynchronously.
> If
> > > someone have a way to address that, it would make thinks much more
> > > attractive.
> > >
> > >
> > >
> > >
> > > On Mon, Feb 8, 2016 at 10:57 AM, Ian Rae <i...@cloudops.com> wrote:
> > >
> > > > I think a service provider backup scenario is more likely to take
> > > advantage
> > > > of SAN snapshot. There are a few reasons for this. Traditional
> backups
> > > > involve access to the file system, and there is an expectation that
> > this
> > > > can be done with reasonably short time frames without negatively
> > > impacting
> > > > VM performance, and that the backup orchestrator can apply various
> > logic
> > > > and or transformations to the data (compress, encrypt, deltas
> etc...).
> > > > While it is true that one could apply a backup process to a cloud
> > > snapshot,
> > > > this would be slow and inefficient requiring the data to be moved
> > several
> > > > times and there are some major bottlenecks with cloud snapshots.
> With a
> > > > cloud snapshot - there seems to be no reasonable expectation of being
> > > able
> > > > to do differential snapshots (I think this depends on the hypervisor)
> > and
> > > > if you do differential snapshots this will make file backups from
> those
> > > > snapshots even more complicated to orchestrate.
> > > >
> > > > Suspect there needs to be a different thread on how to better enable
> > > > backups, as a service. As per Paul's suggestion, but it is a related
> > > > workflow so relevant to this discussion.
> > > >
> > > > Ian
> > > >
> > > > On Monday, February 8, 2016, Mike Tutkowski <
> > > mike.tutkow...@solidfire.com>
> > > > wrote:
> > > >
> > > > > To me it sounds like number two and number three are different uses
> > for
> > > > the
> > > > > same "thing"(which is totally fine).
> > > > >
> > > > > As for taking a fast SAN snapshot and exporting it asynchronously,
> do
> > > we
> > > > > see the SSVM as performing the export?
> > > > >
> > > > > To be backwards compatible with what we have in 4.6 and later for
> > > volume
> > > > > snapshots for managed storage, I think it might be easier if we
> pass
> > > in a
> > > > > flag that says whether or not to archive the SAN snapshot (which, I
> > > > think,
> > > > > is something that you suggested, as well, Pierre-Luc).
> > > > >
> > > > > On Monday, February 8, 2016, Pierre-Luc Dion <pd...@cloudops.com
> > > > > <javascript:;>> wrote:
> > > > >
> > > > > > Hi Mike,
> > > > > >
> > > > > > The reason behind the creation of a SAN snapshot which is
> exported
> > > into
> > > > > > secondary storage, is because creating a copy of the .VHD
> directly
> > > > would
> > > > > > impact uptime of the VM as creating that copy take lots of time.
> > Has
> > > > > oppose
> > > > > > to a SAN snapshot that is close to instantaneous which can
> > afterward
> > > be
> > > > > > clone into Secondary Storage asynchronously.
> > > > > >
> > > > > > I would suspect an extracted VolumeSnapshot taken from a SAN
> > snapshot
> > > > > could
> > > > > > have is SAN snapshot deleted to avoid duplica and space
> consumption
> > > on
> > > > > the
> > > > > > Primary Storage side.
> > > > > >
> > > > > >
> > > > > > I see 3 definitions in our current discussion regarding the term
> > > > snapshot
> > > > > > (these are not official terminology but by own interpretation of
> > > them):
> > > > > >
> > > > > > 1. *Snapshot* (AKA: Storage Snapshot / Mike's definition of a
> > > > snapshot):
> > > > > > it's a volume snapshot at the storage level, point in time of
> your
> > > > data.
> > > > > it
> > > > > > reside on the primary storage. Useful and efficient for software
> > side
> > > > > > incident.
> > > > > > 2. *Cloud Snapshot *( AKA: CloudStack VolumeSnapshot/ cloud
> backup
> > > > aws-S3
> > > > > > style ): Point in time copy of the Virtual Disk that reside on a
> > > > > different
> > > > > > storage array then the original Volume. Facilitate data migration
> > > > between
> > > > > > clusters and, in case of primary storage incident, Volume
> snapshots
> > > are
> > > > > not
> > > > > > impacted and can be reuse.
> > > > > > 3. *Backup*: Archival of your Virtual-machines data that also
> > > validate
> > > > > data
> > > > > > integrity, provide a storage efficient archiving method and an
> > > > > independent
> > > > > > way to restore your data in case of an major infrastructure
> > disaster.
> > > > > >
> > > > > >
> > > > > > Regards,
> > > > > >
> > > > > > PL
> > > > > >
> > > > > >
> > > > > > On Fri, Feb 5, 2016 at 1:34 PM, Mike Tutkowski <
> > > > > > mike.tutkow...@solidfire.com <javascript:;> <javascript:;>
> > > > > > > wrote:
> > > > > >
> > > > > > > So, let's see if I currently follow the requirements:
> > > > > > >
> > > > > > > * Augment volume snapshots for managed storage to conditionally
> > > > export
> > > > > > data
> > > > > > > to NFS. The current process of taking a snapshot on the SAN is
> > > fine,
> > > > > but
> > > > > > > we'd like the option to export the data to NFS, as well.
> > > > > > >
> > > > > > > Questions:
> > > > > > >
> > > > > > > Once the data has been exported to NFS, do we keep the SAN
> > snapshot
> > > > or
> > > > > > > delete it?
> > > > > > >
> > > > > > > If we are deleting the SAN snapshot, then why don't we just
> copy
> > > the
> > > > > VHD
> > > > > > > from primary to secondary the way we do today for non-managed
> > (i.e.
> > > > > > > traditional) storage? Why create a SAN snapshot in that
> scenario?
> > > > > Perhaps
> > > > > > > to have the SSVM mount and perform the VHD copy to secondary
> > > storage
> > > > > > > instead of a XenServer host?
> > > > > > >
> > > > > > > Thanks for the clarification.
> > > > > > >
> > > > > > > By the way, to me a backup is when you copy data from one
> storage
> > > > > system
> > > > > > to
> > > > > > > another (regardless of features, if any, to restore the data in
> > the
> > > > > > > future). A snapshot is a point-in-time view of the data of a
> > volume
> > > > and
> > > > > > > it's stored on the same storage system as the volume.
> > > > > > >
> > > > > > > On Fri, Feb 5, 2016 at 11:09 AM, Pierre-Luc Dion <
> > > pd...@cloudops.com
> > > > > <javascript:;>
> > > > > > <javascript:;>>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > That's fun to see that discussion happening. I 100% agree
> with
> > > > Paul's
> > > > > > > > points of view. VolumeSnapshot are not a backup, but I do
> > > consider
> > > > > them
> > > > > > > as
> > > > > > > > a safety vest against Primary Storage failure, because
> failure
> > > > append
> > > > > > > :-( .
> > > > > > > >
> > > > > > > > The current proposal around snapshots that reside on the
> > primary
> > > > > > storage
> > > > > > > or
> > > > > > > > snapshots that end in the Secondary Storage is not to
> address
> > > any
> > > > > kind
> > > > > > > of
> > > > > > > > backups requirement because a snapshot is not a backup, event
> > an
> > > > > > > extracted
> > > > > > > > VM snapshot.
> > > > > > > >
> > > > > > > > The main idea, and again this is for managed storage;
> > > > > > > >
> > > > > > > > 1. StorageSnapshotAPI: Provide storage side snapshot
> capability
> > > for
> > > > > > fast
> > > > > > > > response time that support rollback to previous timestamp,
> > create
> > > > new
> > > > > > > > volume and maybe create template.
> > > > > > > > not required to be a new API if the work is already
> done, I
> > > > think
> > > > > > > this
> > > > > > > > is a different behaviors than the user expectation of a
> > > > > > volume-snapshot.
> > > > > > > > 2. VolumeSnapshotAPI: Provide current cloudstack behavior
> that
> > > > create
> > > > > > an
> > > > > > > > extraction of a volume into SecondaryStorage which can be
> reuse
> > > to
> > > > > > > create a
> > > > > > > > new volume into another Primary Storage. This type of
> snapshot
> > > is a
> > > > > > slow
> > > > > > > > job since yes it would have to copy the full volume size on
> the
> > > > > > Secondary
> > > > > > > > storage.
> > > > > > > >
> > > > > > > >
> > > > > > > > PL
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On Fri, Feb 5, 2016 at 12:45 PM, Syed Mushtaq <
> > > > > syed1.mush...@gmail.com <javascript:;>
> > > > > > <javascript:;>>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > I think I share you view on the 'Ideal world'. Backup (via
> > > Volume
> > > > > > > > > Snapshots) is a huge bottleneck in Cloudstack. This is
> > > amplified
> > > > > > > > especially
> > > > > > > > > when you have a object storage as your secondary storage
> > > because
> > > > it
> > > > > > > > > requires two copies (one to an NFS staging area and from
> > there
> > > to
> > > > > > > object
> > > > > > > > > storage). And not to mention that all these copies are
> > > consuming
> > > > > > > > hypervisor
> > > > > > > > > resources. Xenserver's Dom0 is also a huge bottleneck as
> all
> > > the
> > > > > > > Network
> > > > > > > > > and I/O flow through it. So our intention of proposing the
> > > > "Storage
> > > > > > > > > Snapshots" is to give a better way of achiving snapshots
> > while
> > > > > still
> > > > > > > > > keeping the original definition of volume snpashots (ie
> > upload
> > > to
> > > > > sec
> > > > > > > > > storage).
> > > > > > > > >
> > > > > > > > > But as Erik pointed out volume snapshots are not backups.
> > They
> > > > > don't
> > > > > > > work
> > > > > > > > > form multi-disk LVM volume groups and dynamic disks. I am
> all
> > > in
> > > > > for
> > > > > > a
> > > > > > > > > better backup solution which handles these use cases and
> > > utilizes
> > > > > the
> > > > > > > > > storage's advanced features.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Fri, Feb 5, 2016 at 12:29 PM, Paul Angus <
> > > > > > paul.an...@shapeblue.com <javascript:;> <javascript:;>>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > In the beginning... there were CloudStack snapshots and
> > they
> > > > were
> > > > > > > > > actually
> > > > > > > > > > volume snapshots not hypervisor point-in-time snapshots.
> > > > > > > > > > Then VM snapshots were created (which are point-in-time
> > > > > hypervisor
> > > > > > > > > > snapshots) and we started referring to the original
> > snapshots
> > > > as
> > > > > > > volume
> > > > > > > > > > snapshots.
> > > > > > > > > >
> > > > > > > > > > CloudStack does not offer 'backups', but many people use
> > > volume
> > > > > > > > snapshots
> > > > > > > > > > as backups. However you can't in-place restore volume
> > > snapshots
> > > > > and
> > > > > > > if
> > > > > > > > > you
> > > > > > > > > > have a VM with multiple volumes, the volume snapshots
> must
> > be
> > > > > done
> > > > > > in
> > > > > > > > > > series, meaning that the state across all of the volumes
> is
> > > > > > unlikely
> > > > > > > to
> > > > > > > > > be
> > > > > > > > > > consistent.
> > > > > > > > > >
> > > > > > > > > > 'Actual Backups' would enable all of the restore options
> > > which
> > > > > > users
> > > > > > > > > might
> > > > > > > > > > expect as well options as to where they might be stored.
> In
> > > my
> > > > > > ideal
> > > > > > > > > world
> > > > > > > > > > they would also be able to leverage back-end hardware
> (such
> > > as
> > > > > > > > Solidfire,
> > > > > > > > > > NetApp etc :) ) and software such as Veeam, Commvault etc
> > to
> > > > > > > accelerate
> > > > > > > > > the
> > > > > > > > > > process.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > [image: ShapeBlue] <http://www.shapeblue.com>
> > > > > > > > > > Paul Angus
> > > > > > > > > > VP Technology , ShapeBlue
> > > > > > > > > > d: *+44 203 617 0528 | s: +44 203 603 0540*
> > > > > > > > > >
> > > <+44%20203%20617%200528%20%7C%20s:%20+44%20203%20603%200540> |
> > > > > m:
> > > > > > > > > > *+44 7711 418784* <+44%207711%20418784>
> > > > > > > > > > e: *paul.an...@shapeblue.com <javascript:;>
> > <javascript:;>
> > > |
> > > > > t: @cloudyangus*
> > > > > > > > > > <paul.an...@shapeblue.com <javascript:;>
> > > > > <javascript:;>%20%7C%20t:%20@cloudyangus>
> > > > > > | w:
> > > > > > > > > > *www.shapeblue.com* <http://www.shapeblue.com>
> > > > > > > > > > a: 53 Chandos Place, Covent Garden London WC2N 4HS UK
> > > > > > > > > > Shape Blue Ltd is a company incorporated in England &
> > Wales.
> > > > > > > ShapeBlue
> > > > > > > > > > Services India LLP is a company incorporated in India and
> > is
> > > > > > operated
> > > > > > > > > under
> > > > > > > > > > license from Shape Blue Ltd. Shape Blue Brasil
> Consultoria
> > > Ltda
> > > > > is
> > > > > > a
> > > > > > > > > > company incorporated in Brasil and is operated under
> > license
> > > > from
> > > > > > > Shape
> > > > > > > > > > Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by
> > The
> > > > > > > Republic
> > > > > > > > of
> > > > > > > > > > South Africa and is traded under license from Shape Blue
> > Ltd.
> > > > > > > ShapeBlue
> > > > > > > > > is
> > > > > > > > > > a registered trademark.
> > > > > > > > > > This email and any attachments to it may be confidential
> > and
> > > > are
> > > > > > > > intended
> > > > > > > > > > solely for the use of the individual to whom it is
> > addressed.
> > > > Any
> > > > > > > views
> > > > > > > > > or
> > > > > > > > > > opinions expressed are solely those of the author and do
> > not
> > > > > > > > necessarily
> > > > > > > > > > represent those of Shape Blue Ltd or related companies.
> If
> > > you
> > > > > are
> > > > > > > not
> > > > > > > > > the
> > > > > > > > > > intended recipient of this email, you must neither take
> any
> > > > > action
> > > > > > > > based
> > > > > > > > > > upon its contents, nor copy or show it to anyone. Please
> > > > contact
> > > > > > the
> > > > > > > > > sender
> > > > > > > > > > if you believe you have received this email in error.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > -----Original Message-----
> > > > > > > > > > From: Syed Mushtaq [mailto:syed1.mush...@gmail.com
> > > > > <javascript:;> <javascript:;>]
> > > > > > > > > > Sent: Friday, February 5, 2016 4:58 PM
> > > > > > > > > > To: dev@cloudstack.apache.org <javascript:;>
> > <javascript:;>
> > > > > > > > > > Subject: Re: [Propose][New Feature] Storage Snapshots
> > > > > > > > > >
> > > > > > > > > > Paul,
> > > > > > > > > >
> > > > > > > > > > When you say actual backups, how would it be different
> from
> > > the
> > > > > > > Volume
> > > > > > > > > > Snapshots that exist currently. My understanding is that
> > > > Backups
> > > > > > end
> > > > > > > up
> > > > > > > > > in
> > > > > > > > > > Sec Storage whereas Snapshots are just a point-in-time
> > state
> > > of
> > > > > > your
> > > > > > > > > volume
> > > > > > > > > > which can be restored back correct?
> > > > > > > > > >
> > > > > > > > > > -Syed
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Fri, Feb 5, 2016 at 11:23 AM, Paul Angus <
> > > > > > > paul.an...@shapeblue.com <javascript:;> <javascript:;>>
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Hi Syed,
> > > > > > > > > > >
> > > > > > > > > > > As I understand it, the SolidFire plugin will export
> the
> > > > > snapshot
> > > > > > > to
> > > > > > > > > > > secondary storage if the user requests a template from
> > the
> > > > > > snapshot
> > > > > > > > or
> > > > > > > > > > > wants to download the snapshot from the cloud. This is
> a
> > > > good,
> > > > > > > > > > > pragmatic approach and yes Mike the SolidFire storage
> is
> > > > super
> > > > > > > > > > > reliable and snapshots on SolidFire arrays take up next
> > to
> > > no
> > > > > > > space.
> > > > > > > > > > > BUT I think that we are talking about a more general
> > > purpose
> > > > > API,
> > > > > > > and
> > > > > > > > > > > other storage systems may not be as awesome as Mike's.
> > > That's
> > > > > my
> > > > > > > > > > > concern. Also, the time to transfer for say 1TB to move
> > > from
> > > > > > > primary
> > > > > > > > > > > to sec storage and then create a VM template out of it
> > may
> > > be
> > > > > too
> > > > > > > > long
> > > > > > > > > > for users.
> > > > > > > > > > >
> > > > > > > > > > > @Mike I don’t think 'we' use the term volume snapshot
> for
> > > > > backup,
> > > > > > > > it's
> > > > > > > > > > > just that users want to do backups and a volume
> snapshot
> > is
> > > > the
> > > > > > > only
> > > > > > > > > > > type of snapshot that copies the disk elsewhere and can
> > be
> > > > > > > scheduled.
> > > > > > > > > > >
> > > > > > > > > > > I'm 'pondering' the implications of enabling actual
> > backups
> > > > > > > (through
> > > > > > > > > > > recognised backup providers) and the user requirements
> > > around
> > > > > > them
> > > > > > > > > > > (particularly restoration use cases) as a separate
> thread
> > > of
> > > > > > work.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > [image: ShapeBlue] <http://www.shapeblue.com> Paul
> Angus
> > > VP
> > > > > > > > Technology
> > > > > > > > > > > , ShapeBlue
> > > > > > > > > > > d: *+44 203 617 0528 | s: +44 203 603 0540*
> > > > > > > > > > >
> > > <+44%20203%20617%200528%20%7C%20s:%20+44%20203%20603%200540>
> > > > |
> > > > > m:
> > > > > > > > > > > *+44 7711 418784* <+44%207711%20418784>
> > > > > > > > > > > e: *paul.an...@shapeblue.com <javascript:;>
> > > <javascript:;> |
> > > > > t: @cloudyangus*
> > > > > > > > > > > <paul.an...@shapeblue.com <javascript:;>
> <javascript:;>
> > > > > > %20%7C%20t:%20@cloudyangus> | w:
> > > > > > > > > > > *www.shapeblue.com* <http://www.shapeblue.com>
> > > > > > > > > > > a: 53 Chandos Place, Covent Garden London WC2N 4HS UK
> > Shape
> > > > > Blue
> > > > > > > Ltd
> > > > > > > > > > > is a company incorporated in England & Wales. ShapeBlue
> > > > > Services
> > > > > > > > India
> > > > > > > > > > > LLP is a company incorporated in India and is operated
> > > under
> > > > > > > license
> > > > > > > > > > > from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda
> > is
> > > a
> > > > > > > company
> > > > > > > > > > > incorporated in Brasil and is operated under license
> from
> > > > Shape
> > > > > > > Blue
> > > > > > > > > > > Ltd. ShapeBlue SA Pty Ltd is a company registered by
> The
> > > > > Republic
> > > > > > > of
> > > > > > > > > > > South Africa and is traded under license from Shape
> Blue
> > > Ltd.
> > > > > > > > > > > ShapeBlue is a registered trademark.
> > > > > > > > > > > This email and any attachments to it may be
> confidential
> > > and
> > > > > are
> > > > > > > > > > > intended solely for the use of the individual to whom
> it
> > is
> > > > > > > > addressed.
> > > > > > > > > > > Any views or opinions expressed are solely those of the
> > > > author
> > > > > > and
> > > > > > > do
> > > > > > > > > > > not necessarily represent those of Shape Blue Ltd or
> > > related
> > > > > > > > > > > companies. If you are not the intended recipient of
> this
> > > > email,
> > > > > > you
> > > > > > > > > > > must neither take any action based upon its contents,
> nor
> > > > copy
> > > > > or
> > > > > > > > show
> > > > > > > > > > > it to anyone. Please contact the sender if you believe
> > you
> > > > have
> > > > > > > > > received
> > > > > > > > > > this email in error.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > -----Original Message-----
> > > > > > > > > > > From: Syed Mushtaq [mailto:syed1.mush...@gmail.com
> > > > > <javascript:;>
> > > > > > <javascript:;>]
> > > > > > > > > > > Sent: 05 February 2016 15:31
> > > > > > > > > > > To: dev@cloudstack.apache.org <javascript:;>
> > > <javascript:;>
> > > > > > > > > > > Subject: Re: [Propose][New Feature] Storage Snapshots
> > > > > > > > > > >
> > > > > > > > > > > I think the terminology confusion comes from AWS where
> > they
> > > > do
> > > > > > EBS
> > > > > > > > > > > snapshots backed up to S3 and CloudStack sort of
> followed
> > > > that.
> > > > > > And
> > > > > > > > as
> > > > > > > > > > > an end user who is oblivious to the internals of my
> > > provider,
> > > > > my
> > > > > > > > > > > expectation would be something similar to what AWS as
> > that
> > > is
> > > > > my
> > > > > > > > > > > biggest reference point.
> > > > > > > > > > >
> > > > > > > > > > > To your point Mike, I agree that a Primary Storage
> > failure
> > > on
> > > > > > > > > > > SolidFire is unlikely, there are other motivations for
> us
> > > to
> > > > > push
> > > > > > > > data
> > > > > > > > > > > to secondary storage. Primary storage (atleast for us)
> > > costs
> > > > > > > around 3
> > > > > > > > > > > times as much as secondary storage and snapshots on
> > primary
> > > > > > storage
> > > > > > > > > > > are rarely used (especially for some of our customers
> who
> > > do
> > > > > > daily
> > > > > > > > > > backups).
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > On Fri, Feb 5, 2016 at 10:18 AM, Mike Tutkowski <
> > > > > > > > > > > mike.tutkow...@solidfire.com <javascript:;>
> > > <javascript:;>>
> > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Some of the weirdness is around terminology.
> > > > > > > > > > > >
> > > > > > > > > > > > For most systems I've worked on, a snapshot and a
> > backup
> > > > are
> > > > > > two
> > > > > > > > > > > > completely different things (but CloudStack has
> > > > traditionally
> > > > > > > used
> > > > > > > > > > > > the term "volume snapshot" to mean backup).
> > > > > > > > > > > >
> > > > > > > > > > > > I will put in a SolidFire "plug" here and say,
> though,
> > > that
> > > > > if
> > > > > > > your
> > > > > > > > > > > > primary storage is running on SolidFire that it is
> > > unlikely
> > > > > > > you'll
> > > > > > > > > > > > encounter an issue where your primary storage goes
> > > offline
> > > > > (and
> > > > > > > > > > > > you'll even maintain your performance guarantees
> during
> > > > > failure
> > > > > > > > > > > > scenarios and upgrades, as well). That being the
> case,
> > it
> > > > is
> > > > > > less
> > > > > > > > > > > > useful to require a backup to Swift (but it's
> perfectly
> > > OK
> > > > if
> > > > > > > > that's
> > > > > > > > > > > > what we want to do
> > > > > > > > > > > here).
> > > > > > > > > > > >
> > > > > > > > > > > > On Fri, Feb 5, 2016 at 8:07 AM, Syed Mushtaq
> > > > > > > > > > > > <syed1.mush...@gmail.com <javascript:;>
> > <javascript:;>>
> > > > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > Paul,
> > > > > > > > > > > > >
> > > > > > > > > > > > > I believe with the current implementation of
> > Snapshots
> > > on
> > > > > > > managed
> > > > > > > > > > > > > storage
> > > > > > > > > > > > > (SolidFire) the snapshots are never exported to the
> > > > > secondary
> > > > > > > > > > storage.
> > > > > > > > > > > > > While this solves the problem of having snapshots
> > > taking
> > > > > > > forever
> > > > > > > > > > > > > to get to sec storage, this leaves us with a
> > > > > > > > > > > > huge
> > > > > > > > > > > > > liability if our primary storage goes down. We see
> > > > > snapshots
> > > > > > as
> > > > > > > > > > > > > our recovery path because we store them in Swift
> > which
> > > is
> > > > > > > > reliable
> > > > > > > > > > > > > and resilient to failures.
> > > > > > > > > > > > >
> > > > > > > > > > > > > With Storage snpashots our goal is to have Volume
> > > > snapshots
> > > > > > > > always
> > > > > > > > > > > > > backed up to secondary storage and Storage
> Snapshots
> > > stay
> > > > > on
> > > > > > > the
> > > > > > > > > > > > > primary
> > > > > > > > > > > > storage.
> > > > > > > > > > > > > A provider could potentially mix both these and
> solve
> > > the
> > > > > > > problem
> > > > > > > > > > > > > that you mentioned where you want to meet user's
> > > > > expectation
> > > > > > > of a
> > > > > > > > > > > > > snapshot (ie backup to sec storage) while having an
> > > > ability
> > > > > > to
> > > > > > > > > > > > > utilize faster sanpshots (i.e. on the device)
> > > > > > > > > > > > >
> > > > > > > > > > > > > Hope this clarifies things.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Thanks,
> > > > > > > > > > > > > -Syed
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Fri, Feb 5, 2016 at 9:04 AM, Paul Angus
> > > > > > > > > > > > > <paul.an...@shapeblue.com <javascript:;>
> > > <javascript:;>>
> > > > > > > > > > > > > wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > > HI guys,
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Could someone point me to the Jira bug of FS for
> > the
> > > > > > > > > > > > > > SAN-snapshot
> > > > > > > > > > > > feature
> > > > > > > > > > > > > > in 4.6 which is mentioned.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > From my discussions with users and operators
> around
> > > > > > snapshots
> > > > > > > > > > > > > > I'd make
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > following observations:
> > > > > > > > > > > > > > a. 'users' use snapshots as backups (both
> long-term
> > > and
> > > > > > short
> > > > > > > > > > > > > > term)
> > > > > > > > > > > > with
> > > > > > > > > > > > > > the expectation that they can use them for
> recovery
> > > if
> > > > > > > > required.
> > > > > > > > > > > > > > b. operators fall back to snapshots if something
> > has
> > > > gone
> > > > > > > wrong
> > > > > > > > > > > > > > with primary storage.
> > > > > > > > > > > > > > c. users sometimes want to be able to export
> > > snapshots
> > > > as
> > > > > > > well
> > > > > > > > > > > > > > as
> > > > > > > > > > > > create
> > > > > > > > > > > > > > new VMs from their snapshots
> > > > > > > > > > > > > > d. snapshots are a currently a massive pain for
> > > > > operators,
> > > > > > I
> > > > > > > > > > > > > > know at
> > > > > > > > > > > > > least
> > > > > > > > > > > > > > one public cloud who have snapshots which take 2
> > days
> > > > to
> > > > > > > > > complete.
> > > > > > > > > > > > > > e. snapshots (as they are) can't be used for
> > multiple
> > > > LVM
> > > > > > > > disks.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > I think the process Mike has used in the
> SolidFire
> > > > plugin
> > > > > > > (only
> > > > > > > > > > > > > > moving
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > disk image to secondary storage when you
> absolutely
> > > > have
> > > > > > to)
> > > > > > > is
> > > > > > > > > > > > > > a very
> > > > > > > > > > > > > good
> > > > > > > > > > > > > > and pragmatic solution. I wonder what problems an
> > > > > operator
> > > > > > > > might
> > > > > > > > > > > > > experience
> > > > > > > > > > > > > > if they have an issue with a given primary
> storage
> > > pool
> > > > > in
> > > > > > a
> > > > > > > > > > cluster.
> > > > > > > > > > > > (I
> > > > > > > > > > > > > > know that that is REALLY unlikely in the
> SolidFire
> > > case
> > > > > > Mike
> > > > > > > :)
> > > > > > > > > > > > > > ) And
> > > > > > > > > > > > if
> > > > > > > > > > > > > > the transfer from primary to secondary is slow,
> the
> > > > time
> > > > > to
> > > > > > > > > > > > > > being able
> > > > > > > > > > > > to
> > > > > > > > > > > > > > create a template or export the volume will be
> > slow.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > So for me the issue is around making sure that
> the
> > > end
> > > > > > users
> > > > > > > > > > > > expectations
> > > > > > > > > > > > > > are met (while improving the speed/efficiency of
> > the
> > > > back
> > > > > > > end)
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > [image: ShapeBlue] <http://www.shapeblue.com>
> Paul
> > > > Angus
> > > > > > VP
> > > > > > > > > > > > > > Technology , ShapeBlue
> > > > > > > > > > > > > > d: *+44 203 617 0528 | s: +44 203 603 0540*
> > > > > > > > > > > > > >
> > > > > > <+44%20203%20617%200528%20%7C%20s:%20+44%20203%20603%200540>
> > > > > > > |
> > > > > > > > m:
> > > > > > > > > > > > > > *+44 7711 418784* <+44%207711%20418784>
> > > > > > > > > > > > > > e: *paul.an...@shapeblue.com <javascript:;>
> > > > > <javascript:;> | t:
> > > > > > @cloudyangus*
> > > > > > > > > > > > > > <paul.an...@shapeblue.com <javascript:;>
> > > > <javascript:;>
> > > > > > %20%7C%20t:%20@cloudyangus> | w:
> > > > > > > > > > > > > > *www.shapeblue.com* <http://www.shapeblue.com>
> > > > > > > > > > > > > > a: 53 Chandos Place, Covent Garden London WC2N
> 4HS
> > UK
> > > > > Shape
> > > > > > > > Blue
> > > > > > > > > > > > > > Ltd is a company incorporated in England & Wales.
> > > > > ShapeBlue
> > > > > > > > > > > > > > Services India LLP is a company incorporated in
> > India
> > > > and
> > > > > > is
> > > > > > > > > > > > > > operated
> > > > > > > > > > > > > under
> > > > > > > > > > > > > > license from Shape Blue Ltd. Shape Blue Brasil
> > > > > Consultoria
> > > > > > > Ltda
> > > > > > > > > > > > > > is a company incorporated in Brasil and is
> operated
> > > > under
> > > > > > > > > > > > > > license from Shape Blue Ltd. ShapeBlue SA Pty Ltd
> > is
> > > a
> > > > > > > company
> > > > > > > > > > > > > > registered by The Republic
> > > > > > > > > > > > of
> > > > > > > > > > > > > > South Africa and is traded under license from
> Shape
> > > > Blue
> > > > > > Ltd.
> > > > > > > > > > > > > > ShapeBlue
> > > > > > > > > > > > > is
> > > > > > > > > > > > > > a registered trademark.
> > > > > > > > > > > > > > This email and any attachments to it may be
> > > > confidential
> > > > > > and
> > > > > > > > are
> > > > > > > > > > > > intended
> > > > > > > > > > > > > > solely for the use of the individual to whom it
> is
> > > > > > addressed.
> > > > > > > > > > > > > > Any views
> > > > > > > > > > > > > or
> > > > > > > > > > > > > > opinions expressed are solely those of the author
> > and
> > > > do
> > > > > > not
> > > > > > > > > > > > necessarily
> > > > > > > > > > > > > > represent those of Shape Blue Ltd or related
> > > companies.
> > > > > If
> > > > > > > you
> > > > > > > > > > > > > > are not
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > intended recipient of this email, you must
> neither
> > > take
> > > > > any
> > > > > > > > > > > > > > action
> > > > > > > > > > > > based
> > > > > > > > > > > > > > upon its contents, nor copy or show it to anyone.
> > > > Please
> > > > > > > > contact
> > > > > > > > > > > > > > the
> > > > > > > > > > > > > sender
> > > > > > > > > > > > > > if you believe you have received this email in
> > error.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > -----Original Message-----
> > > > > > > > > > > > > > From: Pierre-Luc Dion [mailto:pd...@cloudops.com
> > > > > <javascript:;>
> > > > > > <javascript:;>]
> > > > > > > > > > > > > > Sent: Friday, February 5, 2016 12:56 PM
> > > > > > > > > > > > > > To: dev@cloudstack.apache.org <javascript:;>
> > > > > <javascript:;>
> > > > > > > > > > > > > > Subject: Re: [Propose][New Feature] Storage
> > Snapshots
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Hi Mike,
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > The idea of introducing a new API:
> StorageSnapshot
> > > for
> > > > > > > managed
> > > > > > > > > > > > > > storage
> > > > > > > > > > > > is
> > > > > > > > > > > > > > because the VolumeSnapshot default, or expected,
> > > > behavior
> > > > > > is
> > > > > > > to
> > > > > > > > > > > > > > archive snapshots into the Secondary Storage. So
> a
> > > > > > > > > > > > > > StorageSnapshot API would be
> > > > > > > > > > > > > for
> > > > > > > > > > > > > > snapshot that remain on the managed storage
> > > appliance.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Quickly looking at the API doc and I don't see a
> > > strong
> > > > > > > > > > > > > > requirement for volume snapshots to be moved into
> > > > > secondary
> > > > > > > > > > > > > > storage. So, maybe StorageSnapshot API is not
> > useful,
> > > > but
> > > > > > > both
> > > > > > > > > > > > > > use cases are required. A snapshot that remain on
> > the
> > > > > > managed
> > > > > > > > > > > > > > storage, and another type of
> > > > > > > > > > > > snapshot
> > > > > > > > > > > > > > that end up into the secondary storage. Since
> > you've
> > > > > done a
> > > > > > > lot
> > > > > > > > > > > > > > of
> > > > > > > > > > > > work,
> > > > > > > > > > > > > > might easier to just add a parameter to the
> current
> > > > > > snapshot
> > > > > > > > API
> > > > > > > > > > > > > > that
> > > > > > > > > > > > > would
> > > > > > > > > > > > > > trigger an extraction of the storage snapshot
> into
> > > the
> > > > > > > > secondary
> > > > > > > > > > > > storage?
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > PL
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > On Thu, Feb 4, 2016 at 9:02 PM, Mike Tutkowski <
> > > > > > > > > > > > > > mike.tutkow...@solidfire.com <javascript:;>
> > > > > <javascript:;>
> > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > > I think that all sounds reasonable then -
> thanks!
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > On Thu, Feb 4, 2016 at 6:52 PM, Syed Mushtaq <
> > > > > > > > > > > > syed1.mush...@gmail.com <javascript:;>
> <javascript:;>>
> > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >> You are correct Mike in terms of the
> > requirements.
> > > > One
> > > > > > of
> > > > > > > > our
> > > > > > > > > > > > earlier
> > > > > > > > > > > > > > >> iterations on this was to have an argument to
> > the
> > > > > create
> > > > > > > > > > > > > > >> snapshot
> > > > > > > > > > > > API
> > > > > > > > > > > > > > >> which decides whether to backup the volume to
> > sec
> > > > > > storage
> > > > > > > > but
> > > > > > > > > > > > > > >> we realized it would make management of
> > snapshots
> > > > > quite
> > > > > > > > messy
> > > > > > > > > > > > > > >> so we proposed a new api instead.
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > >> On Thu, Feb 4, 2016, 8:29 PM Mike Tutkowski
> > > > > > > > > > > > > > >> <mike.tutkow...@solidfire.com <javascript:;>
> > > > > <javascript:;>>
> > > > > > > > > > > > > > >> wrote:
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > >>> Hi,
> > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > >>> Just to make sure I understand all the
> > > requirements
> > > > > > here:
> > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > >>> 1) This relates only to managed storage (1:1
> > > > mapping
> > > > > > > > between
> > > > > > > > > > > > > > >>> a virtual disk and a backend SAN volume).
> > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > >>> 2) We want to take the current (introduced in
> > > 4.6)
> > > > > > > > > > > > > > >>> functionality, which creates a snapshot on
> the
> > > SAN,
> > > > > and
> > > > > > > > > > > > > > >>> extend it via a config option (or
> > > > > > > > > > > > > > >>> something) to not only take the SAN snapshot,
> > but
> > > > to
> > > > > > copy
> > > > > > > > > > > > > > >>> the underlying VHD (XenServer only) to NFS.
> > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > >>> 3) The SAN snapshot is always taken. It's the
> > > > backup
> > > > > to
> > > > > > > NFS
> > > > > > > > > > > > > > >>> that is optional.
> > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > >>> 4) Templates can be created from the snapshot
> > > > that's
> > > > > on
> > > > > > > the
> > > > > > > > > > > > > > >>> SAN (already works).
> > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > >>> 5) CloudStack volumes can be created from the
> > > > > snapshot
> > > > > > > > > > > > > > >>> that's on
> > > > > > > > > > > > the
> > > > > > > > > > > > > > >>> SAN (already works as long as the new
> > CloudStack
> > > > > volume
> > > > > > > > ends
> > > > > > > > > > > > > > >>> up on the same primary storage).
> > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > >>> Would we have a need for a storage snapshot
> API
> > > > then
> > > > > or
> > > > > > > > > > > > > > >>> would that just be the standard volume
> snapshot
> > > > > without
> > > > > > > the
> > > > > > > > > > > > > > >>> backup to
> > > > > > > > > > > NFS?
> > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > >>> Thanks!
> > > > > > > > > > > > > > >>> Mike
> > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > >>> On Thu, Feb 4, 2016 at 5:24 PM, Syed Mushtaq
> > > > > > > > > > > > > > >>> <syed1.mush...@gmail.com <javascript:;>
> > > > > <javascript:;>>
> > > > > > > > > > > > > > >>> wrote:
> > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > >>>> Is it possible to have both functionalities
> > > > > (snapshot
> > > > > > on
> > > > > > > > > > > > > > >>>> SAN & Sec
> > > > > > > > > > > > > > >>>> Storage) coexist? Because Ideally, we would
> > like
> > > > to
> > > > > > have
> > > > > > > > > both.
> > > > > > > > > > > > > > >>>> For example, some of our customers want to
> > > > implement
> > > > > > > their
> > > > > > > > > > > > > > >>>> own backup strategies and do encryption to
> > their
> > > > > > backups
> > > > > > > > > > > > > > >>>> which is a perfect use case for Storage
> > Snapshot
> > > > > while
> > > > > > > our
> > > > > > > > > > > > > > >>>> other customers will still keep using the
> > > standard
> > > > > > > volume
> > > > > > > > > > > snapshot.
> > > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > > >>>> To keep things backward compatible, we can
> > add a
> > > > > > setting
> > > > > > > > > > > > > > >>>> which
> > > > > > > > > > > > says
> > > > > > > > > > > > > > >>>> to not upload on secondary storage, because,
> > > after
> > > > > > all,
> > > > > > > > you
> > > > > > > > > > > > > > >>>> would take a SAN snapshot first when doing a
> > > > Volume
> > > > > > > > > Snapshot.
> > > > > > > > > > > > > > >>>> You could stop the process there and not do
> > the
> > > > > > upload.
> > > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > > >>>> What do you think about this approach?
> > > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > > >>>> Thanks,
> > > > > > > > > > > > > > >>>> -Syed
> > > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > > >>>> On Thu, Feb 4, 2016 at 4:25 PM, Mike
> > Tutkowski <
> > > > > > > > > > > > > > >>>> mike.tutkow...@solidfire.com <javascript:;>
> > > > > <javascript:;>> wrote:
> > > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > > >>>>> So, this is just me thinking out load here,
> > but
> > > > if
> > > > > a
> > > > > > > > given
> > > > > > > > > > > > > > >>>>> CloudStack cloud doesn't actually need to
> > > provide
> > > > > > both
> > > > > > > > the
> > > > > > > > > > > > ability
> > > > > > > > > > > > > > >>>>> to take a SAN snapshot and export it to NFS
> > (if
> > > > > just
> > > > > > > > > > > > > > >>>>> taking a SAN snapshot is OK), then we might
> > be
> > > > able
> > > > > > to
> > > > > > > > get
> > > > > > > > > > > > > > >>>>> away with no new
> > > > > > > > > > > > API
> > > > > > > > > > > > > > >>>>> calls and simply implement a new custom
> > > snapshot
> > > > > > > strategy
> > > > > > > > > > > > > > >>>>> and
> > > > > > > > > > > > data
> > > > > > > > > > > > > > >>>>> motion strategy to handle the case where
> the
> > > > > > CloudStack
> > > > > > > > > > > > > > >>>>> cloud
> > > > > > > > > > > > does
> > > > > > > > > > > > > > >>>>> want both a SAN snapshot and
> exported-to-NFS
> > > > > backup.
> > > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > > >>>>> In other words, the "default" behavior
> would
> > be
> > > > to
> > > > > > use
> > > > > > > > the
> > > > > > > > > > > > > > >>>>> snapshot strategy and data motion strategy
> > that
> > > > we
> > > > > > > > already
> > > > > > > > > > > > > > >>>>> have (the one that only takes a SAN
> > snapshot).
> > > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > > >>>>> If your CloudStack cloud, however, wants to
> > > take
> > > > a
> > > > > > SAN
> > > > > > > > > > > > > > >>>>> snapshot and have the data exported to NFS,
> > > then
> > > > we
> > > > > > > could
> > > > > > > > > > > > > > >>>>> have you manipulate a Swing config file to
> > make
> > > > use
> > > > > > of
> > > > > > > a
> > > > > > > > > > > > > > >>>>> new snapshot strategy and data motion
> > strategy
> > > > that
> > > > > > > > > > > > > > >>>>> performs both of these
> > > > > > > > > > > > > > activities.
> > > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > > >>>>> This way, the old behavior is still the
> > default
> > > > for
> > > > > > > > users,
> > > > > > > > > > > > > > >>>>> but CloudStack admins can change this
> > behavior
> > > > via
> > > > > > > > > > > configuration.
> > > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > > >>>>> Thoughts?
> > > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > > >>>>> On Thu, Feb 4, 2016 at 11:55 AM, Mike
> > > Tutkowski <
> > > > > > > > > > > > > > >>>>> mike.tutkow...@solidfire.com
> <javascript:;>
> > > > > <javascript:;>> wrote:
> > > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > > >>>>>> Right...I think we will need to come up
> > with a
> > > > > > viable
> > > > > > > > > > > > > > >>>>>> upgrade path or some reasonable way for
> them
> > > to
> > > > > move
> > > > > > > > from
> > > > > > > > > > > > > > >>>>>> the old way to the new way (and some
> obvious
> > > way
> > > > > > that
> > > > > > > > > > > > > > >>>>>> they will know they need
> > > > > > > > > > > > to
> > > > > > > > > > > > > > do this).
> > > > > > > > > > > > > > >>>>>>
> > > > > > > > > > > > > > >>>>>> On Thu, Feb 4, 2016 at 11:45 AM, Syed
> > Mushtaq
> > > <
> > > > > > > > > > > > > > >>>>>> syed1.mush...@gmail.com <javascript:;>
> > > > > <javascript:;>> wrote:
> > > > > > > > > > > > > > >>>>>>
> > > > > > > > > > > > > > >>>>>>> I'm not really sure about the upgrade
> path
> > > > > however,
> > > > > > > > > > > > > > >>>>>>> customers who are using 4.6 and are on a
> > > > managed
> > > > > > > > storage
> > > > > > > > > > > > > > >>>>>>> would no longer have the same
> functionality
> > > > with
> > > > > > > Volume
> > > > > > > > > > > Snapshots.
> > > > > > > > > > > > > > >>>>>>>
> > > > > > > > > > > > > > >>>>>>> On Thu, Feb 4, 2016 at 1:43 PM, Syed
> > Mushtaq
> > > <
> > > > > > > > > > > > > > >>>>>>> syed1.mush...@gmail.com <javascript:;>
> > > > > <javascript:;>> wrote:
> > > > > > > > > > > > > > >>>>>>>
> > > > > > > > > > > > > > >>>>>>>> So if I understand correctly, currently
> > > > taking a
> > > > > > > > Volume
> > > > > > > > > > > > > > >>>>>>>> Snapshots of a volume on a managed
> storage
> > > > keeps
> > > > > > it
> > > > > > > on
> > > > > > > > > > > > > > >>>>>>>> the storage array. As a part of this
> > > feature,
> > > > we
> > > > > > can
> > > > > > > > > > > > > > >>>>>>>> make sure
> > > > > > > > > > > > that
> > > > > > > > > > > > > > >>>>>>>> Volume Snapshots on managed storage are
> > > > uploaded
> > > > > > to
> > > > > > > > the
> > > > > > > > > > > > > > >>>>>>>> secondary storage. This would make the
> > > Volume
> > > > > > > Snapshot
> > > > > > > > > > > > > > >>>>>>>> feature behave the same regardless of
> the
> > > > > storage
> > > > > > > > > > > > > > >>>>>>>> (managed or
> > > > > > > > > > > > > > >>>>>>>> non-managed) And, for utilizing the
> > > efficient
> > > > > > > backend
> > > > > > > > > > > > > > >>>>>>>> storage
> > > > > > > > > > > > > > capabilities, we can use the new storage
> snapshots
> > > API.
> > > > > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > > > > >>>>>>>> On Thu, Feb 4, 2016 at 1:36 PM, Mike
> > > > Tutkowski <
> > > > > > > > > > > > > > >>>>>>>> mike.tutkow...@solidfire.com
> > <javascript:;>
> > > > > <javascript:;>>
> > > > > > wrote:
> > > > > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > > > > >>>>>>>>> Hi everyone,
> > > > > > > > > > > > > > >>>>>>>>>
> > > > > > > > > > > > > > >>>>>>>>> Whatever we do here, we need to have a
> > plan
> > > > to
> > > > > > deal
> > > > > > > > > > > > > > >>>>>>>>> with the fact that we already have a
> > > feature
> > > > > (in
> > > > > > > 4.6
> > > > > > > > > > > > > > >>>>>>>>> and
> > > > > > > > > > > > > > >>>>>>>>> later) that allows you to use the
> > existing
> > > > > > > > > > > > > > >>>>>>>>> volume-snapshot APIs to create a volume
> > > > > snapshot
> > > > > > > (for
> > > > > > > > > > > > > > >>>>>>>>> managed
> > > > > > > > > > > > > > >>>>>>>>> storage) that resides on a backend SAN
> > > > (using a
> > > > > > > > custom
> > > > > > > > > > > > > > >>>>>>>>> snapshot strategy and a custom data
> > motion
> > > > > > > strategy).
> > > > > > > > > > > > > > >>>>>>>>>
> > > > > > > > > > > > > > >>>>>>>>> If these new APIs go in, then how
> should
> > > the
> > > > > > > original
> > > > > > > > > > > > > > >>>>>>>>> implementation (present in 4.6 and
> later)
> > > be
> > > > > > > changed?
> > > > > > > > > > > > > > >>>>>>>>> If it
> > > > > > > > > > > > is
> > > > > > > > > > > > > > >>>>>>>>> changed, how do we support customers
> who
> > > are
> > > > > > > already
> > > > > > > > > > > > > > >>>>>>>>> using
> > > > > > > > > > > > the
> > > > > > > > > > > > > > >>>>>>>>> original volume-snapshot API to take
> > > > snapshots
> > > > > > on a
> > > > > > > > > > > > > > >>>>>>>>> backend
> > > > > > > > > > > > > SAN?
> > > > > > > > > > > > > > >>>>>>>>>
> > > > > > > > > > > > > > >>>>>>>>> Thanks,
> > > > > > > > > > > > > > >>>>>>>>> Mike
> > > > > > > > > > > > > > >>>>>>>>>
> > > > > > > > > > > > > > >>>>>>>>> On Thu, Feb 4, 2016 at 11:27 AM, Will
> > > > Stevens <
> > > > > > > > > > > > > > >>>>>>>>> wstev...@cloudops.com <javascript:;>
> > > > > <javascript:;>> wrote:
> > > > > > > > > > > > > > >>>>>>>>>
> > > > > > > > > > > > > > >>>>>>>>>> Will you be able to create a Template
> > > from a
> > > > > > > > > > > > StorageSnapshot?
> > > > > > > > > > > > > > >>>>>>>>>> If yes, will the template be stored in
> > the
> > > > > > > secondary
> > > > > > > > > > > > > > >>>>>>>>>> storage like normal templates or will
> > that
> > > > be
> > > > > > > > handled
> > > > > > > > > > > > > > >>>>>>>>>> somehow on the
> > > > > > > > > > > > > > vendor side?
> > > > > > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > > > > > >>>>>>>>>> *Will STEVENS*
> > > > > > > > > > > > > > >>>>>>>>>> Lead Developer
> > > > > > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > > > > > >>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
> > > > > > > > > > > > > > >>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec
> *|*
> > > H3J
> > > > > 1S6
> > > > > > w
> > > > > > > > > > > > > > >>>>>>>>>> cloudops.com *|* tw @CloudOps_
> > > > > > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > > > > > >>>>>>>>>> On Thu, Feb 4, 2016 at 1:22 PM, Syed
> > > > Mushtaq <
> > > > > > > > > > > > > > >>>>>>>>>> syed1.mush...@gmail.com
> <javascript:;>
> > > > > <javascript:;>> wrote:
> > > > > > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > > > > > >>>>>>>>>>> Thanks Will!!!
> > > > > > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > > > > > >>>>>>>>>>> On Thu, Feb 4, 2016 at 1:19 PM, Will
> > > > Stevens
> > > > > <
> > > > > > > > > > > > > > >>>>>>>>>>> wstev...@cloudops.com <javascript:;>
> > > > > <javascript:;>> wrote:
> > > > > > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > > > > > >>>>>>>>>>>> I explicitly linked the Design Spec
> in
> > > the
> > > > > > Jira
> > > > > > > > > > > > > > >>>>>>>>>>>> ticket because it was not clear in
> the
> > > > > > 'mention'
> > > > > > > > > > > > > > >>>>>>>>>>>> section because it shows as a page
> > 'you
> > > do
> > > > > not
> > > > > > > > have
> > > > > > > > > > > permission to'.
> > > > > > > > > > > > > > >>>>>>>>>>>>
> > > > > > > > > > > > > > >>>>>>>>>>>> *Will STEVENS*
> > > > > > > > > > > > > > >>>>>>>>>>>> Lead Developer
> > > > > > > > > > > > > > >>>>>>>>>>>>
> > > > > > > > > > > > > > >>>>>>>>>>>> *CloudOps* *| *Cloud Solutions
> Experts
> > > > > > > > > > > > > > >>>>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec
> > *|*
> > > > H3J
> > > > > > 1S6
> > > > > > > w
> > > > > > > > > > > > > > >>>>>>>>>>>> cloudops.com *|* tw @CloudOps_
> > > > > > > > > > > > > > >>>>>>>>>>>>
> > > > > > > > > > > > > > >>>>>>>>>>>> On Thu, Feb 4, 2016 at 1:02 PM, Syed
> > > Ahmed
> > > > > > > > > > > > > > >>>>>>>>>>>> <sah...@cloudops.com <javascript:;>
> > > > > <javascript:;>
> > > > > > > > > > > > > > >>>>>>>>>>>> > wrote:
> > > > > > > > > > > > > > >>>>>>>>>>>>
> > > > > > > > > > > > > > >>>>>>>>>>>>>
> > > > > > > > > > > > > > >>>>>>>>>>>>> Design Spec:
> > > > > > > > > > > > > > >>>>>>>>>>>>>
> > > > > > > > > > > > > > >>>>>>>>>>>>>
> > > > > > > > > > > > > > >>>>>>>>>>>>>
> > > > > > > > > > > >
> > > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Sto
> > > > > > > > > > > > > > >>>>>>>>>>>>> rageSnapshot++API
> > > > > > > > > > > > > > >>>>>>>>>>>>>
> > > > > > > > > > > > > > >>>>>>>>>>>>> Jira Ticket
> > > > > > > > > > > > > > >>>>>>>>>>>>>
> > > > > > > > > > > > > > >>>>>>>>>>>>>
> > > > > > > > https://issues.apache.org/jira/browse/CLOUDSTACK-9
> > > > > > > > > > > > > > >>>>>>>>>>>>> 27
> > > > > > > > > > > > > > >>>>>>>>>>>>> 8
> > > > > > > > > > > > > > >>>>>>>>>>>>>
> > > > > > > > > > > > > > >>>>>>>>>>>>>
> > > > > > > > > > > > > > >>>>>>>>>>>>> Hi All,
> > > > > > > > > > > > > > >>>>>>>>>>>>>
> > > > > > > > > > > > > > >>>>>>>>>>>>> We plan to propose a new set of
> APIs
> > to
> > > > do
> > > > > > > > > > > > > > >>>>>>>>>>>>> snapshots on managed storage
> backends
> > > > like
> > > > > > > > > SolidFire.
> > > > > > > > > > > > > > >>>>>>>>>>>>> Snapshots on current managed
> storage
> > > stay
> > > > > on
> > > > > > > the
> > > > > > > > > > > > > > >>>>>>>>>>>>> device which is contrary to what
> > > > CloudStack
> > > > > > > calls
> > > > > > > > > > > snpshots.
> > > > > > > > > > > > > > >>>>>>>>>>>>> But taking snapshots on storage and
> > > > keeping
> > > > > > it
> > > > > > > > > > > > > > >>>>>>>>>>>>> there has its own advantages
> > > > > > > > > > > > and
> > > > > > > > > > > > > > >>>>>>>>>>>>> we would ideally like to have both
> > ways
> > > > of
> > > > > > > doing
> > > > > > > > > > > > > > >>>>>>>>>>>>> snapshots. This proposal adds 4 new
> > > APIs
> > > > to
> > > > > > > > create
> > > > > > > > > > > > > > >>>>>>>>>>>>> snapshots on backend storage.
> > > > > > > > > > > > > > >>>>>>>>>>>>>
> > > > > > > > > > > > > > >>>>>>>>>>>>> What do you guys think of this
> > > feature? I
> > > > > > would
> > > > > > > > > > > > > > >>>>>>>>>>>>> love to have some feedback. I am
> > > working
> > > > on
> > > > > > > > making
> > > > > > > > > > > > > > >>>>>>>>>>>>> the design
> > > > > > > > > > > > spec
> > > > > > > > > > > > > > >>>>>>>>>>>>> more concrete but wanted to have a
> > high
> > > > > level
> > > > > > > > > > > > > > >>>>>>>>>>>>> feedback first before starting to
> > work
> > > on
> > > > > it.
> > > > > > > > > > > > > > >>>>>>>>>>>>>
> > > > > > > > > > > > > > >>>>>>>>>>>>> Thanks,
> > > > > > > > > > > > > > >>>>>>>>>>>>> -Syed
> > > > > > > > > > > > > > >>>>>>>>>>>>>
> > > > > > > > > > > > > > >>>>>>>>>>>>>
> > > > > > > > > > > > > > >>>>>>>>>>>>
> > > > > > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > > > > > >>>>>>>>>
> > > > > > > > > > > > > > >>>>>>>>>
> > > > > > > > > > > > > > >>>>>>>>> --
> > > > > > > > > > > > > > >>>>>>>>> *Mike Tutkowski*
> > > > > > > > > > > > > > >>>>>>>>> *Senior CloudStack Developer, SolidFire
> > > Inc.*
> > > > > > > > > > > > > > >>>>>>>>> e: mike.tutkow...@solidfire.com
> > > > <javascript:;>
> > > > > <javascript:;>
> > > > > > > > > > > > > > >>>>>>>>> o: 303.746.7302
> > > > > > > > > > > > > > >>>>>>>>> Advancing the way the world uses the
> > cloud
> > > > > > > > > > > > > > >>>>>>>>> <
> > > > > > > http://solidfire.com/solution/overview/?video=play
> > > > > > > > >*™
> > > > > > > > > > > > > > >>>>>>>>> *
> > > > > > > > > > > > > > >>>>>>>>>
> > > > > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > > > > >>>>>>>
> > > > > > > > > > > > > > >>>>>>
> > > > > > > > > > > > > > >>>>>>
> > > > > > > > > > > > > > >>>>>> --
> > > > > > > > > > > > > > >>>>>> *Mike Tutkowski*
> > > > > > > > > > > > > > >>>>>> *Senior CloudStack Developer, SolidFire
> > Inc.*
> > > > > > > > > > > > > > >>>>>> e: mike.tutkow...@solidfire.com
> > > <javascript:;>
> > > > > <javascript:;>
> > > > > > > > > > > > > > >>>>>> o: 303.746.7302
> > > > > > > > > > > > > > >>>>>> Advancing the way the world uses the cloud
> > > > > > > > > > > > > > >>>>>> <
> > > > > http://solidfire.com/solution/overview/?video=play
> > > > > > > >*™*
> > > > > > > > > > > > > > >>>>>>
> > > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > > >>>>> --
> > > > > > > > > > > > > > >>>>> *Mike Tutkowski*
> > > > > > > > > > > > > > >>>>> *Senior CloudStack Developer, SolidFire
> Inc.*
> > > > > > > > > > > > > > >>>>> e: mike.tutkow...@solidfire.com
> > <javascript:;>
> > > > > <javascript:;>
> > > > > > > > > > > > > > >>>>> o: 303.746.7302
> > > > > > > > > > > > > > >>>>> Advancing the way the world uses the cloud
> > > > > > > > > > > > > > >>>>> <
> > > > > http://solidfire.com/solution/overview/?video=play
> > > > > > > >*™*
> > > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > >>> --
> > > > > > > > > > > > > > >>> *Mike Tutkowski*
> > > > > > > > > > > > > > >>> *Senior CloudStack Developer, SolidFire Inc.*
> > > > > > > > > > > > > > >>> e: mike.tutkow...@solidfire.com
> <javascript:;>
> > > > > <javascript:;>
> > > > > > > > > > > > > > >>> o: 303.746.7302
> > > > > > > > > > > > > > >>> Advancing the way the world uses the cloud
> > > > > > > > > > > > > > >>> <
> > > > http://solidfire.com/solution/overview/?video=play
> > > > > > >*™*
> > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > --
> > > > > > > > > > > > > > > *Mike Tutkowski*
> > > > > > > > > > > > > > > *Senior CloudStack Developer, SolidFire Inc.*
> > > > > > > > > > > > > > > e: mike.tutkow...@solidfire.com <javascript:;>
> > > > > <javascript:;>
> > > > > > > > > > > > > > > o: 303.746.7302
> > > > > > > > > > > > > > > Advancing the way the world uses the cloud
> > > > > > > > > > > > > > > <
> > > http://solidfire.com/solution/overview/?video=play
> > > > > >*™*
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > Find out more about ShapeBlue and our range of
> > > > CloudStack
> > > > > > > > > > > > > > related
> > > > > > > > > > > > > services:
> > > > > > > > > > > > > > IaaS Cloud Design & Build
> > > > > > > > > > > > > > <
> > http://shapeblue.com/iaas-cloud-design-and-build//>
> > > |
> > > > > > > > CSForge –
> > > > > > > > > > > > > > rapid IaaS deployment framework <
> > > > > > > http://shapeblue.com/csforge/
> > > > > > > > >
> > > > > > > > > > > > > > CloudStack Consulting
> > > > > > > > > > > > > > <http://shapeblue.com/cloudstack-consultancy/> |
> > > > > > > > > > > > > CloudStack
> > > > > > > > > > > > > > Software Engineering
> > > > > > > > > > > > > > <
> > > http://shapeblue.com/cloudstack-software-engineering/
> > > > >
> > > > > > > > > > > > > > CloudStack Infrastructure Support
> > > > > > > > > > > > > > <
> > > > http://shapeblue.com/cloudstack-infrastructure-support/
> > > > > >
> > > > > > |
> > > > > > > > > > > > > > CloudStack Bootcamp Training Courses
> > > > > > > > > > > > > > <http://shapeblue.com/cloudstack-training/>
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > --
> > > > > > > > > > > > *Mike Tutkowski*
> > > > > > > > > > > > *Senior CloudStack Developer, SolidFire Inc.*
> > > > > > > > > > > > e: mike.tutkow...@solidfire.com <javascript:;>
> > > > > <javascript:;>
> > > > > > > > > > > > o: 303.746.7302
> > > > > > > > > > > > Advancing the way the world uses the cloud
> > > > > > > > > > > > <http://solidfire.com/solution/overview/?video=play
> > >*™*
> > > > > > > > > > > >
> > > > > > > > > > > Find out more about ShapeBlue and our range of
> CloudStack
> > > > > related
> > > > > > > > > > services:
> > > > > > > > > > > IaaS Cloud Design & Build
> > > > > > > > > > > <http://shapeblue.com/iaas-cloud-design-and-build//> |
> > > > > CSForge –
> > > > > > > > rapid
> > > > > > > > > > > IaaS deployment framework <
> http://shapeblue.com/csforge/
> > >
> > > > > > > CloudStack
> > > > > > > > > > > Consulting <
> http://shapeblue.com/cloudstack-consultancy/
> > >
> > > |
> > > > > > > > CloudStack
> > > > > > > > > > > Software Engineering
> > > > > > > > > > > <http://shapeblue.com/cloudstack-software-engineering/
> >
> > > > > > > > > > > CloudStack Infrastructure Support
> > > > > > > > > > > <
> http://shapeblue.com/cloudstack-infrastructure-support/
> > >
> > > |
> > > > > > > > CloudStack
> > > > > > > > > > > Bootcamp Training Courses <
> > > > > > > http://shapeblue.com/cloudstack-training/
> > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > Find out more about ShapeBlue and our range of CloudStack
> > > > related
> > > > > > > > > services:
> > > > > > > > > > IaaS Cloud Design & Build
> > > > > > > > > > <http://shapeblue.com/iaas-cloud-design-and-build//> |
> > > > CSForge –
> > > > > > > rapid
> > > > > > > > > > IaaS deployment framework <http://shapeblue.com/csforge/
> >
> > > > > > > > > > CloudStack Consulting <
> > > > > > http://shapeblue.com/cloudstack-consultancy/>
> > > > > > > |
> > > > > > > > > CloudStack
> > > > > > > > > > Software Engineering
> > > > > > > > > > <http://shapeblue.com/cloudstack-software-engineering/>
> > > > > > > > > > CloudStack Infrastructure Support
> > > > > > > > > > <http://shapeblue.com/cloudstack-infrastructure-support/
> >
> > |
> > > > > > > CloudStack
> > > > > > > > > > Bootcamp Training Courses <
> > > > > > http://shapeblue.com/cloudstack-training/
> > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > *Mike Tutkowski*
> > > > > > > *Senior CloudStack Developer, SolidFire Inc.*
> > > > > > > e: mike.tutkow...@solidfire.com <javascript:;> <javascript:;>
> > > > > > > o: 303.746.7302
> > > > > > > Advancing the way the world uses the cloud
> > > > > > > <http://solidfire.com/solution/overview/?video=play>*™*
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > *Mike Tutkowski*
> > > > > *Senior CloudStack Developer, SolidFire Inc.*
> > > > > e: mike.tutkow...@solidfire.com <javascript:;>
> > > > > o: 303.746.7302
> > > > > Advancing the way the world uses the cloud
> > > > > <http://solidfire.com/solution/overview/?video=play>*™*
> > > > >
> > > >
> > > >
> > > > --
> > > > Ian Rae
> > > > CEO | PDG
> > > > c: 514.944.4008
> > > >
> > > > CloudOps | Cloud Infrastructure and Networking Solutions
> > > > www.cloudops.com | 420 rue Guy | Montreal | Canada | H3J 1S6
> > > >
> > >
> >
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkow...@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the cloud
> > <http://solidfire.com/solution/overview/?video=play>*™*
> >
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkow...@solidfire.com
o: 303.746.7302
Advancing the way the world uses the cloud
<http://solidfire.com/solution/overview/?video=play>*™*
