Re: [Propose][New Feature] Storage Snapshots

2016-02-12 Thread Mike Tutkowski
>> > > > > > > > > > > > > > > > > >> > > > > > On Fri, Feb 5, 2016 at 1:34 PM, Mike > > > Tutkowski < > > > > > > > > > > > >> > > > > > mike

Re: [Propose][New Feature] Storage Snapshots

2016-02-12 Thread Syed Mushtaq
0540* > > > > > > <+44%20203%20617%200528%20%7C%20s:%20+44%20203%20603%200540> | > m: > > > > > > *+44 7711 418784* <+44%207711%20418784> > > > > > > e: *paul.an...@shapeblue.com | t: @cloudyangus* > > > > > >

Re: [Propose][New Feature] Storage Snapshots

2016-02-12 Thread Mike Tutkowski
vent Garden London WC2N 4HS UK > > > > > Shape Blue Ltd is a company incorporated in England & Wales. > > ShapeBlue > > > > > Services India LLP is a company incorporated in India and is > operated > > > > under > > > > > license fro

Re: [Propose][New Feature] Storage Snapshots

2016-02-12 Thread Syed Mushtaq
> > > > > > >> > > > > > > > wrote: > > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > I think I share you view on the 'Ideal > > world'. > > > > > > Backup > >

Re: [Propose][New Feature] Storage Snapshots

2016-02-12 Thread Mike Tutkowski
> > > > > > > >> in > > > > > > > >> > the > > > > > > > >> > > > > > > future). A snapshot is a point-in-time view of > the > > > > data > > &g

Re: [Propose][New Feature] Storage Snapshots

2016-02-12 Thread Syed Mushtaq
Ltd or related companies. If you are not > the > > intended recipient of this email, you must neither take any action based > > upon its contents, nor copy or show it to anyone. Please contact the > sender > > if you believe you have received this email in error. > > &g

Re: [Propose][New Feature] Storage Snapshots

2016-02-08 Thread Mike Tutkowski
if you believe you have received this email in error. > > > -----Original Message- > From: Mike Tutkowski [mailto:mike.tutkow...@solidfire.com] > Sent: Monday, February 8, 2016 7:16 PM > To: dev@cloudstack.apache.org > Subject: Re: [Propose][New Feature] Storage Snapshots

Re: [Propose][New Feature] Storage Snapshots

2016-02-08 Thread Mike Tutkowski
> them > > > > > > >> > > > > > > as > > > > > > >> > > > > > > > a safety vest against Primary Storage failure, > > > because > > > > > > >> failure > > > > > > >> > > > append > &

Re: [Propose][New Feature] Storage Snapshots

2016-02-08 Thread Will Stevens
gt; > snapshots that end in the Secondary Storage is > not > > to > > > > > >> address > > > > > >> > > any > > > > > >> > > > > kind > > > > > >> > > > > > > of > > > >

Re: [Propose][New Feature] Storage Snapshots

2016-02-08 Thread Mike Tutkowski
ageSnapshotAPI: Provide storage side snapshot > > > > >> capability > > > > >> > > for > > > > >> > > > > > fast > > > > >> > > > > > > > response time that support rollback to previous > > > timesta

Re: [Propose][New Feature] Storage Snapshots

2016-02-08 Thread Will Stevens
> >> > > > > > > this > > > >> > > > > > > > is a different behaviors than the user expectation of > a > > > >> > > > > > volume-snapshot. > > > >

Re: [Propose][New Feature] Storage Snapshots

2016-02-08 Thread Mike Tutkowski
>> snapshot > > >> > > is a > > >> > > > > > slow > > >> > > > > > > > job since yes it would have to copy the full volume size > > on > > >> the > > >> > > > &g

Re: [Propose][New Feature] Storage Snapshots

2016-02-08 Thread Mike Tutkowski
;backups', but many > people > > > use > > > >> > > volume > > > >> > > > > > > > snapshots > > > >> > > > > > > > > > as backups. However you can't in-place restore > > volume > > > >> > > snap

Re: [Propose][New Feature] Storage Snapshots

2016-02-08 Thread Ian Rae
gt; snapshot > > >> > > is a > > >> > > > > > slow > > >> > > > > > > > job since yes it would have to copy the full volume size > > on > > >> the > > >> > > > > > Secondary > > >> > > > > > >

Re: [Propose][New Feature] Storage Snapshots

2016-02-08 Thread Syed Mushtaq
> > > > > wrote: > >> > > > > > > > > >> > > > > > > > > I think I share you view on the 'Ideal world'. Backup > (via > >> > > Volume > >> > > > > > > > > Snapshot

Re: [Propose][New Feature] Storage Snapshots

2016-02-08 Thread Will Stevens
016 at 12:45 PM, Syed Mushtaq < > >> > > > > syed1.mush...@gmail.com > >> > > > > > > > >> > > > > > > > wrote: > >> > > > > > > > > >> > > > > > > > >

Re: [Propose][New Feature] Storage Snapshots

2016-02-08 Thread Mike Tutkowski
an NFS staging area and from >> > there >> > > to >> > > > > > > object >> > > > > > > > > storage). And not to mention that all these copies are >> > > consuming >> > >

Re: [Propose][New Feature] Storage Snapshots

2016-02-08 Thread Mike Tutkowski
> > > to > > > > > sec > > > > > > > > > storage). > > > > > > > > > > > > > > > > > > But as Erik pointed out volume snapshots are not backups. > > They > > &g

Re: [Propose][New Feature] Storage Snapshots

2016-02-08 Thread Syed Mushtaq
Feb 5, 2016 at 12:29 PM, Paul Angus < > > > > > paul.an...@shapeblue.com > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > In the beginning... there were CloudStack snapshots and > they > &g

Re: [Propose][New Feature] Storage Snapshots

2016-02-08 Thread Mike Tutkowski
t; > > volume > > > > > > > > snapshots. > > > > > > > > > > > > > > > > CloudStack does not offer 'backups', but many people use > volume > > > > > > snapshots > > > > > > >

Re: [Propose][New Feature] Storage Snapshots

2016-02-08 Thread Pierre-Luc Dion
> > > > to > > > > > > be > > > > > > > consistent. > > > > > > > > > > > > > > 'Actual Backups' would enable all of the restore options which > > > users > > > > > >

Re: [Propose][New Feature] Storage Snapshots

2016-02-08 Thread Ian Rae
Syed, > > > > > > > > > > > > > > As I understand it, the SolidFire plugin will export the > snapshot > > > to > > > > > > > secondary storage if the user requests a template from the > > snapshot > > > > or

Re: [Propose][New Feature] Storage Snapshots

2016-02-08 Thread Mike Tutkowski
..@shapeblue.com | t: @cloudyangus* > > > > > %20%7C%20t:%20@cloudyangus> > | w: > > > > > *www.shapeblue.com* <http://www.shapeblue.com> > > > > > a: 53 Chandos Place, Covent Garden London WC2N 4HS UK > > > > > Shape Blu

Re: [Propose][New Feature] Storage Snapshots

2016-02-08 Thread Pierre-Luc Dion
s a company incorporated in India and is operated > > > under > > > > license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a > > > > company incorporated in Brasil and is operated under license from > Shape > > > > Blue Ltd. ShapeBlue

Re: [Propose][New Feature] Storage Snapshots

2016-02-05 Thread Mike Tutkowski
rom Shape Blue Ltd. ShapeBlue > > is > > > a registered trademark. > > > This email and any attachments to it may be confidential and are > intended > > > solely for the use of the individual to whom it is addressed. Any views > > or > > > opinions expressed a

Re: [Propose][New Feature] Storage Snapshots

2016-02-05 Thread Pierre-Luc Dion
those of Shape Blue Ltd or related companies. If you are not > the > > intended recipient of this email, you must neither take any action based > > upon its contents, nor copy or show it to anyone. Please contact the > sender > > if you believe you have received this email

Re: [Propose][New Feature] Storage Snapshots

2016-02-05 Thread Syed Mushtaq
the > intended recipient of this email, you must neither take any action based > upon its contents, nor copy or show it to anyone. Please contact the sender > if you believe you have received this email in error. > > > -Original Message- > From: Syed Mushtaq [mailto:syed1.

RE: [Propose][New Feature] Storage Snapshots

2016-02-05 Thread Paul Angus
sage- From: Syed Mushtaq [mailto:syed1.mush...@gmail.com] Sent: Friday, February 5, 2016 4:58 PM To: dev@cloudstack.apache.org Subject: Re: [Propose][New Feature] Storage Snapshots Paul, When you say actual backups, how would it be different from the Volume Snapshots that exist currently.

Re: [Propose][New Feature] Storage Snapshots

2016-02-05 Thread Erik Weber
are met (while improving the speed/efficiency of the back end) > > > > > > > > > > > > > > > [image: ShapeBlue] <http://www.shapeblue.com> Paul Angus VP > > > > > Technology , ShapeBlue > > > > > d: *+44 203 617 0528 |

Re: [Propose][New Feature] Storage Snapshots

2016-02-05 Thread Syed Mushtaq
..@shapeblue.com | t: @cloudyangus* > > > > | w: > > > > *www.shapeblue.com* <http://www.shapeblue.com> > > > > a: 53 Chandos Place, Covent Garden London WC2N 4HS UK Shape Blue > > > > Ltd is a company incorporated in England & Wales. ShapeBlue >

RE: [Propose][New Feature] Storage Snapshots

2016-02-05 Thread Paul Angus
ke any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. -Original Message- From: Syed Mushtaq [mailto:syed1.mush...@gmail.com] Sent: 05 February 2016 15:31 To: dev@cloudstack.apache.org Subj

Re: [Propose][New Feature] Storage Snapshots

2016-02-05 Thread Syed Mushtaq
> South Africa and is traded under license from Shape Blue Ltd. ShapeBlue > > is > > > a registered trademark. > > > This email and any attachments to it may be confidential and are > intended > > > solely for the use of the individual to whom it is addressed. Any vi

Re: [Propose][New Feature] Storage Snapshots

2016-02-05 Thread Mike Tutkowski
y those of the author and do not necessarily > > represent those of Shape Blue Ltd or related companies. If you are not > the > > intended recipient of this email, you must neither take any action based > > upon its contents, nor copy or show it to anyone. Please contact the > sender > >

Re: [Propose][New Feature] Storage Snapshots

2016-02-05 Thread Syed Mushtaq
ntact the sender > if you believe you have received this email in error. > > > -Original Message- > From: Pierre-Luc Dion [mailto:pd...@cloudops.com] > Sent: Friday, February 5, 2016 12:56 PM > To: dev@cloudstack.apache.org > Subject: Re: [Propose][New Feature] Storage Sn

Re: [Propose][New Feature] Storage Snapshots

2016-02-05 Thread Mike Tutkowski
Hey Pierre-Luc, For me, it's not a matter of how much work has already been done, but rather maintaining backward compatibility. Today, if using managed storage, if you elect to create a volume snapshot (and the storage driver supports this), it is only taken on the backend system (in my case, th

RE: [Propose][New Feature] Storage Snapshots

2016-02-05 Thread Paul Angus
cloudstack.apache.org Subject: Re: [Propose][New Feature] Storage Snapshots Hi Mike, The idea of introducing a new API: StorageSnapshot for managed storage is because the VolumeSnapshot default, or expected, behavior is to archive snapshots into the Secondary Storage. So a StorageSnapshot API w

Re: [Propose][New Feature] Storage Snapshots

2016-02-05 Thread Pierre-Luc Dion
Hi Mike, The idea of introducing a new API: StorageSnapshot for managed storage is because the VolumeSnapshot default, or expected, behavior is to archive snapshots into the Secondary Storage. So a StorageSnapshot API would be for snapshot that remain on the managed storage appliance. Quickly loo

Re: [Propose][New Feature] Storage Snapshots

2016-02-04 Thread Mike Tutkowski
I think that all sounds reasonable then - thanks! On Thu, Feb 4, 2016 at 6:52 PM, Syed Mushtaq wrote: > You are correct Mike in terms of the requirements. One of our earlier > iterations on this was to have an argument to the create snapshot API which > decides whether to backup the volume to se

Re: [Propose][New Feature] Storage Snapshots

2016-02-04 Thread Syed Mushtaq
You are correct Mike in terms of the requirements. One of our earlier iterations on this was to have an argument to the create snapshot API which decides whether to backup the volume to sec storage but we realized it would make management of snapshots quite messy so we proposed a new api instead.

Re: [Propose][New Feature] Storage Snapshots

2016-02-04 Thread Mike Tutkowski
Hi, Just to make sure I understand all the requirements here: 1) This relates only to managed storage (1:1 mapping between a virtual disk and a backend SAN volume). 2) We want to take the current (introduced in 4.6) functionality, which creates a snapshot on the SAN, and extend it via a config o

Re: [Propose][New Feature] Storage Snapshots

2016-02-04 Thread Syed Mushtaq
Is it possible to have both functionalities (snapshot on SAN & Sec Storage) coexist? Because Ideally, we would like to have both. For example, some of our customers want to implement their own backup strategies and do encryption to their backups which is a perfect use case for Storage Snapshot whil

Re: [Propose][New Feature] Storage Snapshots

2016-02-04 Thread Mike Tutkowski
So, this is just me thinking out load here, but if a given CloudStack cloud doesn't actually need to provide both the ability to take a SAN snapshot and export it to NFS (if just taking a SAN snapshot is OK), then we might be able to get away with no new API calls and simply implement a new custom

Re: [Propose][New Feature] Storage Snapshots

2016-02-04 Thread Will Stevens
Thanks Syed, that is much clearer now. :) *Will STEVENS* Lead Developer *CloudOps* *| *Cloud Solutions Experts 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw @CloudOps_ On Thu, Feb 4, 2016 at 1:36 PM, Syed Mushtaq wrote: > Right now we don't plan to create templates fro

Re: [Propose][New Feature] Storage Snapshots

2016-02-04 Thread Mike Tutkowski
Right...I think we will need to come up with a viable upgrade path or some reasonable way for them to move from the old way to the new way (and some obvious way that they will know they need to do this). On Thu, Feb 4, 2016 at 11:45 AM, Syed Mushtaq wrote: > I'm not really sure about the upgrade

Re: [Propose][New Feature] Storage Snapshots

2016-02-04 Thread Syed Mushtaq
I'm not really sure about the upgrade path however, customers who are using 4.6 and are on a managed storage would no longer have the same functionality with Volume Snapshots. On Thu, Feb 4, 2016 at 1:43 PM, Syed Mushtaq wrote: > So if I understand correctly, currently taking a Volume Snapshots

Re: [Propose][New Feature] Storage Snapshots

2016-02-04 Thread Syed Mushtaq
So if I understand correctly, currently taking a Volume Snapshots of a volume on a managed storage keeps it on the storage array. As a part of this feature, we can make sure that Volume Snapshots on managed storage are uploaded to the secondary storage. This would make the Volume Snapshot feature b

Re: [Propose][New Feature] Storage Snapshots

2016-02-04 Thread Mike Tutkowski
Hi everyone, Whatever we do here, we need to have a plan to deal with the fact that we already have a feature (in 4.6 and later) that allows you to use the existing volume-snapshot APIs to create a volume snapshot (for managed storage) that resides on a backend SAN (using a custom snapshot strateg

Re: [Propose][New Feature] Storage Snapshots

2016-02-04 Thread Syed Mushtaq
Right now we don't plan to create templates from this functionality mainly because you can achieve the same effect using the standard Volume Snapshots API. Since the template would always reside on the secondary storage, it doesn't really make sense to use the backed cloning if a copy to secondary

Re: [Propose][New Feature] Storage Snapshots

2016-02-04 Thread Will Stevens
Will you be able to create a Template from a StorageSnapshot? If yes, will the template be stored in the secondary storage like normal templates or will that be handled somehow on the vendor side? *Will STEVENS* Lead Developer *CloudOps* *| *Cloud Solutions Experts 420 rue Guy *|* Montreal *|* Q

Re: [Propose][New Feature] Storage Snapshots

2016-02-04 Thread Syed Mushtaq
Thanks Will!!! On Thu, Feb 4, 2016 at 1:19 PM, Will Stevens wrote: > I explicitly linked the Design Spec in the Jira ticket because it was not > clear in the 'mention' section because it shows as a page 'you do not have > permission to'. > > *Will STEVENS* > Lead Developer > > *CloudOps* *| *Clo

Re: [Propose][New Feature] Storage Snapshots

2016-02-04 Thread Will Stevens
I explicitly linked the Design Spec in the Jira ticket because it was not clear in the 'mention' section because it shows as a page 'you do not have permission to'. *Will STEVENS* Lead Developer *CloudOps* *| *Cloud Solutions Experts 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com

[Propose][New Feature] Storage Snapshots

2016-02-04 Thread Syed Ahmed
Design Spec: https://cwiki.apache.org/confluence/display/CLOUDSTACK/StorageSnapshot++API Jira Ticket https://issues.apache.org/jira/browse/CLOUDSTACK-9278 Hi All, We plan to propose a new set of APIs to do snapshots on managed storage backends like SolidFire. Snapshots on current managed stor