I have created a github issue to capture this here: https://github.com/apache/cloudstack/issues/3096
I’m getting ready to create a PR to add this feature back in, but it will require adding a configuration item back. Should this be considered a bug and go into the 4.11.3 migration, or should it be added to master? If master / 4.12? Thanks, Nathan > On May 21, 2018, at 2:28 PM, Simon Weller <swel...@ena.com.INVALID> wrote: > > yeah, I like that idea Mike. > > > Also, I want to be clear that breaking out the backup to secondary into a > separate thread is a great feature, so kudos to those that developed it. > That along with the ability to turn it off completely makes for a nice > overall improvement to snapshots. > > > - Si > > ________________________________ > From: Tutkowski, Mike <mike.tutkow...@netapp.com> > Sent: Saturday, May 19, 2018 8:40 PM > To: dev@cloudstack.apache.org > Subject: Re: Snapshots only on Primary Storage feature > > Perhaps instead of renaming the setting, we can note in its description the > hypervisors it currently pertains to. > >> On May 19, 2018, at 7:03 PM, Glen Baars <g...@onsitecomputers.com.au> wrote: >> >> Based on the responses, I think it is a worthy feature to be retained. Maybe >> the following changes? >> >> Rename the setting to something like kvmxen.snapshot.primaryonly ( I have no >> idea of the naming scheme that Cloudstack uses ) >> Ensure the code for vmware snapshots does not get impacted by the setting >> Record in the DB that the snapshot is only on the primary storage >> When the create template or download template features are used, use the >> primary storage as the source. >> >> Kind regards, >> Glen Baars >> >> -----Original Message----- >> From: Will Stevens <wstev...@cloudops.com> >> Sent: Saturday, 19 May 2018 12:57 PM >> To: dev@cloudstack.apache.org >> Subject: Re: Snapshots only on Primary Storage feature >> >> I think reverting the change in 4.11.1 is probably a good idea. >> >> *Will Stevens* >> Chief Technology Officer >> c 514.826.0190 >> >> <https://goo.gl/NYZ8KK> >> >> >> On Fri, May 18, 2018 at 2:52 PM ilya musayev <ilya.mailing.li...@gmail.com> >> wrote: >> >>> Perhaps bring it back into 4.11.1? >>> >>> On Fri, May 18, 2018 at 9:28 AM Suresh Kumar Anaparti < >>> sureshkumar.anapa...@gmail.com> wrote: >>> >>>> Si / Will, >>>> >>>> That is just FYI, if anyone uses VMware with that flag set to false. >>>> I'm neither against the feature nor telling to rip that out. >>>> >>>> You are correct, the PR 2081 supports KVM and Xen as the volume >>>> snapshots are directly supported on them and backup operation is not >>>> tightly >>> coupled >>>> with the create operation. >>>> >>>> -Suresh >>>> >>>> On Fri, May 18, 2018 at 7:38 PM, Simon Weller >>>> <swel...@ena.com.invalid> >>>> wrote: >>>> >>>>> There are plenty of features in ACS that are particular to a >>>>> certain hypervisor (or hypervisor set), including VMware specific items. >>>>> >>>>> It was never claimed this feature worked across all hypervisors. >>>>> In addition to that, the default was to leave the existing >>>>> functionality exactly the way it was originally implemented and if >>>>> a user wished to change the functionality they could via a global config >>>>> variable. >>>>> >>>>> Your original spec for PR 2081 in confluence states that the PR >>>>> was targeted towards KVM and Xen, so I'm confused as to why VMware >>>>> is even being mentioned here. >>>>> >>>>> >>>>> This is a major feature regression that a number of >>> organizations/service >>>>> providers are relying on and it wasn't called out when the PR was >>>> submitted. >>>>> >>>>> >>>>> ________________________________ >>>>> From: Will Stevens <wstev...@cloudops.com> >>>>> Sent: Friday, May 18, 2018 6:12 AM >>>>> To: dev@cloudstack.apache.org >>>>> Subject: Re: Snapshots only on Primary Storage feature >>>>> >>>>> Just because it does not work for VMware should not a reason to >>>>> rip out >>>> the >>>>> functionality for other hypervisors where it is being used though. >>>>> >>>>> I know we also have the requirement that snapshots are not >>> automatically >>>>> replicated to secondary storage, so this feature is useful to us. >>>>> >>>>> I don't understand the rational for removing the feature just >>>>> because >>> it >>>>> does not work on VMware. >>>>> >>>>> On Fri, May 18, 2018, 6:27 AM Suresh Kumar Anaparti, < >>>>> sureshkumar.anapa...@gmail.com> wrote: >>>>> >>>>>> Si, >>>>>> >>>>>> The PR# 1697 with the global setting >>>>>> *snapshot.backup.rightafter** - >>>>>> false* doesn't >>>>>> work for VMware as create snapshot never takes a snapshot in >>>>>> Primary >>>>> pool, >>>>>> it just returns the snapshot uuid. The backup snapshot does the >>>> complete >>>>>> job - creates a VM snapshot with the uuid, extracts and exports >>>>>> the >>>>> target >>>>>> volume to secondary. On demand backup snapshot doesn't work as >>>>>> there >>> is >>>>> no >>>>>> snapshot in primary. Also, there'll be only one entry with >>>>>> Primary >>>> store >>>>>> role in snapshot_store_ref, which is the latest snapshot taken >>>>>> for >>> that >>>>>> volume. >>>>>> >>>>>> -Suresh >>>>>> >>>>>> On Fri, May 18, 2018 at 1:03 AM, Simon Weller >>> <swel...@ena.com.invalid >>>>> >>>>>> wrote: >>>>>> >>>>>>> The whole point of the original PR was to optionally disable >>>>>>> this functionality. >>>>>>> >>>>>>> We don't expose views of the backup state to our customers (we >>>>>>> have >>>> our >>>>>>> own customer interfaces) and it's a large waste of space for >>>>>>> us to >>> be >>>>>>> backing up tons of VM images when we have a solid primary >>>>>>> storage infrastructure that already has lots of resiliency. >>>>>>> >>>>>>> >>>>>>> I guess we're going to have to revisit this again before we >>>>>>> can >>>>> consider >>>>>>> rebasing on 4.11. >>>>>>> >>>>>>> ________________________________ >>>>>>> From: Suresh Kumar Anaparti <sureshkumar.anapa...@gmail.com> >>>>>>> Sent: Thursday, May 17, 2018 2:21 PM >>>>>>> To: dev >>>>>>> Subject: Re: Snapshots only on Primary Storage feature >>>>>>> >>>>>>> Hi Si, >>>>>>> >>>>>>> No. not possible to disable the backup to secondary. It copies >>>>>>> the >>>>> volume >>>>>>> snapshot to secondary in a background thread using asyncBackup >>> param >>>>> (set >>>>>>> to true) and allows other operations during that time. >>>>>>> >>>>>>> I understand that the backup was on demand when any operations >>>>>>> are performed on the snapshot. But, backup during that time >>>>>>> may take considerable time (depending on the snapshot size and >>>>>>> the network bandwidth), which can result in the job timeout >>>>>>> and the User may >>>> assume >>>>>>> that it is already Backed up based on its state, unless it is >>>>> documented. >>>>>>> >>>>>>> -Suresh >>>>>>> >>>>>>> On Fri, May 18, 2018 at 12:23 AM, Simon Weller >>>> <swel...@ena.com.invalid >>>>>> >>>>>>> wrote: >>>>>>> >>>>>>>> Suresh, >>>>>>>> >>>>>>>> >>>>>>>> With this new merged PR, is it possible to disable the >>>>>>>> backup to secondary completely? I can't tell from the >>>>>>>> reference spec and >>>> we're >>>>>> not >>>>>>> on >>>>>>>> a 4.10/4.11 base yet. >>>>>>>> >>>>>>>> For the record, in the instances where a volume or template >>>>>>>> from >>>>>> snapshot >>>>>>>> was required, the backup image was copied on demand to secondary. >>>>>>>> >>>>>>>> In an ideal world, secondary storage wouldn't even be >>>>>>>> involved in >>>>> most >>>>>> of >>>>>>>> these options, instead using the native clone features of >>>>>>>> the >>>>>> underlying >>>>>>>> storage. >>>>>>>> >>>>>>>> >>>>>>>> - Si >>>>>>>> >>>>>>>> ________________________________ >>>>>>>> From: Suresh Kumar Anaparti <sureshkumar.anapa...@gmail.com> >>>>>>>> Sent: Thursday, May 17, 2018 1:37 PM >>>>>>>> To: dev@cloudstack.apache.org >>>>>>>> Cc: Nathan Johnson >>>>>>>> Subject: Re: Snapshots only on Primary Storage feature >>>>>>>> >>>>>>>> Hi Glen / Si, >>>>>>>> >>>>>>>> In PR# 1697, the global setting *snapshot.backup.rightafter* >>>>>>>> if >>> set >>>>> to >>>>>>>> true, it'll be the default behaviour and snapshot is copied >>>>>>>> to >>> the >>>>>>>> secondary storage. If set to false, then the snapshot state >>>>> transitions >>>>>>> are >>>>>>>> mocked and Snapshot would be in BackedUp state even though >>>>>>>> it is >>>> not >>>>>>> really >>>>>>>> in Secondary storage, which doesn't make sense. Also, that >>>>>>>> will >>>>> enable >>>>>> to >>>>>>>> create a volume or template from the snapshot, which will >>> obviously >>>>>> fail. >>>>>>>> >>>>>>>> This behavior was changed with the PR >>>>>>>> https://github.com/apache/cloudstack/pull/2081. There is a >>>>>>>> clear separation of create and backup volume snapshot >>>>>>>> operations. The global >>> setting >>>>>>>> *snapshot.backup.rightafter* has been removed in PR# 2081. >>>>>>>> >>>>>>>> -Suresh >>>>>>>> >>>>>>>> On Thu, May 17, 2018 at 8:40 PM, Simon Weller >>>>> <swel...@ena.com.invalid >>>>>>> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Glen, >>>>>>>>> >>>>>>>>> >>>>>>>>> This feature was implemented in 4.9 by my colleague Nathan >>>> Johnson. >>>>>>> You >>>>>>>>> enable it by changing the global setting >>>>> snapshot.backup.rightafter >>>>>> to >>>>>>>>> false. >>>>>>>>> >>>>>>>>> >>>>>>>>> The PR is reference here: https://github.com/apache/ >>>>>>> cloudstack/pull/1697 >>>>>>>>> >>>>>>>>> >>>>>>>>> We have the exact same use case as you, as we also use Ceph. >>>>>>>>> >>>>>>>>> >>>>>>>>> - Si >>>>>>>>> >>>>>>>>> >>>>>>>>> ________________________________ >>>>>>>>> From: Glen Baars <g...@onsitecomputers.com.au> >>>>>>>>> Sent: Thursday, May 17, 2018 9:46 AM >>>>>>>>> To: dev@cloudstack.apache.org >>>>>>>>> Subject: Snapshots only on Primary Storage feature >>>>>>>>> >>>>>>>>> >>>>>>>>> Hello Devs, >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> I have been thinking about a feature request and want to >>>>>>>>> see >>> what >>>>>>> people >>>>>>>>> think about the use case. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> We use KVM + Ceph RBD as storage. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Currently, when a client takes a snapshot, Cloudstack >>>>>>>>> takes a >>>> Ceph >>>>>>>>> snapshot and then uses qemu-img to export to secondary storage. >>>>> This >>>>>>>>> creates a full backup of the server. Clients want to use >>>>>>>>> this >>> as >>>> a >>>>>>> daily >>>>>>>>> snapshot and it isn’t feasible due to the space requirements. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> We would like create the snapshot only on primary storage. >>>>>>>>> It >>> is >>>>>>>>> replicated offsite and fault tolerant. I can see that the >>>> download >>>>>>>> snapshot >>>>>>>>> and create template features may be an issue. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> I have seen the below features in the recent releases and >>>> wondered >>>>> if >>>>>>>> this >>>>>>>>> was the direction that the development was going. >>>>>>>>> >>>>>>>>> Separation of volume snapshot creation on primary storage >>>>>>>>> and >>>>> backing >>>>>>>>> operation on secondary storage. >>>>>>>>> >>>>>>>>> Bypass secondary storage template copy/transfer for KVM. >>>>>>>>> >>>>>>>>> Kind regards, >>>>>>>>> >>>>>>>>> Glen Baars >>>>>>>>> >>>>>>>>> BackOnline Manager >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> T 1300 733 328 / +61 8 6102 3276 >>>>>>>>> >>>>>>>>> NZ +64 9280 3561 >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> www.timg.com<http://www.timg.com/> >>>>>>>>> >>>>>>>>> [http://images.dbonline.com.au/images/fb.png] Facebook< >>>>>>>>> https://www.facebook.com/TheInformationManagementGroup> [ >>>>>>>>> http://images.dbonline.com.au/images/li.png] LinkedIn< >>>>>>>> http://www.linkedin. >>>>>>>>> com/company/the-information-management-group?trk=hb_tab_ >>>>>>>> compy_id_2724246> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Watch a short video about what we >>>>>>>>> do!<https://www.youtube.com/ watch?v=scLGLwSIFQc> >>>>>>>>> >>>>>>>>> [http://images.dbonline.com.au/images/timgv3.jpg]<https:// >>>>>>> goo.gl/eAHLO7> >>>>>>>>> >>>>>>>>> This e-mail may contain confidential and/or privileged >>>>> information.If >>>>>>> you >>>>>>>>> are not the intended recipient (or have received this >>>>>>>>> e-mail in >>>>>> error) >>>>>>>>> please notify the sender immediately and destroy this e-mail. >>> Any >>>>>>>>> unauthorized copying, disclosure or distribution of the >>> material >>>> in >>>>>>> this >>>>>>>>> e-mail is strictly forbidden. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> This e-mail is intended solely for the benefit of the >>>> addressee(s) >>>>>> and >>>>>>>> any >>>>>>>>> other named recipient. It is confidential and may contain >>> legally >>>>>>>>> privileged or confidential information. If you are not the >>>>> recipient, >>>>>>> any >>>>>>>>> use, distribution, disclosure or copying of this e-mail is >>>>>> prohibited. >>>>>>>> The >>>>>>>>> confidentiality and legal privilege attached to this >>>> communication >>>>> is >>>>>>> not >>>>>>>>> waived or lost by reason of the mistaken transmission or >>> delivery >>>>> to >>>>>>> you. >>>>>>>>> If you have received this e-mail in error, please notify >>>>>>>>> us >>>>>>> immediately. >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> This e-mail is intended solely for the benefit of the addressee(s) and any >> other named recipient. It is confidential and may contain legally privileged >> or confidential information. If you are not the recipient, any use, >> distribution, disclosure or copying of this e-mail is prohibited. The >> confidentiality and legal privilege attached to this communication is not >> waived or lost by reason of the mistaken transmission or delivery to you. If >> you have received this e-mail in error, please notify us immediately.
smime.p7s
Description: S/MIME cryptographic signature
