Have you enabled live snapshots in nova.conf?
The default for this option is "true", so you should check that:
disable_libvirt_livesnapshot = false
Is it really a live snaphot? What's in the nova-compute.log? It should
say something like
[instance: XXX] Beginning live snapshot process
Regards,
Eugen
Zitat von John Petrini <jpetr...@coredial.com>:
Hi All,
Following up after making this change. Adding write permissions to the
images pool in Ceph did the trick and RBD snapshots now work. However the
instance is still paused for the duration of the snapshot. Is it possible
to do a live snapshot without pausing the instance?
Thanks,
John
On Fri, Jan 13, 2017 at 5:49 AM, Eugen Block <ebl...@nde.ag> wrote:
Thanks,
for anyone interested in this issue, I filed a bug report:
https://bugs.launchpad.net/nova/+bug/1656242
Regards,
Eugen
Zitat von Mohammed Naser <mna...@vexxhost.com>:
It is likely because this has been tested with QEMU only. I think you
might want to bring this up with the Nova team.
Sent from my iPhone
On Jan 12, 2017, at 11:28 AM, Eugen Block <ebl...@nde.ag> wrote:
I'm not sure if this is the right spot, but I added some log statements
into driver.py.
First, there's this if-block:
if (self._host.has_min_version(MIN_LIBVIRT_LIVESNAPSHOT_VERSION,
MIN_QEMU_LIVESNAPSHOT_VERSION,
host.HV_DRIVER_QEMU)
and source_type not in ('lvm')
and not CONF.ephemeral_storage_encryption.enabled
and not CONF.workarounds.disable_libvirt_livesnapshot):
live_snapshot = True
[...]
else:
live_snapshot = False
And I know that it lands in the else-statement. Turns out that
_host.has_min_version is "false", because of host.HV_DRIVER_QEMU. We are
running on Xen hypervisors. So I tried it with host.HV_DRIVER_XEN and now
nova-compute says:
[instance: 14b75237-7619-481f-9636-792b64d1be17] instance snapshotting
[instance: 14b75237-7619-481f-9636-792b64d1be17] Beginning live
snapshot process
Now I'm waiting for the result, but at least the VM is still running, so
it looks quite promising...
And there it is:
[instance: 14b75237-7619-481f-9636-792b64d1be17] Snapshot image upload
complete
I'm testing the image now, and it works!
Now the question is, why is it defaulting to HV_DRIVER_QEMU and is it
really necessary to change this directly in the code? Is there any other
way?
Regards,
Eugen
Zitat von Eugen Block <ebl...@nde.ag>:
Yes, I truncated the file and uploaded it:
http://dropcanvas.com/ta7nu
(First time I used this service, please give me feedback if this
doesn't work for you)
I see the "Beginning cold snapshot process" message, but I don't know
why. Any help is appreciated!
Regards,
Eugen
Zitat von Mohammed Naser <mna...@vexxhost.com>:
Would you be able to share the logs of a full snapshot run with the
compute node in debug?
Sent from my iPhone
On Jan 12, 2017, at 7:47 AM, Eugen Block <ebl...@nde.ag> wrote:
That's strange, I also searched for this message, but nothing there.
I have debug logs enabled on compute node but I don't see anything
regarding ceph. No matter, what I do, my instance is always
shutdown before
a snapshot is taken. What else can I try?
Zitat von John Petrini <jpetr...@coredial.com>:
Mohammed,
It looks like you may be right. Just found the permissions issue in
the
nova log on the compute node.
4-e8f52e4fbcfb 691caf1c10354efab3e3c8ed61b7d89a
49bc5e5bf2684bd0948d9f94c7875027 - - -] Performing standard snapshot
because direct snapshot failed: no write permission on storage pool
images
I'm going to test the change and will send an update you all with the
results.
Thank You,
___
John Petrini
Yes, we are also running Mitaka and I also read Sebastien Han's
blogs ;-)
our snapshots are not happening at the RBD level,
they are being copied and uploaded to glance which takes up a lot
of space
and is very slow.
Unfortunately, that's what we are experiencing, too. I don't know if
there's something I missed in the nova configs or somewhere else,
but I'm
relieved that I'm not the only one :-)
While writing this email I searched again and found something:
https://specs.openstack.org/openstack/nova-specs/specs/mitak
a/implemented/rbd-instance-snapshots.html
https://review.openstack.org/#/c/205282/
It seems to be implemented already, I'm looking for the config
options to
set. If you manage to get nova to make rbd snapshots, please let me
know ;-)
Regards,
Eugen
Zitat von John Petrini <jpetr...@coredial.com>:
Hi Eugen,
Thanks for the response! That makes a lost of sense and is what I
figured
was going on but I missed it in the documentation. We use Ceph as
well and
I had considered doing the snapshots at the RBD level but I was
hoping
there was someway to accomplish this via nova. I came across this
Sebastien
Han write-up that claims this functionality was added to Mitaka:
http://www.sebastien-han.fr/blog/2015/10/05/openstack-nova-
snapshots-on-ceph-rbd/
We are running Mitaka but our snapshots are not happening at the
RBD
level,
they are being copied and uploaded to glance which takes up a lot
of space
and is very slow.
Have you or anyone else implemented this in Mitaka? Other than
Sebastian's
blog I haven't found any documentation on this.
Thank You,
___
John Petrini
On Wed, Jan 11, 2017 at 3:32 AM, Eugen Block <ebl...@nde.ag>
wrote:
Hi,
this seems to be exptected, the docs say:
"Shut down the source VM before you take the snapshot to ensure
that all
data is flushed to disk."
So if the VM is not shut down, it's freezed to prevent data loss
(I
guess). Depending on your storage backend, there are other ways to
perform
backups of your VMs.
We use Ceph as backend for nova, glance and cinder. Ceph stores
the
disks,
images and volumes as Rados block device objects. We have a
backup script
that creates snapshots of these RBDs, which are exported to our
backup
drive. This way the running VM is not stopped or freezed, the user
doesn't
notice any issues. Unlike a nova snapshot, the rbd snapshot is
created
immediately within a few seconds. After a successful backup the
snapshots
are removed.
Hope this helps! If you are interested in Ceph, visit [1].
Regards,
Eugen
[1] http://docs.ceph.com/docs/giant/start/intro/
Zitat von John Petrini <jpetr...@coredial.com>:
Hello,
I've just started experimenting with nova backup and discovered
that
there
is a period of time during the snapshot where the instance
becomes
unreachable. Is this behavior expected during a live snapshot?
Is there
any
way to prevent this?
___
John Petrini
--
Eugen Block voice : +49-40-559 51
75
NDE Netzdesign und -entwicklung AG fax : +49-40-559 51
77
Postfach 61 03 15
D-22423 Hamburg e-mail : ebl...@nde.ag
Vorsitzende des Aufsichtsrates: Angelika Mozdzen
Sitz und Registergericht: Hamburg, HRB 90934
Vorstand: Jens-U. Mozdzen
USt-IdNr. DE 814 013 983
_______________________________________________
Mailing list: http://lists.openstack.org/cgi
-bin/mailman/listinfo/openstac
k
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi
-bin/mailman/listinfo/openstac
k
--
Eugen Block voice : +49-40-559 51 75
NDE Netzdesign und -entwicklung AG fax : +49-40-559 51 77
Postfach 61 03 15
D-22423 Hamburg e-mail : ebl...@nde.ag
Vorsitzende des Aufsichtsrates: Angelika Mozdzen
Sitz und Registergericht: Hamburg, HRB 90934
Vorstand: Jens-U. Mozdzen
USt-IdNr. DE 814 013 983
--
Eugen Block voice : +49-40-559 51 75
NDE Netzdesign und -entwicklung AG fax : +49-40-559 51 77
Postfach 61 03 15
D-22423 Hamburg e-mail : ebl...@nde.ag
Vorsitzende des Aufsichtsrates: Angelika Mozdzen
Sitz und Registergericht: Hamburg, HRB 90934
Vorstand: Jens-U. Mozdzen
USt-IdNr. DE 814 013 983
_______________________________________________
Mailing list: http://lists.openstack.org/cgi
-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi
-bin/mailman/listinfo/openstack
--
Eugen Block voice : +49-40-559 51 75
NDE Netzdesign und -entwicklung AG fax : +49-40-559 51 77
Postfach 61 03 15
D-22423 Hamburg e-mail : ebl...@nde.ag
Vorsitzende des Aufsichtsrates: Angelika Mozdzen
Sitz und Registergericht: Hamburg, HRB 90934
Vorstand: Jens-U. Mozdzen
USt-IdNr. DE 814 013 983
--
Eugen Block voice : +49-40-559 51 75
NDE Netzdesign und -entwicklung AG fax : +49-40-559 51 77
Postfach 61 03 15
D-22423 Hamburg e-mail : ebl...@nde.ag
Vorsitzende des Aufsichtsrates: Angelika Mozdzen
Sitz und Registergericht: Hamburg, HRB 90934
Vorstand: Jens-U. Mozdzen
USt-IdNr. DE 814 013 983
--
Eugen Block voice : +49-40-559 51 75
NDE Netzdesign und -entwicklung AG fax : +49-40-559 51 77
Postfach 61 03 15
D-22423 Hamburg e-mail : ebl...@nde.ag
Vorsitzende des Aufsichtsrates: Angelika Mozdzen
Sitz und Registergericht: Hamburg, HRB 90934
Vorstand: Jens-U. Mozdzen
USt-IdNr. DE 814 013 983
--
Eugen Block voice : +49-40-559 51 75
NDE Netzdesign und -entwicklung AG fax : +49-40-559 51 77
Postfach 61 03 15
D-22423 Hamburg e-mail : ebl...@nde.ag
Vorsitzende des Aufsichtsrates: Angelika Mozdzen
Sitz und Registergericht: Hamburg, HRB 90934
Vorstand: Jens-U. Mozdzen
USt-IdNr. DE 814 013 983
_______________________________________________
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
