Would you be able to share the logs of a full snapshot run with the compute node in debug?
Sent from my iPhone > On Jan 12, 2017, at 7:47 AM, Eugen Block <ebl...@nde.ag> wrote: > > That's strange, I also searched for this message, but nothing there. I have > debug logs enabled on compute node but I don't see anything regarding ceph. > No matter, what I do, my instance is always shutdown before a snapshot is > taken. What else can I try? > > > Zitat von John Petrini <jpetr...@coredial.com>: > >> Mohammed, >> >> It looks like you may be right. Just found the permissions issue in the >> nova log on the compute node. >> >> 4-e8f52e4fbcfb 691caf1c10354efab3e3c8ed61b7d89a >> 49bc5e5bf2684bd0948d9f94c7875027 - - -] Performing standard snapshot >> because direct snapshot failed: no write permission on storage pool images >> >> I'm going to test the change and will send an update you all with the >> results. >> >> Thank You, >> >> ___ >> >> John Petrini >> >> >> >>>> >>> Yes, we are also running Mitaka and I also read Sebastien Han's blogs ;-) >>> >>> our snapshots are not happening at the RBD level, >>>> they are being copied and uploaded to glance which takes up a lot of space >>>> and is very slow. >>>> >>> >>> Unfortunately, that's what we are experiencing, too. I don't know if >>> there's something I missed in the nova configs or somewhere else, but I'm >>> relieved that I'm not the only one :-) >>> >>> While writing this email I searched again and found something: >>> >>> https://specs.openstack.org/openstack/nova-specs/specs/mitak >>> a/implemented/rbd-instance-snapshots.html >>> >>> https://review.openstack.org/#/c/205282/ >>> >>> It seems to be implemented already, I'm looking for the config options to >>> set. If you manage to get nova to make rbd snapshots, please let me know ;-) >>> >>> Regards, >>> Eugen >>> >>> >>> >>> Zitat von John Petrini <jpetr...@coredial.com>: >>> >>> Hi Eugen, >>>> >>>> Thanks for the response! That makes a lost of sense and is what I figured >>>> was going on but I missed it in the documentation. We use Ceph as well and >>>> I had considered doing the snapshots at the RBD level but I was hoping >>>> there was someway to accomplish this via nova. I came across this >>>> Sebastien >>>> Han write-up that claims this functionality was added to Mitaka: >>>> http://www.sebastien-han.fr/blog/2015/10/05/openstack-nova- >>>> snapshots-on-ceph-rbd/ >>>> >>>> We are running Mitaka but our snapshots are not happening at the RBD >>>> level, >>>> they are being copied and uploaded to glance which takes up a lot of space >>>> and is very slow. >>>> >>>> Have you or anyone else implemented this in Mitaka? Other than Sebastian's >>>> blog I haven't found any documentation on this. >>>> >>>> Thank You, >>>> >>>> ___ >>>> >>>> John Petrini >>>> >>>> On Wed, Jan 11, 2017 at 3:32 AM, Eugen Block <ebl...@nde.ag> wrote: >>>> >>>> Hi, >>>>> >>>>> this seems to be exptected, the docs say: >>>>> >>>>> "Shut down the source VM before you take the snapshot to ensure that all >>>>> data is flushed to disk." >>>>> >>>>> So if the VM is not shut down, it's freezed to prevent data loss (I >>>>> guess). Depending on your storage backend, there are other ways to >>>>> perform >>>>> backups of your VMs. >>>>> We use Ceph as backend for nova, glance and cinder. Ceph stores the >>>>> disks, >>>>> images and volumes as Rados block device objects. We have a backup script >>>>> that creates snapshots of these RBDs, which are exported to our backup >>>>> drive. This way the running VM is not stopped or freezed, the user >>>>> doesn't >>>>> notice any issues. Unlike a nova snapshot, the rbd snapshot is created >>>>> immediately within a few seconds. After a successful backup the snapshots >>>>> are removed. >>>>> >>>>> Hope this helps! If you are interested in Ceph, visit [1]. >>>>> >>>>> Regards, >>>>> Eugen >>>>> >>>>> [1] http://docs.ceph.com/docs/giant/start/intro/ >>>>> >>>>> >>>>> Zitat von John Petrini <jpetr...@coredial.com>: >>>>> >>>>> >>>>> Hello, >>>>> >>>>>> >>>>>> I've just started experimenting with nova backup and discovered that >>>>>> there >>>>>> is a period of time during the snapshot where the instance becomes >>>>>> unreachable. Is this behavior expected during a live snapshot? Is there >>>>>> any >>>>>> way to prevent this? >>>>>> >>>>>> ___ >>>>>> >>>>>> John Petrini >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Eugen Block voice : +49-40-559 51 75 >>>>> NDE Netzdesign und -entwicklung AG fax : +49-40-559 51 77 >>>>> Postfach 61 03 15 >>>>> D-22423 Hamburg e-mail : ebl...@nde.ag >>>>> >>>>> Vorsitzende des Aufsichtsrates: Angelika Mozdzen >>>>> Sitz und Registergericht: Hamburg, HRB 90934 >>>>> Vorstand: Jens-U. Mozdzen >>>>> USt-IdNr. DE 814 013 983 >>>>> >>>>> >>>>> _______________________________________________ >>>>> Mailing list: http://lists.openstack.org/cgi >>>>> -bin/mailman/listinfo/openstac >>>>> k >>>>> Post to : openstack@lists.openstack.org >>>>> Unsubscribe : http://lists.openstack.org/cgi >>>>> -bin/mailman/listinfo/openstac >>>>> k >>>>> >>>>> >>> >>> >>> -- >>> Eugen Block voice : +49-40-559 51 75 >>> NDE Netzdesign und -entwicklung AG fax : +49-40-559 51 77 >>> Postfach 61 03 15 >>> D-22423 Hamburg e-mail : ebl...@nde.ag >>> >>> Vorsitzende des Aufsichtsrates: Angelika Mozdzen >>> Sitz und Registergericht: Hamburg, HRB 90934 >>> Vorstand: Jens-U. Mozdzen >>> USt-IdNr. DE 814 013 983 >>> >>> > > > > -- > Eugen Block voice : +49-40-559 51 75 > NDE Netzdesign und -entwicklung AG fax : +49-40-559 51 77 > Postfach 61 03 15 > D-22423 Hamburg e-mail : ebl...@nde.ag > > Vorsitzende des Aufsichtsrates: Angelika Mozdzen > Sitz und Registergericht: Hamburg, HRB 90934 > Vorstand: Jens-U. Mozdzen > USt-IdNr. DE 814 013 983 > > > _______________________________________________ > Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack > Post to : openstack@lists.openstack.org > Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack _______________________________________________ Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : openstack@lists.openstack.org Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
