Josh, Thanks so much for the pointers. I did not figure out that libvirt was trying to read /etc/ceph/ceph.conf. It was in fact being denied by apparmor. I fixed my problem by editing /etc/apparmor.d/libvirt/TEMPLATE and adding "/etc/ceph/** r," to the template. Now it works for newly spawned VMs.
Again, thanks a lot. - Travis On Fri, Jul 13, 2012 at 7:40 PM, Josh Durgin <josh.dur...@inktank.com>wrote: > On 07/13/2012 03:09 PM, Travis Rhoden wrote: > >> Was this problem ever resolved? I'm having the identical issue right now >> (perhaps because I am following Sebastien's guide [0]). I found this >> question raised on LP [1], but no helpful responses, and no linked bug. I >> can confirm that I can attach an RBD instance manually through virsh, no >> problem. So it is not a libvirt thing. It sure appears to be a faulty >> disk >> attachment definition from OpenStack. >> > > I didn't notice this launchpad question before, but I'll copy my answer > below for the mailing list archives: > > Right now the RBD driver in OpenStack requires you to have your monitor > addresses in /etc/ceph/ceph.conf. When you have them there, putting them in > the libvirt xml is unnecessary. You do have to make sure that the unix user > who ends up running qemu/kvm has read access to /etc/ceph/ceph.conf, and > e.g. apparmor isn't preventing qemu/kvm from reading it. > > This dependency on /etc/ceph/ceph.conf should be removed in the future. > > > I'm running the latest packages on Ubuntu 12.04 for OpensStack (essex) and >> Ceph (argonaut). >> >> I can confirm when using nova volume-attach it does not matter what device >> you pick, dev/vdb, dev/vdf, whatever. Always fails. /dev/vdb manually >> works great. >> > > If the answer above doesn't help, what error message do you get? > Is there any info in the libvirt logs for the instance? > > Josh > > [0] >> http://www.sebastien-han.fr/**blog/2012/06/10/introducing-** >> ceph-to-openstack/<http://www.sebastien-han.fr/blog/2012/06/10/introducing-ceph-to-openstack/> >> [1] >> https://answers.launchpad.net/**nova/+question/201366<https://answers.launchpad.net/nova/+question/201366> >> >> - Travis >> >> On Fri, May 25, 2012 at 8:32 PM, Josh Durgin <josh.dur...@inktank.com>** >> wrote: >> >> On 05/25/2012 01:31 AM, Sébastien Han wrote: >>> >>> Hi everyone, >>>> >>>> I setup a ceph cluster and I use the RBD driver for nova-volume. >>>> I can create volumes and snapshots but currently I can't attach them to >>>> an instance. >>>> Apparently the volume is detected as busy but it's not, no matter which >>>> name I choose. >>>> I tried from horizon and the command line, same issue. >>>> >>>> $ nova --debug volume-attach de3743be-69be-4ec3-8de3-****fc7df6903a97 >>>> 12 >>>> >>>> /dev/vdc >>>> connect: (localhost, 5000) >>>> send: 'POST /v2.0/tokens HTTP/1.1\r\nHost: >>>> localhost:5000\r\nContent-****Length: 100\r\ncontent-type: >>>> application/json\r\naccept-****encoding: gzip, deflate\r\naccept: >>>> application/json\r\nuser-****agent: python-novaclient\r\n\r\n{"**** >>>> auth": >>>> >>>> {"tenantName": "admin", "passwordCredentials": {"username": "admin", >>>> "password": "admin"}}}' >>>> reply: 'HTTP/1.1 200 OK\r\n' >>>> header: Content-Type: application/json >>>> header: Vary: X-Auth-Token >>>> header: Date: Fri, 25 May 2012 08:29:46 GMT >>>> header: Transfer-Encoding: chunked >>>> connect: (172.17.1.6, 8774) >>>> send: u'GET >>>> /v2/****d1f5d27ccf594cdbb034c8a4123494****e9/servers/de3743be-69be-** >>>> 4ec3-** >>>> >>>> 8de3-fc7df6903a97 >>>> HTTP/1.1\r\nHost: 172.17.1.6:8774 >>>> <http://172.17.1.6:8774>\r\nx-****auth-project-id: >>>> admin\r\nx-auth-token: >>>> >>>> 37ff6354b9114178889128175494c6****66\r\naccept-encoding: gzip, >>>> deflate\r\naccept: application/json\r\nuser-****agent: >>>> >>>> python-novaclient\r\n\r\n' >>>> reply: 'HTTP/1.1 200 OK\r\n' >>>> header: X-Compute-Request-Id: req-fc7686dd-7dff-4ea2-b2dd-**** >>>> be584dadb7c1 >>>> >>>> header: Content-Type: application/json >>>> header: Content-Length: 1371 >>>> header: Date: Fri, 25 May 2012 08:29:46 GMT >>>> send: u'POST >>>> /v2/****d1f5d27ccf594cdbb034c8a4123494****e9/servers/de3743be-69be-** >>>> 4ec3-** >>>> 8de3-fc7df6903a97/os-volume_****attachments >>>> >>>> HTTP/1.1\r\nHost: 172.17.1.6:8774 >>>> <http://172.17.1.6:8774>\r\****nContent-Length: >>>> 60\r\nx-auth-project-id: >>>> >>>> >>>> admin\r\naccept-encoding: gzip, deflate\r\naccept: >>>> application/json\r\nx-auth-****token: >>>> 37ff6354b9114178889128175494c6****66\r\nuser-agent: >>>> python-novaclient\r\ncontent-****type: >>>> application/json\r\n\r\n{"****volumeAttachment": {"device": "/dev/vdc", >>>> >>>> "volumeId": 12}}' >>>> reply: 'HTTP/1.1 200 OK\r\n' >>>> header: X-Compute-Request-Id: req-4a22f603-4aad-4ae9-99a2-**** >>>> 6a7bcacf974f >>>> >>>> header: Content-Type: application/json >>>> header: Content-Length: 48 >>>> header: Date: Fri, 25 May 2012 08:29:46 GMT >>>> $ echo $? >>>> 0 >>>> >>>> See the logs here: >>>> http://paste.openstack.org/****show/18163/<http://paste.openstack.org/**show/18163/> >>>> <http://paste.**openstack.org/show/18163/<http://paste.openstack.org/show/18163/> >>>> > >>>> >>>> >>> Hi Seb, >>> >>> It looks like the DeviceIsBusy exception is raised when libvirt >>> returns failure for any reason when attaching a volume. Try turning on >>> debug logging for libvirt and attaching again, and posting the logs >>> from libvirt. >>> >>> You might need to upgrade libvirt - the version in Ubuntu 12.04 should >>> work, or upstream libvirt 0.9.12. >>> >>> Josh >>> >>> >>> If you need more information let me know :) >>> >>>> Thank you in advance. >>>> >>>> Cheers. >>>> >>>> ~Seb >>>> >>>> >>> ______________________________****_________________ >>> Mailing list: >>> https://launchpad.net/~****openstack<https://launchpad.net/%7E**openstack> >>> <https://launchpad.**net/%7Eopenstack<https://launchpad.net/%7Eopenstack> >>> > >>> Post to : openstack@lists.launchpad.net >>> Unsubscribe : >>> https://launchpad.net/~****openstack<https://launchpad.net/%7E**openstack> >>> <https://launchpad.**net/%7Eopenstack<https://launchpad.net/%7Eopenstack> >>> > >>> More help : >>> https://help.launchpad.net/****ListHelp<https://help.launchpad.net/**ListHelp> >>> <https://help.**launchpad.net/ListHelp<https://help.launchpad.net/ListHelp> >>> > >>> >>> >> > >
_______________________________________________ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
