I would suggest on nova compute host (particularly if you have separate compute
nodes),
(1) make sure "rbd ls -l -p " works and /etc/ceph/ceph.conf is readable by user
nova!!(2) make sure you can start up a regular ephemeral instance on the same
nova node (ie, nova-compute is working correctly)(3) if you are using cephx,
make sure libvirt secret is set up correct per instruction at ceph.com(4) look
at /var/lib/nova/instance/xxxxxxxxxxxxx/libvirt.xml and the disk file is
pointing to the rbd volume(5) If all above look fine and you still couldn't
perform nova boot with the volume, you can try last thing to manually start up
a kvm session with the volume similar to below. At least this will tell you if
you qemu has the correct rbd enablement.
/usr/bin/kvm -m 2048 -drive
file=rbd:ceph-openstack-volumes/volume-3f964f79-febe-4251-b2ba-ac9423af419f,index=0,if=none,id=drive-virtio-disk0
-boot c -net nic -net user -nographic -vnc :1000 -device
piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device
virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1
--weiguo
> Date: Thu, 30 May 2013 16:37:40 +0200
> From: mar...@tuxadero.com
> To: ceph-us...@ceph.com
> CC: openst...@lists.launchpad.net
> Subject: [ceph-users] Openstack with Ceph, boot from volume
>
> Hi Josh,
>
> I am trying to use ceph with openstack (grizzly), I have a multi host setup.
> I followed the instruction http://ceph.com/docs/master/rbd/rbd-openstack/.
> Glance is working without a problem.
> With cinder I can create and delete volumes without a problem.
>
> But I cannot boot from volumes.
> I doesn't matter if use horizon or the cli, the vm goes to the error state.
>
> From the nova-compute.log I get this.
>
> 2013-05-30 16:08:45.224 ERROR nova.compute.manager
> [req-5679ddfe-79e3-4adb-b220-915f4a38b532
> 8f9630095810427d865bc90c5ea04d35 43b2bbbf5daf4badb15d67d87ed2f3dc]
> [instance: 059589a3-72fc-444d-b1f0-ab1567c725fc] Instance failed block
> device setup
> .....
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc] ConnectionError: [Errno 101]
> ENETUNREACH
>
> What tries nova to reach? How could I debug that further?
>
> Full Log included.
>
> -martin
>
> Log:
>
> ceph --version
> ceph version 0.61 (237f3f1e8d8c3b85666529860285dcdffdeda4c5)
>
> root@compute1:~# dpkg -l|grep -e ceph-common -e cinder
> ii ceph-common 0.61-1precise
> common utilities to mount and interact with a ceph storage
> cluster
> ii python-cinderclient 1:1.0.3-0ubuntu1~cloud0
> python bindings to the OpenStack Volume API
>
>
> nova-compute.log
>
> 2013-05-30 16:08:45.224 ERROR nova.compute.manager
> [req-5679ddfe-79e3-4adb-b220-915f4a38b532
> 8f9630095810427d865bc90c5ea04d35 43b2bbbf5daf4badb15d67d87ed2f3dc]
> [instance: 059589a3-72fc-444d-b1f0-ab1567c725fc] Instance failed block
> device setup
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc] Traceback (most recent call last):
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc] File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1071,
> in _prep_block_device
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc] return
> self._setup_block_device_mapping(context, instance, bdms)
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc] File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 721, in
> _setup_block_device_mapping
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc] volume =
> self.volume_api.get(context, bdm['volume_id'])
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc] File
> "/usr/lib/python2.7/dist-packages/nova/volume/cinder.py", line 193, in get
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc]
> self._reraise_translated_volume_exception(volume_id)
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc] File
> "/usr/lib/python2.7/dist-packages/nova/volume/cinder.py", line 190, in get
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc] item =
> cinderclient(context).volumes.get(volume_id)
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc] File
> "/usr/lib/python2.7/dist-packages/cinderclient/v1/volumes.py", line 180,
> in get
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc] return self._get("/volumes/%s"
> % volume_id, "volume")
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc] File
> "/usr/lib/python2.7/dist-packages/cinderclient/base.py", line 141, in _get
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc] resp, body =
> self.api.client.get(url)
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc] File
> "/usr/lib/python2.7/dist-packages/cinderclient/client.py", line 185, in get
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc] return self._cs_request(url,
> 'GET', **kwargs)
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc] File
> "/usr/lib/python2.7/dist-packages/cinderclient/client.py", line 153, in
> _cs_request
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc] **kwargs)
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc] File
> "/usr/lib/python2.7/dist-packages/cinderclient/client.py", line 123, in
> request
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc] **kwargs)
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc] File
> "/usr/lib/python2.7/dist-packages/requests/api.py", line 44, in request
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc] return
> session.request(method=method, url=url, **kwargs)
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc] File
> "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 279, in
> request
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc] resp = self.send(prep,
> stream=stream, timeout=timeout, verify=verify, cert=cert, proxies=proxies)
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc] File
> "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 374, in send
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc] r = adapter.send(request,
> **kwargs)
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc] File
> "/usr/lib/python2.7/dist-packages/requests/adapters.py", line 206, in send
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc] raise ConnectionError(sockerr)
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc] ConnectionError: [Errno 101]
> ENETUNREACH
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc]
> 2013-05-30 16:08:45.329 AUDIT nova.compute.manager
> [req-5679ddfe-79e3-4adb-b220-915f4a38b532
> 8f9630095810427d865bc90c5ea04d35 43b2bbbf5daf4badb15d67d87ed2f3dc]
> [instance: 059589a3-72fc-444d-b1f0-ab1567c725fc] Terminating instance
> _______________________________________________
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
