This is my cinder.conf [DEFAULT]
api_paste_config=/etc/cinder/api-paste.ini state_path = /var/lib/cinder glance_host=localhost glance_api_version=2 rootwrap_config=/etc/cinder/rootwrap.conf auth_strategy=keystone lock_path=/var/lock/cinder qpid_hostname=localhost qpid_port=5672 qpid_heartbeat=60 qpid_tcp_nodelay=true iscsi_ip_address=10.198.0.191 iscsi_helper=tgtadm volumes_dir = /var/lib/cinder/volumes volume_group=cinder-volumes rbd_pool=nova rbd_user=nova rbd_ceph_conf=/etc/ceph/ceph.conf rbd_secret_uuid=ae396956-cf03-483f-b356-2f447dc7a637 volume_driver=cinder.volume.drivers.rbd.RBDDriver sql_connection = mysql://cinder:notcinder@173.194.106.60/cinder my_ip = 10.198.0.191 rpc_backend = cinder.openstack.common.rpc.impl_qpid qpid_hostname = 10.198.0.191 notification_driver=cinder.openstack.common.notifier.rpc_notifier Edward Lezondra IT IMC Financial Markets | 233 South Wacker Drive # 4300 | Chicago, IL 60606 | http://www.imc-chicago.com Phone: +13122755425 | E-Mail: edward.lezon...@imc-chicago.com -----Original Message----- From: Haomai Wang [mailto:haomaiw...@gmail.com] Sent: Monday, December 09, 2013 10:03 PM To: Edward Lezondra Cc: Openstack@lists.openstack.org Subject: Re: [Openstack] Ceph bootable cinder volume not working in Havana Could you give out your cinder.conf? On Tue, Dec 10, 2013 at 4:10 AM, Edward Lezondra <edward.lezon...@imc-chicago.com> wrote: > Hello, > > > > I’m having trouble getting a bootable cinder volume to work in Havana > using Ceph. I managed to get glance working correctly. I got cinder > volume to work, but once I try to create bootable cinder volume to an > image it errors out. > > My cinder.conf reflect exactly what the ceph docs tell one to do to > get it working. I was able to get this to work in grizzly using the > CEPH_ARGS="--id volumes" statement on the volume service, but the docs > and as a thread in the mailing list who got his to work didn’t use the > CEPH_ARGS="--id volumes" anymore for Havana. Judging from the logs I have > permission issue on the rbd image. I made sure my keyrings have the > proper ownership and gave full permissions to everyone to them. > > My logs on cinder/volume shows the following: > > > > 2013-12-09 12:29:12.377 8468 ERROR cinder.volume.drivers.rbd > [req-75308325-0b69-41f9-8099-1c9d64c025b6 > ce3265c65e304ad3b018a9fba063df88 0c3cbff39d97448d8cff1816793aea94] > error opening rbd image > 9e967bd7-ed6c-41bc-932d-1157bd72b778 > > 2013-12-09 12:29:12.377 8468 TRACE cinder.volume.drivers.rbd Traceback > (most recent call last): > > 2013-12-09 12:29:12.377 8468 TRACE cinder.volume.drivers.rbd File > "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line > 209, in __init__ > > 2013-12-09 12:29:12.377 8468 TRACE cinder.volume.drivers.rbd > read_only=read_only) > > 2013-12-09 12:29:12.377 8468 TRACE cinder.volume.drivers.rbd File > "/usr/lib/python2.7/site-packages/rbd.py", line 351, in __init__ > > 2013-12-09 12:29:12.377 8468 TRACE cinder.volume.drivers.rbd raise > make_ex(ret, 'error opening image %s at snapshot %s' % (name, > snapshot)) > > 2013-12-09 12:29:12.377 8468 TRACE cinder.volume.drivers.rbd > PermissionError: error opening image > 9e967bd7-ed6c-41bc-932d-1157bd72b778 at snapshot snap > > 2013-12-09 12:29:12.377 8468 TRACE cinder.volume.drivers.rbd > > 2013-12-09 12:29:12.380 8468 WARNING cinder.taskflow.utils [-] > Activating 5 rollbacks due to <cinder.taskflow.utils.FlowFailure object at > 0x328c690>. > > 2013-12-09 12:29:12.485 8468 ERROR cinder.volume.flows.create_volume > [req-75308325-0b69-41f9-8099-1c9d64c025b6 > ce3265c65e304ad3b018a9fba063df88 0c3cbff39d97448d8cff1816793aea94] Unexpected > build error: > > 2013-12-09 12:29:12.485 8468 TRACE cinder.volume.flows.create_volume > Traceback (most recent call last): > > 2013-12-09 12:29:12.485 8468 TRACE cinder.volume.flows.create_volume File > "/usr/lib/python2.7/site-packages/cinder/taskflow/patterns/linear_flow > .py", > line 172, in run_it > > 2013-12-09 12:29:12.485 8468 TRACE cinder.volume.flows.create_volume > result = runner(context, *args, **kwargs) > > 2013-12-09 12:29:12.485 8468 TRACE cinder.volume.flows.create_volume File > "/usr/lib/python2.7/site-packages/cinder/taskflow/utils.py", line 260, > in __call__ > > 2013-12-09 12:29:12.485 8468 TRACE cinder.volume.flows.create_volume > self.result = self.task(*args, **kwargs) > > 2013-12-09 12:29:12.485 8468 TRACE cinder.volume.flows.create_volume File > "/usr/lib/python2.7/site-packages/cinder/volume/flows/create_volume/__ > init__.py", > line 1499, in __call__ > > 2013-12-09 12:29:12.485 8468 TRACE cinder.volume.flows.create_volume > **volume_spec) > > 2013-12-09 12:29:12.485 8468 TRACE cinder.volume.flows.create_volume File > "/usr/lib/python2.7/site-packages/cinder/volume/flows/create_volume/__ > init__.py", > line 1441, in _create_from_image > > 2013-12-09 12:29:12.485 8468 TRACE cinder.volume.flows.create_volume > volume_ref, image_location, image_id) > > 2013-12-09 12:29:12.485 8468 TRACE cinder.volume.flows.create_volume File > "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line > 729, in clone_image > > 2013-12-09 12:29:12.485 8468 TRACE cinder.volume.flows.create_volume if > image_location is None or not self._is_cloneable(image_location): > > 2013-12-09 12:29:12.485 8468 TRACE cinder.volume.flows.create_volume File > "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line > 720, in _is_cloneable > > 2013-12-09 12:29:12.485 8468 TRACE cinder.volume.flows.create_volume > read_only=True): > > 2013-12-09 12:29:12.485 8468 TRACE cinder.volume.flows.create_volume File > "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line > 213, in __init__ > > 2013-12-09 12:29:12.485 8468 TRACE cinder.volume.flows.create_volume > raise > > 2013-12-09 12:29:12.485 8468 TRACE cinder.volume.flows.create_volume > TypeError: exceptions must be old-style classes or derived from > BaseException, not NoneType > > 2013-12-09 12:29:12.485 8468 TRACE cinder.volume.flows.create_volume > > 2013-12-09 12:29:12.486 8468 ERROR cinder.openstack.common.rpc.amqp > [req-75308325-0b69-41f9-8099-1c9d64c025b6 > ce3265c65e304ad3b018a9fba063df88 0c3cbff39d97448d8cff1816793aea94] > Exception during message handling > > 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp > Traceback (most recent call last): > > 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp File > "/usr/lib/python2.7/site-packages/cinder/openstack/common/rpc/amqp.py" > , line 441, in _process_data > > 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp > **args) > > 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp File > "/usr/lib/python2.7/site-packages/cinder/openstack/common/rpc/dispatch > er.py", > line 148, in dispatch > > 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp > return getattr(proxyobj, method)(ctxt, **kwargs) > > 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp File > "/usr/lib/python2.7/site-packages/cinder/utils.py", line 808, in > wrapper > > 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp > return func(self, *args, **kwargs) > > 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp File > "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 257, > in create_volume > > 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp > flow.run(context.elevated()) > > 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp File > "/usr/lib/python2.7/site-packages/cinder/taskflow/decorators.py", line > 105, in wrapper > > 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp > return f(self, *args, **kwargs) > > 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp File > "/usr/lib/python2.7/site-packages/cinder/taskflow/patterns/linear_flow > .py", > line 232, in run > > 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp > run_it(r) > > 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp File > "/usr/lib/python2.7/site-packages/cinder/taskflow/patterns/linear_flow > .py", > line 212, in run_it > > 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp > self.rollback(context, cause) > > 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp File > "/usr/lib64/python2.7/contextlib.py", line 24, in __exit__ > > 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp > self.gen.next() > > 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp File > "/usr/lib/python2.7/site-packages/cinder/taskflow/patterns/linear_flow > .py", > line 172, in run_it > > 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp > result = runner(context, *args, **kwargs) > > 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp File > "/usr/lib/python2.7/site-packages/cinder/taskflow/utils.py", line 260, > in __call__ > > 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp > self.result = self.task(*args, **kwargs) > > 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp File > "/usr/lib/python2.7/site-packages/cinder/volume/flows/create_volume/__ > init__.py", > line 1499, in __call__ > > 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp > **volume_spec) > > 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp File > "/usr/lib/python2.7/site-packages/cinder/volume/flows/create_volume/__ > init__.py", > line 1441, in _create_from_image > > 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp > volume_ref, image_location, image_id) > > 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp File > "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line > 729, in clone_image > > 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp if > image_location is None or not self._is_cloneable(image_location): > > 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp File > "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line > 720, in _is_cloneable > > 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp > read_only=True): > > 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp File > "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line > 213, in __init__ > > 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp > raise > > 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp > TypeError: exceptions must be old-style classes or derived from > BaseException, not NoneType > > > > > ________________________________ > > The information in this e-mail is intended only for the person or > entity to which it is addressed. > > It may contain confidential and /or privileged material. If someone > other than the intended recipient should receive this e-mail, he / she > shall not be entitled to read, disseminate, disclose or duplicate it. > > If you receive this e-mail unintentionally, please inform us > immediately by "reply" and then delete it from your system. Although > this information has been compiled with great care, neither IMC > Financial Markets & Asset Management nor any of its related entities > shall accept any responsibility for any errors, omissions or other > inaccuracies in this information or for the consequences thereof, nor > shall it be bound in any way by the contents of this e-mail or its > attachments. In the event of incomplete or incorrect transmission, > please return the e-mail to the sender and permanently delete this message > and any attachments. > > Messages and attachments are scanned for all known viruses. Always > scan attachments before opening them. > > _______________________________________________ > Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack > Post to : openstack@lists.openstack.org > Unsubscribe : > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack > -- Best Regards, Wheat ________________________________ The information in this e-mail is intended only for the person or entity to which it is addressed. It may contain confidential and /or privileged material. If someone other than the intended recipient should receive this e-mail, he / she shall not be entitled to read, disseminate, disclose or duplicate it. If you receive this e-mail unintentionally, please inform us immediately by "reply" and then delete it from your system. Although this information has been compiled with great care, neither IMC Financial Markets & Asset Management nor any of its related entities shall accept any responsibility for any errors, omissions or other inaccuracies in this information or for the consequences thereof, nor shall it be bound in any way by the contents of this e-mail or its attachments. In the event of incomplete or incorrect transmission, please return the e-mail to the sender and permanently delete this message and any attachments. Messages and attachments are scanned for all known viruses. Always scan attachments before opening them. _______________________________________________ Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : openstack@lists.openstack.org Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
