Hi Evan, Thanks for your answer :)
Yes, I have defined similar permissions and users. After performing some tests, I realized that Glance was not able to read ceph.conf file because I did not have the right variables defined. Now, I can confirm that the Glance (Newton) integration with Ceph (Jewel) works following the steps defined on http://docs.ceph.com/docs/jewel/rbd/rbd-openstack/ and with the following configuration (at least for my use case and in a fresh installation on CentOS 7): [glance_store] stores = file,http,rbd default_store = rbd rbd_store_pool = images rbd_store_user = glance rbd_store_ceph_conf = /etc/ceph/ceph.conf rbd_store_chunk_size = 8 Currently, I am working on Cinder and Nova integration with Ceph. I will let you know when I manage to get it working. Cheers, Esteban El Lunes 6 de marzo de 2017 18:08, Evan Bollig PhD <boll0...@umn.edu> escribió: Hey Esteban, I got that same error at one point. Check your file permissions on the /etc/ceph directory and contents. In particular, make sure the glance user can access its keyring and the ceph.conf is readable for the group as well. Here's an example: drwxr-xr-x. 2 root root 4.0K Jan 17 12:25 . drwxr-xr-x. 93 root root 8.0K Mar 3 14:27 .. -rw-------. 1 root root 63 Jan 11 14:00 ceph.client.admin.keyring -rw-------. 1 cinder cinder 71 Jan 11 13:59 ceph.client.cinder-backup.keyring -r--------+ 1 cinder cinder 64 Jan 11 13:59 ceph.client.cinder.keyring -rw-------. 1 glance glance 64 Jan 11 13:59 ceph.client.glance.keyring -r--------+ 1 gnocchi gnocchi 65 Jan 17 12:25 ceph.client.gnocchi.keyring -rw-r--r--. 1 root root 220 Jan 11 13:59 ceph.conf Let me know how things in Newton are working; we're only as far as Mitaka at the moment. Cheers, -E -- Evan F. Bollig, PhD Scientific Computing Consultant, Application Developer | Scientific Computing Solutions (SCS) Minnesota Supercomputing Institute | msi.umn.edu University of Minnesota | umn.edu boll0...@umn.edu | 612-624-1447 | Walter Lib Rm 556 On Thu, Mar 2, 2017 at 3:13 AM, Esteban Freire <caroub...@yahoo.es> wrote: > Hello all, > > I am testing OpenStack Newton on CentOS 7 and now I already have a OpenStack > cloud infrastructure working, I would like to integrate Ceph with Cinder, > Glance and Nova services. > > I have found some information about how to performance this on previous > releases for Openstack and Ceph: > > http://docs.ceph.com/docs/jewel/rbd/rbd-openstack/ > > And I tried to update the variables according to > https://docs.openstack.org/newton/config-reference/block-storage/drivers/ceph-rbd-volume-driver.html > > This is my current glance-api.conf and which is working: > > [glance_store] > stores = file,http > default_store = file > filesystem_store_datadir = /var/lib/glance/images/ > > > At this is what I tried (at the moment, I have only tried it with the glance > service but I did not get success): > > * Install python-rbd and python-rados from centos-ceph-jewel repo on the > controller node. > * Create a ceph user and add it to sudoers. > * On ceph admin node: > > sudo ceph osd pool create images 150 > sudo ceph auth get-or-create client.glance mon 'allow r' osd 'allow > class-read object_prefix rbd_children, allow rwx pool=images' > sudo ceph auth get-or-create client.glance | ssh > ceph@controller-node1 sudo tee /etc/ceph/ceph.client.glance.keyring > ssh ceph@controller-node1 sudo chown glance:glance > /etc/ceph/ceph.client.glance.keyring > > * On the controller node, I edited the glance-api.conf file with the > following variables: > > [glance_store] > stores = file,http,rbd > default_store = rbd > rbd_pool = images > rbd_user = glance > rbd_ceph_conf = /etc/ceph/ceph.conf > rbd_store_chunk_size = 8 > > (!) I also have tried with stores = rbd but without success. > > * And restart the service, systemctl restart openstack-glance-api > > But when I try to create a new image, I get the following issue: > > [openstackadmin@controller-node1 ~]$ openstack image create "cirros > ceph" --file /home/openstackadmin/cirros-0.3.4-x86_64-disk.raw --disk-format > raw --container-format bare --public > 500 Internal Server Error > The server has either erred or is incapable of performing the requested > operation. > (HTTP 500) > > Is there any documentation about how to integrate Ceph jewel with OpenStack > newton (Cinder, Glance and Nova services)? If it is so, could you please > provide me the link? > > On the other hand, Is there any way to chose the store when creating an > image? I mean, to choose for example if save the image on > /var/lib/glance/images/ or on Ceph. > > I would appreciate if you could help me to set up this integration. > > This is the most relevant info I can see on the logs and as far as I see > from it, I have a permissions error but I am not sure what I need to modify. > It is my first installation with OpenStack by the way and I am trying at > home to see how it works :) > > {{{ > /var/log/glance/api.log > > 2017-03-01 23:43:41.419 4197 INFO eventlet.wsgi.server > [req-eea55909-c963-4158-b30c-3f2779fd78c6 c41043a1ddc14ffba1b45c0a3287e0bf > b2d1547f0e734f87a84feea75ccd6453 - default default] 192.168.56.2 - - > [01/Mar/2017 23:43:41] "GET /v2/schemas/image HTTP/1.1" 200 4352 0.421281 > 2017-03-01 23:43:41.570 4197 INFO eventlet.wsgi.server > [req-f5ef0b43-0ddf-4698-8459-2f75fb1822a3 c41043a1ddc14ffba1b45c0a3287e0bf > b2d1547f0e734f87a84feea75ccd6453 - default default] 192.168.56.2 - - > [01/Mar/2017 23:43:41] "POST /v2/images HTTP/1.1" 201 859 0.110681 > 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data > [req-98c406fb-7bc0-47de-bace-2b0a2097b699 c41043a1ddc14ffba1b45c0a3287e0bf > b2d1547f0e734f87a84feea75ccd6453 - default default] Failed to upload image > data due to internal error > 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data Traceback (most > recent call last): > 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File > "/usr/lib/python2.7/site-packages/glance/api/v2/image_data.py", line 115, in > upload > 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data > image.set_data(data, size) > 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File > "/usr/lib/python2.7/site-packages/glance/domain/proxy.py", line 195, in > set_data > 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data > self.base.set_data(data, size) > 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File > "/usr/lib/python2.7/site-packages/glance/notifier.py", line 479, in set_data > 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data > _send_notification(notify_error, 'image.upload', msg) > 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File > "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in > __exit__ > 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data > self.force_reraise() > 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File > "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in > force_reraise > 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data > six.reraise(self.type_, self.value, self.tb) > 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File > "/usr/lib/python2.7/site-packages/glance/notifier.py", line 426, in set_data > 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data > self.repo.set_data(data, size) > 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File > "/usr/lib/python2.7/site-packages/glance/api/policy.py", line 185, in > set_data > 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data return > self.image.set_data(*args, **kwargs) > 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File > "/usr/lib/python2.7/site-packages/glance/quota/__init__.py", line 304, in > set_data > 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data > self.image.set_data(data, size=size) > 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File > "/usr/lib/python2.7/site-packages/glance/location.py", line 440, in set_data > 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data > verifier=verifier) > 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File > "/usr/lib/python2.7/site-packages/glance_store/backend.py", line 447, in > add_to_backend > 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data verifier) > 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File > "/usr/lib/python2.7/site-packages/glance_store/backend.py", line 420, in > store_add_to_backend > 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data > verifier=verifier) > 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File > "/usr/lib/python2.7/site-packages/glance_store/capabilities.py", line 225, > in op_checker > 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data return > store_op_fun(store, *args, **kwargs) > 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File > "/usr/lib/python2.7/site-packages/glance_store/_drivers/rbd.py", line 447, > in add > 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data > rados_id=self.user) as conn: > 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File > "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ > 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data return > self.gen.next() > 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File > "/usr/lib/python2.7/site-packages/glance_store/_drivers/rbd.py", line 270, > in get_connection > 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data client = > rados.Rados(conffile=conffile, rados_id=rados_id) > 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File > "rados.pyx", line 525, in rados.Rados.__init__ (rados.c:5719) > 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File > "rados.pyx", line 425, in rados.requires.wrapper.validate_func > (rados.c:4106) > 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File > "rados.pyx", line 568, in rados.Rados.__setup (rados.c:6448) > 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File > "rados.pyx", line 425, in rados.requires.wrapper.validate_func > (rados.c:4106) > 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File > "rados.pyx", line 631, in rados.Rados.conf_read_file (rados.c:7384) > 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data > PermissionDeniedError: error calling conf_read_file > 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi > [req-98c406fb-7bc0-47de-bace-2b0a2097b699 c41043a1ddc14ffba1b45c0a3287e0bf > b2d1547f0e734f87a84feea75ccd6453 - default default] Caught error: error > calling conf_read_file > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi Traceback (most recent > call last): > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File > "/usr/lib/python2.7/site-packages/glance/common/wsgi.py", line 1113, in > __call__ > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi request, > **action_args) > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File > "/usr/lib/python2.7/site-packages/glance/common/wsgi.py", line 1152, in > dispatch > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi return > method(*args, **kwargs) > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File > "/usr/lib/python2.7/site-packages/glance/common/utils.py", line 363, in > wrapped > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi return func(self, > req, *args, **kwargs) > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File > "/usr/lib/python2.7/site-packages/glance/api/v2/image_data.py", line 250, in > upload > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi > self._restore(image_repo, image) > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File > "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in > __exit__ > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi > self.force_reraise() > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File > "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in > force_reraise > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi > six.reraise(self.type_, self.value, self.tb) > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File > "/usr/lib/python2.7/site-packages/glance/api/v2/image_data.py", line 115, in > upload > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi > image.set_data(data, size) > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File > "/usr/lib/python2.7/site-packages/glance/domain/proxy.py", line 195, in > set_data > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi > self.base.set_data(data, size) > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File > "/usr/lib/python2.7/site-packages/glance/notifier.py", line 479, in set_data > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi > _send_notification(notify_error, 'image.upload', msg) > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File > "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in > __exit__ > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi > self.force_reraise() > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File > "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in > force_reraise > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi > six.reraise(self.type_, self.value, self.tb) > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File > "/usr/lib/python2.7/site-packages/glance/notifier.py", line 426, in set_data > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi > self.repo.set_data(data, size) > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File > "/usr/lib/python2.7/site-packages/glance/api/policy.py", line 185, in > set_data > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi return > self.image.set_data(*args, **kwargs) > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File > "/usr/lib/python2.7/site-packages/glance/quota/__init__.py", line 304, in > set_data > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi > self.image.set_data(data, size=size) > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File > "/usr/lib/python2.7/site-packages/glance/location.py", line 440, in set_data > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi verifier=verifier) > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File > "/usr/lib/python2.7/site-packages/glance_store/backend.py", line 447, in > add_to_backend > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi verifier) > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File > "/usr/lib/python2.7/site-packages/glance_store/backend.py", line 420, in > store_add_to_backend > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi verifier=verifier) > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File > "/usr/lib/python2.7/site-packages/glance_store/capabilities.py", line 225, > in op_checker > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi return > store_op_fun(store, *args, **kwargs) > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File > "/usr/lib/python2.7/site-packages/glance_store/_drivers/rbd.py", line 447, > in add > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi > rados_id=self.user) as conn: > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File > "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi return > self.gen.next() > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File > "/usr/lib/python2.7/site-packages/glance_store/_drivers/rbd.py", line 270, > in get_connection > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi client = > rados.Rados(conffile=conffile, rados_id=rados_id) > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "rados.pyx", > line 525, in rados.Rados.__init__ (rados.c:5719) > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "rados.pyx", > line 425, in rados.requires.wrapper.validate_func (rados.c:4106) > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "rados.pyx", > line 568, in rados.Rados.__setup (rados.c:6448) > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "rados.pyx", > line 425, in rados.requires.wrapper.validate_func (rados.c:4106) > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "rados.pyx", > line 631, in rados.Rados.conf_read_file (rados.c:7384) > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi PermissionDeniedError: > error calling conf_read_file > 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi > }}} > > Thanks in advance, > Esteban > > > _______________________________________________ > Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack > Post to : openstack@lists.openstack.org > Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack >
_______________________________________________ Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : openstack@lists.openstack.org Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
