Public bug reported: Sheepdog driver fail to execute 'collie' commands.
How to reproduce: $ glance image-create --name img00 --disk-format raw --container-format bare --file /tmp/data.img +------------------+--------------------------------------+ | Property | Value | +------------------+--------------------------------------+ | checksum | None | | container_format | bare | | created_at | 2016-09-05T07:58:51Z | | disk_format | raw | | id | 66e9827a-e254-4b37-b889-07c6764118d6 | | min_disk | 0 | | min_ram | 0 | | name | img00 | | owner | cb433b293d6c498d9a8a3989bb4275ab | | protected | False | | size | None | | status | queued | | tags | [] | | updated_at | 2016-09-05T07:58:51Z | | virtual_size | None | | visibility | private | +------------------+--------------------------------------+ 500 Internal Server Error The server has either erred or is incapable of performing the requested operation. (HTTP 500) traceback of g-api: 2016-09-05 07:58:52.207 DEBUG oslo_concurrency.processutils [req-6e1899f4-5c47-4d88-9a08-cbf8a7a67e0c de40eaf63efe4b3ea8d9c83b97482736 cb433b293d6c498d9a8a3989bb4275ab] Running cmd (subprocess): collie vdi list -r -a 127.0.0.1 -p 7000 66e9827a-e254-4b37-b889-07c6764118d6 from (pid=17867) execute /usr/local/lib/python2.7/dist-packages/oslo_concurrency/processutils.py:344 2016-09-05 07:58:52.222 DEBUG oslo_concurrency.processutils [req-6e1899f4-5c47-4d88-9a08-cbf8a7a67e0c de40eaf63efe4b3ea8d9c83b97482736 cb433b293d6c498d9a8a3989bb4275ab] u'collie vdi list -r -a 127.0.0.1 -p 7000 66e9827a-e254-4b37-b889-07c6764118d6 ' failed. Not Retrying. from (pid=17867) execute /usr/local/lib/python2.7/dist-packages/oslo_concurrency/processutils.py:422 2016-09-05 07:58:52.224 DEBUG oslo_messaging._drivers.amqpdriver [req-6e1899f4-5c47-4d88-9a08-cbf8a7a67e0c de40eaf63efe4b3ea8d9c83b97482736 cb433b293d6c498d9a8a3989bb4275ab] CAST unique_id: fa25c1bf930b44ceb9ff56246e4f2666 NOTIFY exchange 'glance' topic 'notifications.error' from (pid=17867) _send /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:432 2016-09-05 07:58:52.229 ERROR glance.api.v2.image_data [req-6e1899f4-5c47-4d88-9a08-cbf8a7a67e0c de40eaf63efe4b3ea8d9c83b97482736 cb433b293d6c498d9a8a3989bb4275ab] Failed to upload image data due to interna l error 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data Traceback (most recent call last): 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data File "/opt/stack/glance/glance/api/v2/image_data.py", line 114, in upload 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data image.set_data(data, size) 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data File "/opt/stack/glance/glance/domain/proxy.py", line 195, in set_data 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data self.base.set_data(data, size) 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data File "/opt/stack/glance/glance/notifier.py", line 485, in set_data 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data _send_notification(notify_error, 'image.upload', msg) 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data self.force_reraise() 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data six.reraise(self.type_, self.value, self.tb) 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data File "/opt/stack/glance/glance/notifier.py", line 432, in set_data 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data self.repo.set_data(data, size) 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data File "/opt/stack/glance/glance/api/policy.py", line 185, in set_data 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data return self.image.set_data(*args, **kwargs) 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data File "/opt/stack/glance/glance/quota/__init__.py", line 304, in set_data 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data self.image.set_data(data, size=size) 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data File "/opt/stack/glance/glance/location.py", line 430, in set_data 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data verifier=verifier) 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data File "/opt/stack/glance_store/glance_store/backend.py", line 447, in add_to_backend 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data verifier) 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data File "/opt/stack/glance_store/glance_store/backend.py", line 420, in store_add_to_backend 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data verifier=verifier) 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data File "/opt/stack/glance_store/glance_store/capabilities.py", line 225, in op_checker 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data return store_op_fun(store, *args, **kwargs) 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data File "/opt/stack/glance_store/glance_store/_drivers/sheepdog.py", line 362, in add 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data if image.exist(): 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data File "/opt/stack/glance_store/glance_store/_drivers/sheepdog.py", line 200, in exist 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data out = self._run_command("list -r", None) 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data File "/opt/stack/glance_store/glance_store/_drivers/sheepdog.py", line 137, in _run_command 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data cmd, process_input=data)[0] 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data File "/usr/local/lib/python2.7/dist-packages/oslo_concurrency/processutils.py", line 363, in execute 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data env=env_variables) 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data File "/usr/local/lib/python2.7/dist-packages/eventlet/green/subprocess.py", line 54, in __init__ 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data subprocess_orig.Popen.__init__(self, args, 0, *argss, **kwds) 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data File "/usr/lib/python2.7/subprocess.py", line 710, in __init__ 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data errread, errwrite) 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data File "/usr/lib/python2.7/subprocess.py", line 1327, in _execute_child 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data raise child_exception 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data OSError: [Errno 2] No such file or directory 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data ** Affects: glance Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to Glance. https://bugs.launchpad.net/bugs/1620214 Title: Sheepdog: command execution failure Status in Glance: New Bug description: Sheepdog driver fail to execute 'collie' commands. How to reproduce: $ glance image-create --name img00 --disk-format raw --container-format bare --file /tmp/data.img +------------------+--------------------------------------+ | Property | Value | +------------------+--------------------------------------+ | checksum | None | | container_format | bare | | created_at | 2016-09-05T07:58:51Z | | disk_format | raw | | id | 66e9827a-e254-4b37-b889-07c6764118d6 | | min_disk | 0 | | min_ram | 0 | | name | img00 | | owner | cb433b293d6c498d9a8a3989bb4275ab | | protected | False | | size | None | | status | queued | | tags | [] | | updated_at | 2016-09-05T07:58:51Z | | virtual_size | None | | visibility | private | +------------------+--------------------------------------+ 500 Internal Server Error The server has either erred or is incapable of performing the requested operation. (HTTP 500) traceback of g-api: 2016-09-05 07:58:52.207 DEBUG oslo_concurrency.processutils [req-6e1899f4-5c47-4d88-9a08-cbf8a7a67e0c de40eaf63efe4b3ea8d9c83b97482736 cb433b293d6c498d9a8a3989bb4275ab] Running cmd (subprocess): collie vdi list -r -a 127.0.0.1 -p 7000 66e9827a-e254-4b37-b889-07c6764118d6 from (pid=17867) execute /usr/local/lib/python2.7/dist-packages/oslo_concurrency/processutils.py:344 2016-09-05 07:58:52.222 DEBUG oslo_concurrency.processutils [req-6e1899f4-5c47-4d88-9a08-cbf8a7a67e0c de40eaf63efe4b3ea8d9c83b97482736 cb433b293d6c498d9a8a3989bb4275ab] u'collie vdi list -r -a 127.0.0.1 -p 7000 66e9827a-e254-4b37-b889-07c6764118d6 ' failed. Not Retrying. from (pid=17867) execute /usr/local/lib/python2.7/dist-packages/oslo_concurrency/processutils.py:422 2016-09-05 07:58:52.224 DEBUG oslo_messaging._drivers.amqpdriver [req-6e1899f4-5c47-4d88-9a08-cbf8a7a67e0c de40eaf63efe4b3ea8d9c83b97482736 cb433b293d6c498d9a8a3989bb4275ab] CAST unique_id: fa25c1bf930b44ceb9ff56246e4f2666 NOTIFY exchange 'glance' topic 'notifications.error' from (pid=17867) _send /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:432 2016-09-05 07:58:52.229 ERROR glance.api.v2.image_data [req-6e1899f4-5c47-4d88-9a08-cbf8a7a67e0c de40eaf63efe4b3ea8d9c83b97482736 cb433b293d6c498d9a8a3989bb4275ab] Failed to upload image data due to inter nal error 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data Traceback (most recent call last): 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data File "/opt/stack/glance/glance/api/v2/image_data.py", line 114, in upload 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data image.set_data(data, size) 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data File "/opt/stack/glance/glance/domain/proxy.py", line 195, in set_data 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data self.base.set_data(data, size) 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data File "/opt/stack/glance/glance/notifier.py", line 485, in set_data 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data _send_notification(notify_error, 'image.upload', msg) 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data self.force_reraise() 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data six.reraise(self.type_, self.value, self.tb) 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data File "/opt/stack/glance/glance/notifier.py", line 432, in set_data 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data self.repo.set_data(data, size) 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data File "/opt/stack/glance/glance/api/policy.py", line 185, in set_data 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data return self.image.set_data(*args, **kwargs) 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data File "/opt/stack/glance/glance/quota/__init__.py", line 304, in set_data 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data self.image.set_data(data, size=size) 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data File "/opt/stack/glance/glance/location.py", line 430, in set_data 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data verifier=verifier) 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data File "/opt/stack/glance_store/glance_store/backend.py", line 447, in add_to_backend 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data verifier) 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data File "/opt/stack/glance_store/glance_store/backend.py", line 420, in store_add_to_backend 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data verifier=verifier) 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data File "/opt/stack/glance_store/glance_store/capabilities.py", line 225, in op_checker 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data return store_op_fun(store, *args, **kwargs) 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data File "/opt/stack/glance_store/glance_store/_drivers/sheepdog.py", line 362, in add 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data if image.exist(): 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data File "/opt/stack/glance_store/glance_store/_drivers/sheepdog.py", line 200, in exist 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data out = self._run_command("list -r", None) 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data File "/opt/stack/glance_store/glance_store/_drivers/sheepdog.py", line 137, in _run_command 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data cmd, process_input=data)[0] 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data File "/usr/local/lib/python2.7/dist-packages/oslo_concurrency/processutils.py", line 363, in execute 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data env=env_variables) 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data File "/usr/local/lib/python2.7/dist-packages/eventlet/green/subprocess.py", line 54, in __init__ 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data subprocess_orig.Popen.__init__(self, args, 0, *argss, **kwds) 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data File "/usr/lib/python2.7/subprocess.py", line 710, in __init__ 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data errread, errwrite) 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data File "/usr/lib/python2.7/subprocess.py", line 1327, in _execute_child 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data raise child_exception 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data OSError: [Errno 2] No such file or directory 2016-09-05 07:58:52.229 TRACE glance.api.v2.image_data To manage notifications about this bug go to: https://bugs.launchpad.net/glance/+bug/1620214/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp