Public bug reported: If we had two volume, one is attached by a instance, and the other is still available. Due to https://bugs.launchpad.net/nova/+bug/1489744, those volumes will stay in wrong status. It will swap successful after fix the issue. Volumes will be available and in-use status. But when I try to swap in-use volume with other available volume. Nova compute will throw following exception:
2015-08-30 04:55:13.772 ERROR oslo_messaging.rpc.dispatcher [req-4d999362-7a13-4b43-8c6a-0d85f3b9aa5b admin admin] Exception during message handling: No volume Block Device Mapping with id cafc833a-8645-47db-b464-999142afa7be. Traceback (most recent call last): File "/opt/stack/nova/nova/conductor/manager.py", line 443, in _object_dispatch return getattr(target, method)(*args, **kwargs) File "/usr/local/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 169, in wrapper result = fn(cls, context, *args, **kwargs) File "/opt/stack/nova/nova/objects/block_device.py", line 204, in get_by_volume_id raise exception.VolumeBDMNotFound(volume_id=volume_id) VolumeBDMNotFound: No volume Block Device Mapping with id cafc833a-8645-47db-b464-999142afa7be. 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher Traceback (most recent call last): 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher executor_callback)) 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher executor_callback) 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 129, in _do_dispatch 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher result = func(ctxt, **new_args) 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/exception.py", line 89, in wrapped 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher payload) 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 119, in __exit__ 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/exception.py", line 72, in wrapped 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher return f(self, context, *args, **kw) 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/compute/manager.py", line 345, in decorated_function 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher LOG.warning(msg, e, instance_uuid=instance_uuid) 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 119, in __exit__ 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/compute/manager.py", line 316, in decorated_function 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher return function(self, context, *args, **kwargs) 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/compute/manager.py", line 373, in decorated_function 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher kwargs['instance'], e, sys.exc_info()) 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 119, in __exit__ 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/compute/manager.py", line 361, in decorated_function 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher return function(self, context, *args, **kwargs) 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/compute/manager.py", line 4706, in swap_volume 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher resize_to) 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/compute/manager.py", line 4658, in _swap_volume 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher self.volume_api.unreserve_volume(context, new_volume_id) 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 119, in __exit__ 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/compute/manager.py", line 4639, in _swap_volume 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher resize_to) 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 1217, in swap_volume 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher nova_context.get_admin_context(), volume_id) 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 167, in wrapper 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher args, kwargs) 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/conductor/rpcapi.py", line 239, in object_class_action 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher objver=objver, args=args, kwargs=kwargs) 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 158, in call 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher retry=self.retry) 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 90, in _send 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher timeout=timeout, retry=retry) 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 431, in send 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher retry=retry) 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 422, in _send 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher raise result 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher VolumeBDMNotFound_Remote: No volume Block Device Mapping with id cafc833a-8645-47db-b464-999142afa7be. 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher Traceback (most recent call last): 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/conductor/manager.py", line 443, in _object_dispatch 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher return getattr(target, method)(*args, **kwargs) 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 169, in wrapper 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher result = fn(cls, context, *args, **kwargs) 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/objects/block_device.py", line 204, in get_by_volume_id 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher raise exception.VolumeBDMNotFound(volume_id=volume_id) 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher VolumeBDMNotFound: No volume Block Device Mapping with id cafc833a-8645-47db-b464-999142afa7be. 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher In first swap, nova will save serial with old volume id in block device mapping. Therefore nova will try to find block device mapping through old volume id at second swap. It had been change to new volume id, so nova will raise no bdm not found. ** Affects: nova Importance: Undecided Assignee: Chung Chih, Hung (lyanchih) Status: New ** Changed in: nova Assignee: (unassigned) => Chung Chih, Hung (lyanchih) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1490236 Title: Swapping volume can't been swap again Status in OpenStack Compute (nova): New Bug description: If we had two volume, one is attached by a instance, and the other is still available. Due to https://bugs.launchpad.net/nova/+bug/1489744, those volumes will stay in wrong status. It will swap successful after fix the issue. Volumes will be available and in-use status. But when I try to swap in-use volume with other available volume. Nova compute will throw following exception: 2015-08-30 04:55:13.772 ERROR oslo_messaging.rpc.dispatcher [req-4d999362-7a13-4b43-8c6a-0d85f3b9aa5b admin admin] Exception during message handling: No volume Block Device Mapping with id cafc833a-8645-47db-b464-999142afa7be. Traceback (most recent call last): File "/opt/stack/nova/nova/conductor/manager.py", line 443, in _object_dispatch return getattr(target, method)(*args, **kwargs) File "/usr/local/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 169, in wrapper result = fn(cls, context, *args, **kwargs) File "/opt/stack/nova/nova/objects/block_device.py", line 204, in get_by_volume_id raise exception.VolumeBDMNotFound(volume_id=volume_id) VolumeBDMNotFound: No volume Block Device Mapping with id cafc833a-8645-47db-b464-999142afa7be. 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher Traceback (most recent call last): 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher executor_callback)) 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher executor_callback) 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 129, in _do_dispatch 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher result = func(ctxt, **new_args) 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/exception.py", line 89, in wrapped 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher payload) 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 119, in __exit__ 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/exception.py", line 72, in wrapped 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher return f(self, context, *args, **kw) 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/compute/manager.py", line 345, in decorated_function 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher LOG.warning(msg, e, instance_uuid=instance_uuid) 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 119, in __exit__ 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/compute/manager.py", line 316, in decorated_function 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher return function(self, context, *args, **kwargs) 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/compute/manager.py", line 373, in decorated_function 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher kwargs['instance'], e, sys.exc_info()) 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 119, in __exit__ 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/compute/manager.py", line 361, in decorated_function 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher return function(self, context, *args, **kwargs) 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/compute/manager.py", line 4706, in swap_volume 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher resize_to) 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/compute/manager.py", line 4658, in _swap_volume 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher self.volume_api.unreserve_volume(context, new_volume_id) 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 119, in __exit__ 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/compute/manager.py", line 4639, in _swap_volume 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher resize_to) 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 1217, in swap_volume 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher nova_context.get_admin_context(), volume_id) 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 167, in wrapper 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher args, kwargs) 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/conductor/rpcapi.py", line 239, in object_class_action 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher objver=objver, args=args, kwargs=kwargs) 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 158, in call 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher retry=self.retry) 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 90, in _send 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher timeout=timeout, retry=retry) 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 431, in send 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher retry=retry) 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 422, in _send 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher raise result 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher VolumeBDMNotFound_Remote: No volume Block Device Mapping with id cafc833a-8645-47db-b464-999142afa7be. 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher Traceback (most recent call last): 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/conductor/manager.py", line 443, in _object_dispatch 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher return getattr(target, method)(*args, **kwargs) 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 169, in wrapper 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher result = fn(cls, context, *args, **kwargs) 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/objects/block_device.py", line 204, in get_by_volume_id 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher raise exception.VolumeBDMNotFound(volume_id=volume_id) 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher VolumeBDMNotFound: No volume Block Device Mapping with id cafc833a-8645-47db-b464-999142afa7be. 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher 2015-08-30 04:55:13.772 TRACE oslo_messaging.rpc.dispatcher In first swap, nova will save serial with old volume id in block device mapping. Therefore nova will try to find block device mapping through old volume id at second swap. It had been change to new volume id, so nova will raise no bdm not found. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1490236/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp