Hi Sang Tran, hope this message finds you well. From the logs, looks like a nova problem. ` ERROR oslo_messaging.rpc.server novaclient.exceptions.Conflict: Cannot 'swap_volume' instance a14b7344-3054-4d9a-9357-c0adc1b00650 while it is in vm_state stopped (HTTP 409) (Request-ID: req-7913c7fb-51d0-4a2a-8463-d8a9883dea0b) `
Question: are you trying to migrate in-use volumes (attached volumes)? Just for the sake of completeness I need to mentioned that migration of ‘in-use’ volumes will not succeed for any backend when they are attached to an instance in any of these states: SHUTOFF, SUSPENDED, or SOFT-DELETED[1]. Adding Nova team for more feedback. Thanks, Sofia [1] https://docs.openstack.org/cinder/latest/contributor/migration.html ** Tags added: migration retype ** Changed in: cinder Importance: Undecided => Medium ** Also affects: nova Importance: Undecided Status: New ** Changed in: cinder Status: New => Incomplete -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1999125 Title: cinder retype volume raise exception with shutoff vm Status in Cinder: Incomplete Status in OpenStack Compute (nova): New Bug description: Environment: - openstack victoria version - iSCSI storage using Netapp driver Step to reproduce: 1. Shutoff vm and wait for its complete 2. Run the command cinder retype --migration-policy on-demand <vol-id> <new-type> 3. Task will fail with Error log below 2022-12-07 16:49:29.381 966 ERROR cinder.volume.manager [req-aea12cc1-b7f5-4c38-85d7-1c9094a1006a d3802c30ff814feaa3197d29e8a3a731 2c97f9ded4c843279d2de2c4a5c837 01 - - -] Failed to copy volume f81a905d-05ea-4534-9b3b-a6762adf87f3 to 099ce4e2-8539-43dc-a2fa-8feca2845a3d: novaclient.exceptions.Conflict: Cannot 'swap_volume' instance a14b7344-3054-4d9a-9357-c0adc1b00650 while it is in vm_state stopped (HTTP 409) (Request-ID: req-7913c7fb-51d0-4a2a-8463-d8a9883dea0b) 2022-12-07 16:49:29.381 966 ERROR cinder.volume.manager Traceback (most recent call last): 2022-12-07 16:49:29.381 966 ERROR cinder.volume.manager File "/var/lib/kolla/venv/lib/python3.8/site-packages/cinder/volume/manager.py", line 2292, in _migrate_volume_generic 2022-12-07 16:49:29.381 966 ERROR cinder.volume.manager nova_api.update_server_volume(ctxt, instance_uuid, 2022-12-07 16:49:29.381 966 ERROR cinder.volume.manager File "/var/lib/kolla/venv/lib/python3.8/site-packages/cinder/compute/nova.py", line 186, in update_server_volume 2022-12-07 16:49:29.381 966 ERROR cinder.volume.manager nova.volumes.update_server_volume(server_id, 2022-12-07 16:49:29.381 966 ERROR cinder.volume.manager File "/var/lib/kolla/venv/lib/python3.8/site-packages/novaclient/api_versions.py", line 393, in substitution 2022-12-07 16:49:29.381 966 ERROR cinder.volume.manager return methods[-1].func(obj, *args, **kwargs) 2022-12-07 16:49:29.381 966 ERROR cinder.volume.manager File "/var/lib/kolla/venv/lib/python3.8/site-packages/novaclient/v2/volumes.py", line 124, in update_server_volume 2022-12-07 16:49:29.381 966 ERROR cinder.volume.manager return self._update("/servers/%s/os-volume_attachments/%s" % 2022-12-07 16:49:29.381 966 ERROR cinder.volume.manager File "/var/lib/kolla/venv/lib/python3.8/site-packages/novaclient/base.py", line 380, in _update 2022-12-07 16:49:29.381 966 ERROR cinder.volume.manager resp, body = self.api.client.put(url, body=body) 2022-12-07 16:49:29.381 966 ERROR cinder.volume.manager File "/var/lib/kolla/venv/lib/python3.8/site-packages/keystoneauth1/adapter.py", line 404, in put 2022-12-07 16:49:29.381 966 ERROR cinder.volume.manager return self.request(url, 'PUT', **kwargs) 2022-12-07 16:49:29.381 966 ERROR cinder.volume.manager File "/var/lib/kolla/venv/lib/python3.8/site-packages/novaclient/client.py", line 78, in request 2022-12-07 16:49:29.381 966 ERROR cinder.volume.manager raise exceptions.from_response(resp, body, url, method) 2022-12-07 16:49:29.381 966 ERROR cinder.volume.manager novaclient.exceptions.Conflict: Cannot 'swap_volume' instance a14b7344-3054-4d9a-9357-c0adc1b00650 whileit is in vm_state stopped (HTTP 409) (Request-ID: req-7913c7fb-51d0-4a2a-8463-d8a9883dea0b) 2022-12-07 16:49:29.381 966 ERROR cinder.volume.manager 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server [req-aea12cc1-b7f5-4c38-85d7-1c9094a1006a d3802c30ff814feaa3197d29e8a3a731 2c97f9ded4c843279d2de2c4a5c83701 - - -] Exception during message handling: novaclient.exceptions.Conflict: Cannot 'swap_volume' instance a14b7344-3054-4d9a-9357-c0adc1b00650 while it is in vm_state stopped (HTTP 409) (Request-ID: req-7913c7fb-51d0-4a2a-8463-d8a9883dea0b) 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_messaging/rpc/dispatcher.py", line 309,in dispatch 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_messaging/rpc/dispatcher.py", line 229,in _do_dispatch 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/cinder/volume/manager.py", line 2957, in retype 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server _retype_error(context, volume, old_reservations, 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server self.force_reraise() 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.8/dist-packages/six.py", line 703, in reraise 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server raise value 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/cinder/volume/manager.py", line 2953, in retype 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server self.migrate_volume(context, volume, host, 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/cinder/volume/manager.py", line 2546, in migrate_volume 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server volume.save() 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server self.force_reraise() 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.8/dist-packages/six.py", line 703, in reraise 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server raise value 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/cinder/volume/manager.py", line 2539, in migrate_volume 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server self._migrate_volume_generic(ctxt, volume, host, new_type_id) 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/cinder/volume/manager.py", line 2300, in _migrate_volume_generic 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server self._clean_temporary_volume(ctxt, volume, 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server self.force_reraise() 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.8/dist-packages/six.py", line 703, in reraise 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server raise value 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/cinder/volume/manager.py", line 2292, in _migrate_volume_generic 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server nova_api.update_server_volume(ctxt, instance_uuid, 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/cinder/compute/nova.py", line 186, in update_server_volume 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server nova.volumes.update_server_volume(server_id, 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/novaclient/api_versions.py", line 393, in substitution 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server return methods[-1].func(obj, *args, **kwargs) 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/novaclient/v2/volumes.py", line 124, in update_server_volume 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server return self._update("/servers/%s/os-volume_attachments/%s" % 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/novaclient/base.py", line 380, in _update 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server resp, body = self.api.client.put(url, body=body) 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/keystoneauth1/adapter.py", line 404, in put 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server return self.request(url, 'PUT', **kwargs) 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/novaclient/client.py", line 78, in request 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server raise exceptions.from_response(resp, body, url, method) 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server novaclient.exceptions.Conflict: Cannot 'swap_volume' instance a14b7344-3054-4d9a-9357-c0adc1b00650 while it is in vm_state stopped (HTTP 409) (Request-ID: req-7913c7fb-51d0-4a2a-8463-d8a9883dea0b) 2022-12-07 16:49:29.408 966 ERROR oslo_messaging.rpc.server To manage notifications about this bug go to: https://bugs.launchpad.net/cinder/+bug/1999125/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
