Public bug reported: When a libvirt based VM is unshelved to a ironic compute host then nova does not fail the operation gracefully with NoValidHost, but instead the scheduler fails with a stack trace:
2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server [None req-a45169a4-4024-48d0-bd04-a081d12bfbbc 99a273a7d2014e82a7d043c6e93a6ff6 be4c3131a4cd4f9f9652d3f6f4987f42 - - default default] Exception during message handling: oslo_versionedobjects.exception.ObjectActionError: Object action set_defaults failed because: No default set for field node 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/oslo_messaging/rpc/server.py", line 244, in inner 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server return func(*args, **kwargs) 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/nova/scheduler/manager.py", line 224, in select_destinations 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server selections = self._select_destinations( 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/nova/scheduler/manager.py", line 251, in _select_destinations 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server selections = self._schedule( 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/nova/scheduler/manager.py", line 388, in _schedule 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server hosts = self._get_sorted_hosts(spec_obj, hosts, num) 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/nova/scheduler/manager.py", line 672, in _get_sorted_hosts 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server filtered_hosts = self.host_manager.get_filtered_hosts(host_states, 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/nova/scheduler/host_manager.py", line 590, in get_filtered_hosts 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server hosts = _get_hosts_matching_request(hosts, requested_node) 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/nova/scheduler/host_manager.py", line 562, in _get_hosts_matching_request 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server requested_destination.node) 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/oslo_versionedobjects/base.py", line 67, in getter 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server self.obj_load_attr(name) 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/nova/objects/request_spec.py", line 1140, in obj_load_attr 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server self.obj_set_defaults(attrname) 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/oslo_versionedobjects/base.py", line 588, in obj_set_defaults 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server raise exception.ObjectActionError( 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server oslo_versionedobjects.exception.ObjectActionError: Object action set_defaults failed because: No default set for field node 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server [00m The unshelve code creates a Destination object with only the host field set and does not set the node filed. https://github.com/openstack/nova/blob/db263370ec9e306eb03addf34b197890996054d7/nova/compute/api.py#L4660-L4662 However the same code seems to work without the stack trace if the target host is a libvirt compute. Seen on 2023.1 in tempest test test_unshelve_to_specific_host ** Affects: nova Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/2075504 Title: Unhselve to specific host can fail with oslo_versionedobjects.exception.ObjectActionError: Object action set_defaults failed because: No default set for field node Status in OpenStack Compute (nova): New Bug description: When a libvirt based VM is unshelved to a ironic compute host then nova does not fail the operation gracefully with NoValidHost, but instead the scheduler fails with a stack trace: 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server [None req-a45169a4-4024-48d0-bd04-a081d12bfbbc 99a273a7d2014e82a7d043c6e93a6ff6 be4c3131a4cd4f9f9652d3f6f4987f42 - - default default] Exception during message handling: oslo_versionedobjects.exception.ObjectActionError: Object action set_defaults failed because: No default set for field node 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/oslo_messaging/rpc/server.py", line 244, in inner 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server return func(*args, **kwargs) 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/nova/scheduler/manager.py", line 224, in select_destinations 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server selections = self._select_destinations( 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/nova/scheduler/manager.py", line 251, in _select_destinations 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server selections = self._schedule( 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/nova/scheduler/manager.py", line 388, in _schedule 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server hosts = self._get_sorted_hosts(spec_obj, hosts, num) 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/nova/scheduler/manager.py", line 672, in _get_sorted_hosts 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server filtered_hosts = self.host_manager.get_filtered_hosts(host_states, 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/nova/scheduler/host_manager.py", line 590, in get_filtered_hosts 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server hosts = _get_hosts_matching_request(hosts, requested_node) 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/nova/scheduler/host_manager.py", line 562, in _get_hosts_matching_request 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server requested_destination.node) 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/oslo_versionedobjects/base.py", line 67, in getter 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server self.obj_load_attr(name) 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/nova/objects/request_spec.py", line 1140, in obj_load_attr 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server self.obj_set_defaults(attrname) 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/oslo_versionedobjects/base.py", line 588, in obj_set_defaults 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server raise exception.ObjectActionError( 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server oslo_versionedobjects.exception.ObjectActionError: Object action set_defaults failed because: No default set for field node 2024-08-01 00:56:36.414 1 ERROR oslo_messaging.rpc.server [00m The unshelve code creates a Destination object with only the host field set and does not set the node filed. https://github.com/openstack/nova/blob/db263370ec9e306eb03addf34b197890996054d7/nova/compute/api.py#L4660-L4662 However the same code seems to work without the stack trace if the target host is a libvirt compute. Seen on 2023.1 in tempest test test_unshelve_to_specific_host To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/2075504/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
