Public bug reported:
Description
===========
cold migration failed when server is specified with a numa topology
Steps to reproduce
==================
create server from a flavor specified with numa topology parameters and then do
a cold migrate or resize
Expected
========
success
Actual
======
failed with following messages
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server
[req-c299c521-2a07-483b-b19e-deb136572da0 dde6a5842265470a8e2f40938ae66097
f3d6994dfaf043479c9cf5bbac19ab87 - default default] Exception during message
handling: ValueError: Circular reference detected
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server Traceback (most
recent call last):
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server File
"/var/lib/openstack/lib/python3.6/site-packages/oslo_messaging/rpc/server.py",
line 166, in _process_incoming
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server res =
self.dispatcher.dispatch(message)
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server File
"/var/lib/openstack/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py",
line 265, in dispatch
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server return
self._do_dispatch(endpoint, method, ctxt, args)
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server File
"/var/lib/openstack/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py",
line 194, in _do_dispatch
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server result =
func(ctxt, **new_args)
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server File
"/var/lib/openstack/lib/python3.6/site-packages/oslo_messaging/rpc/server.py",
line 229, in inner
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server return
func(*args, **kwargs)
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server File
"/var/lib/openstack/lib/python3.6/site-packages/nova/conductor/manager.py",
line 94, in wrapper
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server return fn(self,
context, *args, **kwargs)
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server File
"/var/lib/openstack/lib/python3.6/site-packages/nova/compute/utils.py", line
1164, in decorated_function
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server return
function(self, context, *args, **kwargs)
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server File
"/var/lib/openstack/lib/python3.6/site-packages/nova/conductor/manager.py",
line 298, in migrate_server
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server host_list)
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server File
"/var/lib/openstack/lib/python3.6/site-packages/nova/conductor/manager.py",
line 358, in _cold_migrate
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server updates, ex,
request_spec)
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server File
"/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line
220, in __exit__
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server
self.force_reraise()
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server File
"/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line
196, in force_reraise
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server
six.reraise(self.type_, self.value, self.tb)
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server File
"/var/lib/openstack/lib/python3.6/site-packages/six.py", line 693, in reraise
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server raise value
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server File
"/var/lib/openstack/lib/python3.6/site-packages/nova/conductor/manager.py",
line 327, in _cold_migrate
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server task.execute()
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server File
"/var/lib/openstack/lib/python3.6/site-packages/nova/conductor/tasks/base.py",
line 27, in wrap
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server self.rollback()
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server File
"/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line
220, in __exit__
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server
self.force_reraise()
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server File
"/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line
196, in force_reraise
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server
six.reraise(self.type_, self.value, self.tb)
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server File
"/var/lib/openstack/lib/python3.6/site-packages/six.py", line 693, in reraise
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server raise value
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server File
"/var/lib/openstack/lib/python3.6/site-packages/nova/conductor/tasks/base.py",
line 24, in wrap
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server return
original(self)
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server File
"/var/lib/openstack/lib/python3.6/site-packages/nova/conductor/tasks/base.py",
line 42, in execute
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server return
self._execute()
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server File
"/var/lib/openstack/lib/python3.6/site-packages/nova/conductor/tasks/migrate.py",
line 295, in _execute
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server legacy_spec =
jsonutils.loads(jsonutils.dumps(legacy_spec))
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server File
"/var/lib/openstack/lib/python3.6/site-packages/oslo_serialization/jsonutils.py",
line 213, in dumps
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server return
json.dumps(obj, default=default, **kwargs)
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server File
"/usr/lib/python3.6/json/__init__.py", line 238, in dumps
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server **kw).encode(obj)
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server File
"/usr/lib/python3.6/json/encoder.py", line 199, in encode
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server chunks =
self.iterencode(o, _one_shot=True)
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server File
"/usr/lib/python3.6/json/encoder.py", line 257, in iterencode
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server return
_iterencode(o, 0)
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server ValueError: Circular
reference detected
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server
properties | SERVICE='ECS', SPEC='GENERAL',
hw:cpu_cores='4', hw:cpu_policy='dedicated', hw:cpu_sockets='1',
hw:cpu_thread_policy='isolate', hw:cpu_threads='1',
hw:emulator_threads_policy='share', quota:vif_inbound_average='1048576',
quota:vif_outbound_average='1048576' |
** Affects: nova
Importance: Undecided
Status: Invalid
** Changed in: nova
Status: New => Fix Released
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1931209
Title:
Circular reference detected during cold migration
Status in OpenStack Compute (nova):
Invalid
Bug description:
Description
===========
cold migration failed when server is specified with a numa topology
Steps to reproduce
==================
create server from a flavor specified with numa topology parameters and then
do
a cold migrate or resize
Expected
========
success
Actual
======
failed with following messages
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server
[req-c299c521-2a07-483b-b19e-deb136572da0 dde6a5842265470a8e2f40938ae66097
f3d6994dfaf043479c9cf5bbac19ab87 - default default] Exception during message
handling: ValueError: Circular reference detected
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server Traceback (most
recent call last):
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server File
"/var/lib/openstack/lib/python3.6/site-packages/oslo_messaging/rpc/server.py",
line 166, in _process_incoming
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server res =
self.dispatcher.dispatch(message)
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server File
"/var/lib/openstack/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py",
line 265, in dispatch
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server return
self._do_dispatch(endpoint, method, ctxt, args)
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server File
"/var/lib/openstack/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py",
line 194, in _do_dispatch
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server result =
func(ctxt, **new_args)
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server File
"/var/lib/openstack/lib/python3.6/site-packages/oslo_messaging/rpc/server.py",
line 229, in inner
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server return
func(*args, **kwargs)
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server File
"/var/lib/openstack/lib/python3.6/site-packages/nova/conductor/manager.py",
line 94, in wrapper
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server return
fn(self, context, *args, **kwargs)
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server File
"/var/lib/openstack/lib/python3.6/site-packages/nova/compute/utils.py", line
1164, in decorated_function
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server return
function(self, context, *args, **kwargs)
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server File
"/var/lib/openstack/lib/python3.6/site-packages/nova/conductor/manager.py",
line 298, in migrate_server
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server host_list)
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server File
"/var/lib/openstack/lib/python3.6/site-packages/nova/conductor/manager.py",
line 358, in _cold_migrate
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server updates, ex,
request_spec)
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server File
"/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line
220, in __exit__
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server
self.force_reraise()
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server File
"/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line
196, in force_reraise
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server
six.reraise(self.type_, self.value, self.tb)
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server File
"/var/lib/openstack/lib/python3.6/site-packages/six.py", line 693, in reraise
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server raise value
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server File
"/var/lib/openstack/lib/python3.6/site-packages/nova/conductor/manager.py",
line 327, in _cold_migrate
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server task.execute()
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server File
"/var/lib/openstack/lib/python3.6/site-packages/nova/conductor/tasks/base.py",
line 27, in wrap
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server self.rollback()
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server File
"/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line
220, in __exit__
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server
self.force_reraise()
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server File
"/var/lib/openstack/lib/python3.6/site-packages/oslo_utils/excutils.py", line
196, in force_reraise
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server
six.reraise(self.type_, self.value, self.tb)
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server File
"/var/lib/openstack/lib/python3.6/site-packages/six.py", line 693, in reraise
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server raise value
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server File
"/var/lib/openstack/lib/python3.6/site-packages/nova/conductor/tasks/base.py",
line 24, in wrap
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server return
original(self)
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server File
"/var/lib/openstack/lib/python3.6/site-packages/nova/conductor/tasks/base.py",
line 42, in execute
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server return
self._execute()
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server File
"/var/lib/openstack/lib/python3.6/site-packages/nova/conductor/tasks/migrate.py",
line 295, in _execute
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server legacy_spec =
jsonutils.loads(jsonutils.dumps(legacy_spec))
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server File
"/var/lib/openstack/lib/python3.6/site-packages/oslo_serialization/jsonutils.py",
line 213, in dumps
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server return
json.dumps(obj, default=default, **kwargs)
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server File
"/usr/lib/python3.6/json/__init__.py", line 238, in dumps
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server
**kw).encode(obj)
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server File
"/usr/lib/python3.6/json/encoder.py", line 199, in encode
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server chunks =
self.iterencode(o, _one_shot=True)
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server File
"/usr/lib/python3.6/json/encoder.py", line 257, in iterencode
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server return
_iterencode(o, 0)
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server ValueError:
Circular reference detected
2021-06-08 04:24:52.963 19 ERROR oslo_messaging.rpc.server
properties | SERVICE='ECS', SPEC='GENERAL',
hw:cpu_cores='4', hw:cpu_policy='dedicated', hw:cpu_sockets='1',
hw:cpu_thread_policy='isolate', hw:cpu_threads='1',
hw:emulator_threads_policy='share',
quota:vif_inbound_average='1048576',
quota:vif_outbound_average='1048576' |
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1931209/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : [email protected]
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
