Public bug reported:
Description
===========
I get an error after upgrade half of my cluster. Can't place any VMs.
"RemoteError: Remote error: IncompatibleObjectVersion Version 2.1 of
InstanceList is not supported"
Steps to reproduce
==================
1) Install 4 nodes with mitaka
2) Disable 2 nodes (1 api controller and 1 compute): nova service-disable
3) Upgrade to newton on the disable nodes
4) compute=mitaka to [upgrade_levels]
5) db sync
6) Start newton
7) Try to place any VMs, it will fail
8) See nova-compute.log on the mitaka nodes
Expected result
===============
Successful upgrade one half of cluster, then another half
Actual result
=============
Nova can't place any VMs.
Compute logs:
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task
[req-41e6df10-b33b-47f5-be0c-86793cbcae6e - - - - -] Error during
ComputeManager._sync_scheduler_instance_info
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task Traceback (most
recent call last):
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task File
"/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in
run_periodic_tasks
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task task(self,
context)
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1637, in
_sync_scheduler_instance_info
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task
use_slave=True)
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task File
"/usr/lib/python2.7/site-packages/oslo_versionedobjects/base.py", line 177, in
wrapper
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task args, kwargs)
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task File
"/usr/lib/python2.7/site-packages/nova/conductor/rpcapi.py", line 236, in
object_class_action_versions
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task args=args,
kwargs=kwargs)
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line 169, in
call
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task
retry=self.retry)
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task File
"/usr/lib/python2.7/site-packages/oslo_messaging/transport.py", line 97, in
_send
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task
timeout=timeout, retry=retry)
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task File
"/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line
464, in send
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task retry=retry)
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task File
"/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line
455, in _send
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task raise result
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task RemoteError:
Remote error: IncompatibleObjectVersion Version 2.1 of InstanceList is not
supported
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task [u'Traceback
(most recent call last):\n', u' File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 138,
in _dispatch_and_reply\n', u' File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 185,
in _dispatch\n', u' File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 127,
in _do_dispatch\n :param incoming: incoming message\n', u' File
"/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 92, in
object_class_action_versions\n objname, object_versions[objname])\n', u'
File "/usr/lib/python2.7/site-packages/oslo_versionedobjects/base.py", line
374, in obj_class_from_name\n supported=latest_ver)\n',
u'IncompatibleObjectVersion: Version 2.1 of InstanceList is not supported\n'].
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task
nova-conductor:
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher
[req-7a6eccb3-ade3-4c9f-b6f2-1588f9eb7ed0 - - - - -] Exception during message
handling: Version 2.1 of InstanceList is not supported
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher Traceback
(most recent call last):
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 138,
in _dispatch_and_reply
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 185,
in _dispatch
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 127,
in _do_dispatch
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher :param
incoming: incoming message
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 92, in
object_class_action_versions
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher objname,
object_versions[objname])
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/site-packages/oslo_versionedobjects/base.py", line 374, in
obj_class_from_name
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher
supported=latest_ver)
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher
IncompatibleObjectVersion: Version 2.1 of InstanceList is not supported
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher
2016-12-23 08:01:00.491 9958 ERROR oslo_messaging._drivers.common
[req-7a6eccb3-ade3-4c9f-b6f2-1588f9eb7ed0 - - - - -] Returning exception
Version 2.1 of InstanceList is not supported to caller
2016-12-23 08:01:00.492 9958 ERROR oslo_messaging._drivers.common
[req-7a6eccb3-ade3-4c9f-b6f2-1588f9eb7ed0 - - - - -] ['Traceback (most recent
call last):\n', ' File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 138,
in _dispatch_and_reply\n', ' File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 185,
in _dispatch\n', ' File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 127,
in _do_dispatch\n :param incoming: incoming message\n', ' File
"/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 92, in
object_class_action_versions\n objname, object_versions[objname])\n', '
File "/usr/lib/python2.7/site-packages/oslo_versionedobjects/base.py", line
374, in obj_class_from_name\n supported=latest_ver)\n',
'IncompatibleObjectVersion: Version 2.1 of InstanceList is not supported\n']
Environment
===========
CentOS Linux 7 (Core) 3.10.0-514.2.2.el7.x86_64 #1 SMP Tue Dec 6
23:06:41 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
mitaka nodes:
compute:
python2-novaclient-3.3.2-1.el7.noarch
openstack-nova-compute-13.1.2-1.el7.noarch
python-nova-13.1.2-1.el7.noarch
openstack-nova-common-13.1.2-1.el7.noarch
contoller:
openstack-nova-scheduler-13.1.2-1.el7.noarch
openstack-nova-conductor-13.1.2-1.el7.noarch
nova-solver-compute-lib-2016.2-2.noarch
python-nova-13.1.2-1.el7.noarch
openstack-nova-api-13.1.2-1.el7.noarch
novameter-2016.2-3.noarch
openstack-nova-common-13.1.2-1.el7.noarch
openstack-nova-console-13.1.2-1.el7.noarch
python2-novaclient-3.3.2-1.el7.noarch
openstack-nova-novncproxy-13.1.2-1.el7.noarch
netonw nodes:
compute:
python2-novaclient-3.3.2-1.el7.noarch
python-nova-14.0.2-1.el7.noarch
openstack-nova-common-14.0.2-1.el7.noarch
openstack-nova-compute-14.0.2-1.el7.noarch
controller:
openstack-nova-common-14.0.2-1.el7.noarch
openstack-nova-console-14.0.2-1.el7.noarch
novameter-2016.2-3.noarch
openstack-nova-conductor-14.0.2-1.el7.noarch
openstack-nova-novncproxy-14.0.2-1.el7.noarch
openstack-nova-api-14.0.2-1.el7.noarch
openstack-nova-scheduler-14.0.2-1.el7.noarch
python2-novaclient-3.3.2-1.el7.noarch
nova-solver-compute-lib-2016.2-2.noarch
python-nova-14.0.2-1.el7.noarch
** Affects: nova
Importance: Undecided
Status: New
** Description changed:
Description
===========
I get an error after upgrade half of my cluster. Can't place any VMs.
"RemoteError: Remote error: IncompatibleObjectVersion Version 2.1 of
InstanceList is not supported"
Steps to reproduce
==================
1) Install 4 nodes with mitaka
2) Disable 2 nodes (1 api controller and 1 compute): nova service-disable
- 3) Upgrade to newton on disable nodes
+ 3) Upgrade to newton on the disable nodes
4) compute=mitaka to [upgrade_levels]
5) db sync
6) Start newton
- 7) Try to place any VMs, it fill fail
+ 7) Try to place any VMs, it will fail
8) see nova-compute.log on the mitaka nodes
Expected result
===============
Successful upgrade one half of cluster, then another half
Actual result
=============
Nova can't place any VMs.
Compute logs:
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task
[req-41e6df10-b33b-47f5-be0c-86793cbcae6e - - - - -] Error during
ComputeManager._sync_scheduler_instance_info
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task Traceback
(most recent call last):
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task File
"/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in
run_periodic_tasks
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task task(self,
context)
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1637, in
_sync_scheduler_instance_info
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task
use_slave=True)
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task File
"/usr/lib/python2.7/site-packages/oslo_versionedobjects/base.py", line 177, in
wrapper
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task args,
kwargs)
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task File
"/usr/lib/python2.7/site-packages/nova/conductor/rpcapi.py", line 236, in
object_class_action_versions
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task args=args,
kwargs=kwargs)
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line 169, in
call
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task
retry=self.retry)
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task File
"/usr/lib/python2.7/site-packages/oslo_messaging/transport.py", line 97, in
_send
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task
timeout=timeout, retry=retry)
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task File
"/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line
464, in send
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task
retry=retry)
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task File
"/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line
455, in _send
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task raise
result
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task RemoteError:
Remote error: IncompatibleObjectVersion Version 2.1 of InstanceList is not
supported
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task [u'Traceback
(most recent call last):\n', u' File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 138,
in _dispatch_and_reply\n', u' File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 185,
in _dispatch\n', u' File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 127,
in _do_dispatch\n :param incoming: incoming message\n', u' File
"/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 92, in
object_class_action_versions\n objname, object_versions[objname])\n', u'
File "/usr/lib/python2.7/site-packages/oslo_versionedobjects/base.py", line
374, in obj_class_from_name\n supported=latest_ver)\n',
u'IncompatibleObjectVersion: Version 2.1 of InstanceList is not supported\n'].
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task
-
nova-conductor:
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher
[req-7a6eccb3-ade3-4c9f-b6f2-1588f9eb7ed0 - - - - -] Exception during message
handling: Version 2.1 of InstanceList is not supported
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher Traceback
(most recent call last):
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 138,
in _dispatch_and_reply
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 185,
in _dispatch
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 127,
in _do_dispatch
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher :param
incoming: incoming message
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 92, in
object_class_action_versions
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher objname,
object_versions[objname])
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/site-packages/oslo_versionedobjects/base.py", line 374, in
obj_class_from_name
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher
supported=latest_ver)
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher
IncompatibleObjectVersion: Version 2.1 of InstanceList is not supported
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher
2016-12-23 08:01:00.491 9958 ERROR oslo_messaging._drivers.common
[req-7a6eccb3-ade3-4c9f-b6f2-1588f9eb7ed0 - - - - -] Returning exception
Version 2.1 of InstanceList is not supported to caller
2016-12-23 08:01:00.492 9958 ERROR oslo_messaging._drivers.common
[req-7a6eccb3-ade3-4c9f-b6f2-1588f9eb7ed0 - - - - -] ['Traceback (most recent
call last):\n', ' File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 138,
in _dispatch_and_reply\n', ' File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 185,
in _dispatch\n', ' File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 127,
in _do_dispatch\n :param incoming: incoming message\n', ' File
"/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 92, in
object_class_action_versions\n objname, object_versions[objname])\n', '
File "/usr/lib/python2.7/site-packages/oslo_versionedobjects/base.py", line
374, in obj_class_from_name\n supported=latest_ver)\n',
'IncompatibleObjectVersion: Version 2.1 of InstanceList is not supported\n']
-
Environment
===========
CentOS Linux 7 (Core) 3.10.0-514.2.2.el7.x86_64 #1 SMP Tue Dec 6
23:06:41 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
mitaka nodes:
compute:
python2-novaclient-3.3.2-1.el7.noarch
openstack-nova-compute-13.1.2-1.el7.noarch
python-nova-13.1.2-1.el7.noarch
openstack-nova-common-13.1.2-1.el7.noarch
contoller:
openstack-nova-scheduler-13.1.2-1.el7.noarch
openstack-nova-conductor-13.1.2-1.el7.noarch
nova-solver-compute-lib-2016.2-2.noarch
python-nova-13.1.2-1.el7.noarch
openstack-nova-api-13.1.2-1.el7.noarch
novameter-2016.2-3.noarch
openstack-nova-common-13.1.2-1.el7.noarch
openstack-nova-console-13.1.2-1.el7.noarch
python2-novaclient-3.3.2-1.el7.noarch
openstack-nova-novncproxy-13.1.2-1.el7.noarch
netonw nodes:
compute:
python2-novaclient-3.3.2-1.el7.noarch
python-nova-14.0.2-1.el7.noarch
openstack-nova-common-14.0.2-1.el7.noarch
openstack-nova-compute-14.0.2-1.el7.noarch
controller:
openstack-nova-common-14.0.2-1.el7.noarch
openstack-nova-console-14.0.2-1.el7.noarch
novameter-2016.2-3.noarch
openstack-nova-conductor-14.0.2-1.el7.noarch
openstack-nova-novncproxy-14.0.2-1.el7.noarch
openstack-nova-api-14.0.2-1.el7.noarch
openstack-nova-scheduler-14.0.2-1.el7.noarch
python2-novaclient-3.3.2-1.el7.noarch
nova-solver-compute-lib-2016.2-2.noarch
python-nova-14.0.2-1.el7.noarch
** Description changed:
Description
===========
I get an error after upgrade half of my cluster. Can't place any VMs.
"RemoteError: Remote error: IncompatibleObjectVersion Version 2.1 of
InstanceList is not supported"
Steps to reproduce
==================
1) Install 4 nodes with mitaka
2) Disable 2 nodes (1 api controller and 1 compute): nova service-disable
3) Upgrade to newton on the disable nodes
4) compute=mitaka to [upgrade_levels]
5) db sync
6) Start newton
7) Try to place any VMs, it will fail
- 8) see nova-compute.log on the mitaka nodes
+ 8) See nova-compute.log on the mitaka nodes
Expected result
===============
Successful upgrade one half of cluster, then another half
Actual result
=============
Nova can't place any VMs.
Compute logs:
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task
[req-41e6df10-b33b-47f5-be0c-86793cbcae6e - - - - -] Error during
ComputeManager._sync_scheduler_instance_info
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task Traceback
(most recent call last):
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task File
"/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in
run_periodic_tasks
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task task(self,
context)
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1637, in
_sync_scheduler_instance_info
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task
use_slave=True)
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task File
"/usr/lib/python2.7/site-packages/oslo_versionedobjects/base.py", line 177, in
wrapper
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task args,
kwargs)
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task File
"/usr/lib/python2.7/site-packages/nova/conductor/rpcapi.py", line 236, in
object_class_action_versions
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task args=args,
kwargs=kwargs)
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line 169, in
call
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task
retry=self.retry)
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task File
"/usr/lib/python2.7/site-packages/oslo_messaging/transport.py", line 97, in
_send
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task
timeout=timeout, retry=retry)
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task File
"/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line
464, in send
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task
retry=retry)
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task File
"/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line
455, in _send
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task raise
result
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task RemoteError:
Remote error: IncompatibleObjectVersion Version 2.1 of InstanceList is not
supported
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task [u'Traceback
(most recent call last):\n', u' File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 138,
in _dispatch_and_reply\n', u' File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 185,
in _dispatch\n', u' File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 127,
in _do_dispatch\n :param incoming: incoming message\n', u' File
"/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 92, in
object_class_action_versions\n objname, object_versions[objname])\n', u'
File "/usr/lib/python2.7/site-packages/oslo_versionedobjects/base.py", line
374, in obj_class_from_name\n supported=latest_ver)\n',
u'IncompatibleObjectVersion: Version 2.1 of InstanceList is not supported\n'].
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task
nova-conductor:
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher
[req-7a6eccb3-ade3-4c9f-b6f2-1588f9eb7ed0 - - - - -] Exception during message
handling: Version 2.1 of InstanceList is not supported
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher Traceback
(most recent call last):
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 138,
in _dispatch_and_reply
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 185,
in _dispatch
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 127,
in _do_dispatch
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher :param
incoming: incoming message
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 92, in
object_class_action_versions
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher objname,
object_versions[objname])
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/site-packages/oslo_versionedobjects/base.py", line 374, in
obj_class_from_name
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher
supported=latest_ver)
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher
IncompatibleObjectVersion: Version 2.1 of InstanceList is not supported
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher
2016-12-23 08:01:00.491 9958 ERROR oslo_messaging._drivers.common
[req-7a6eccb3-ade3-4c9f-b6f2-1588f9eb7ed0 - - - - -] Returning exception
Version 2.1 of InstanceList is not supported to caller
2016-12-23 08:01:00.492 9958 ERROR oslo_messaging._drivers.common
[req-7a6eccb3-ade3-4c9f-b6f2-1588f9eb7ed0 - - - - -] ['Traceback (most recent
call last):\n', ' File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 138,
in _dispatch_and_reply\n', ' File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 185,
in _dispatch\n', ' File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 127,
in _do_dispatch\n :param incoming: incoming message\n', ' File
"/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 92, in
object_class_action_versions\n objname, object_versions[objname])\n', '
File "/usr/lib/python2.7/site-packages/oslo_versionedobjects/base.py", line
374, in obj_class_from_name\n supported=latest_ver)\n',
'IncompatibleObjectVersion: Version 2.1 of InstanceList is not supported\n']
Environment
===========
CentOS Linux 7 (Core) 3.10.0-514.2.2.el7.x86_64 #1 SMP Tue Dec 6
23:06:41 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
mitaka nodes:
compute:
python2-novaclient-3.3.2-1.el7.noarch
openstack-nova-compute-13.1.2-1.el7.noarch
python-nova-13.1.2-1.el7.noarch
openstack-nova-common-13.1.2-1.el7.noarch
contoller:
openstack-nova-scheduler-13.1.2-1.el7.noarch
openstack-nova-conductor-13.1.2-1.el7.noarch
nova-solver-compute-lib-2016.2-2.noarch
python-nova-13.1.2-1.el7.noarch
openstack-nova-api-13.1.2-1.el7.noarch
novameter-2016.2-3.noarch
openstack-nova-common-13.1.2-1.el7.noarch
openstack-nova-console-13.1.2-1.el7.noarch
python2-novaclient-3.3.2-1.el7.noarch
openstack-nova-novncproxy-13.1.2-1.el7.noarch
netonw nodes:
compute:
python2-novaclient-3.3.2-1.el7.noarch
python-nova-14.0.2-1.el7.noarch
openstack-nova-common-14.0.2-1.el7.noarch
openstack-nova-compute-14.0.2-1.el7.noarch
controller:
openstack-nova-common-14.0.2-1.el7.noarch
openstack-nova-console-14.0.2-1.el7.noarch
novameter-2016.2-3.noarch
openstack-nova-conductor-14.0.2-1.el7.noarch
openstack-nova-novncproxy-14.0.2-1.el7.noarch
openstack-nova-api-14.0.2-1.el7.noarch
openstack-nova-scheduler-14.0.2-1.el7.noarch
python2-novaclient-3.3.2-1.el7.noarch
nova-solver-compute-lib-2016.2-2.noarch
python-nova-14.0.2-1.el7.noarch
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1652233
Title:
mitaka is incompatible with newton
Status in OpenStack Compute (nova):
New
Bug description:
Description
===========
I get an error after upgrade half of my cluster. Can't place any VMs.
"RemoteError: Remote error: IncompatibleObjectVersion Version 2.1 of
InstanceList is not supported"
Steps to reproduce
==================
1) Install 4 nodes with mitaka
2) Disable 2 nodes (1 api controller and 1 compute): nova service-disable
3) Upgrade to newton on the disable nodes
4) compute=mitaka to [upgrade_levels]
5) db sync
6) Start newton
7) Try to place any VMs, it will fail
8) See nova-compute.log on the mitaka nodes
Expected result
===============
Successful upgrade one half of cluster, then another half
Actual result
=============
Nova can't place any VMs.
Compute logs:
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task
[req-41e6df10-b33b-47f5-be0c-86793cbcae6e - - - - -] Error during
ComputeManager._sync_scheduler_instance_info
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task Traceback
(most recent call last):
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task File
"/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in
run_periodic_tasks
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task task(self,
context)
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1637, in
_sync_scheduler_instance_info
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task
use_slave=True)
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task File
"/usr/lib/python2.7/site-packages/oslo_versionedobjects/base.py", line 177, in
wrapper
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task args,
kwargs)
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task File
"/usr/lib/python2.7/site-packages/nova/conductor/rpcapi.py", line 236, in
object_class_action_versions
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task args=args,
kwargs=kwargs)
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line 169, in
call
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task
retry=self.retry)
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task File
"/usr/lib/python2.7/site-packages/oslo_messaging/transport.py", line 97, in
_send
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task
timeout=timeout, retry=retry)
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task File
"/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line
464, in send
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task
retry=retry)
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task File
"/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line
455, in _send
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task raise
result
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task RemoteError:
Remote error: IncompatibleObjectVersion Version 2.1 of InstanceList is not
supported
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task [u'Traceback
(most recent call last):\n', u' File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 138,
in _dispatch_and_reply\n', u' File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 185,
in _dispatch\n', u' File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 127,
in _do_dispatch\n :param incoming: incoming message\n', u' File
"/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 92, in
object_class_action_versions\n objname, object_versions[objname])\n', u'
File "/usr/lib/python2.7/site-packages/oslo_versionedobjects/base.py", line
374, in obj_class_from_name\n supported=latest_ver)\n',
u'IncompatibleObjectVersion: Version 2.1 of InstanceList is not supported\n'].
2016-12-23 07:26:11.434 15392 ERROR oslo_service.periodic_task
nova-conductor:
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher
[req-7a6eccb3-ade3-4c9f-b6f2-1588f9eb7ed0 - - - - -] Exception during message
handling: Version 2.1 of InstanceList is not supported
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher Traceback
(most recent call last):
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 138,
in _dispatch_and_reply
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 185,
in _dispatch
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 127,
in _do_dispatch
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher :param
incoming: incoming message
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 92, in
object_class_action_versions
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher objname,
object_versions[objname])
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/site-packages/oslo_versionedobjects/base.py", line 374, in
obj_class_from_name
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher
supported=latest_ver)
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher
IncompatibleObjectVersion: Version 2.1 of InstanceList is not supported
2016-12-23 08:01:00.489 9958 ERROR oslo_messaging.rpc.dispatcher
2016-12-23 08:01:00.491 9958 ERROR oslo_messaging._drivers.common
[req-7a6eccb3-ade3-4c9f-b6f2-1588f9eb7ed0 - - - - -] Returning exception
Version 2.1 of InstanceList is not supported to caller
2016-12-23 08:01:00.492 9958 ERROR oslo_messaging._drivers.common
[req-7a6eccb3-ade3-4c9f-b6f2-1588f9eb7ed0 - - - - -] ['Traceback (most recent
call last):\n', ' File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 138,
in _dispatch_and_reply\n', ' File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 185,
in _dispatch\n', ' File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 127,
in _do_dispatch\n :param incoming: incoming message\n', ' File
"/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 92, in
object_class_action_versions\n objname, object_versions[objname])\n', '
File "/usr/lib/python2.7/site-packages/oslo_versionedobjects/base.py", line
374, in obj_class_from_name\n supported=latest_ver)\n',
'IncompatibleObjectVersion: Version 2.1 of InstanceList is not supported\n']
Environment
===========
CentOS Linux 7 (Core) 3.10.0-514.2.2.el7.x86_64 #1 SMP Tue Dec 6
23:06:41 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
mitaka nodes:
compute:
python2-novaclient-3.3.2-1.el7.noarch
openstack-nova-compute-13.1.2-1.el7.noarch
python-nova-13.1.2-1.el7.noarch
openstack-nova-common-13.1.2-1.el7.noarch
contoller:
openstack-nova-scheduler-13.1.2-1.el7.noarch
openstack-nova-conductor-13.1.2-1.el7.noarch
nova-solver-compute-lib-2016.2-2.noarch
python-nova-13.1.2-1.el7.noarch
openstack-nova-api-13.1.2-1.el7.noarch
novameter-2016.2-3.noarch
openstack-nova-common-13.1.2-1.el7.noarch
openstack-nova-console-13.1.2-1.el7.noarch
python2-novaclient-3.3.2-1.el7.noarch
openstack-nova-novncproxy-13.1.2-1.el7.noarch
netonw nodes:
compute:
python2-novaclient-3.3.2-1.el7.noarch
python-nova-14.0.2-1.el7.noarch
openstack-nova-common-14.0.2-1.el7.noarch
openstack-nova-compute-14.0.2-1.el7.noarch
controller:
openstack-nova-common-14.0.2-1.el7.noarch
openstack-nova-console-14.0.2-1.el7.noarch
novameter-2016.2-3.noarch
openstack-nova-conductor-14.0.2-1.el7.noarch
openstack-nova-novncproxy-14.0.2-1.el7.noarch
openstack-nova-api-14.0.2-1.el7.noarch
openstack-nova-scheduler-14.0.2-1.el7.noarch
python2-novaclient-3.3.2-1.el7.noarch
nova-solver-compute-lib-2016.2-2.noarch
python-nova-14.0.2-1.el7.noarch
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1652233/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
