Reviewed: https://review.openstack.org/615347 Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=097693aaa3f57011e38188a94687e02bc9b074ab Submitter: Zuul Branch: master
commit 097693aaa3f57011e38188a94687e02bc9b074ab Author: Matt Riedemann <[email protected]> Date: Fri Nov 2 17:36:41 2018 -0400 Drop pre-cellsv2 compat in compute API.get() This removes the nova-osapi_compute service version compatibility check for cells v2 which was added in Newton. A "nova-status upgrade check" was added for that in Rocky: Ie2bc4616439352850cf29a9de7d33a06c8f7c2b8 And it is also causing issues with functional tests where instances are buried in cell0 and because we don't use the instance mapping, we fail to find the instance. As a result of this, compute API tests that rely on retrieving a real instance out of the database will also need to make sure an associated instance mapping record exists. Change-Id: I11083aa3c78bd8b6201558561457f3d65007a177 Closes-Bug: #1800472 ** Changed in: nova Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1800472 Title: nova.tests.functional.test_server_group.ServerGroupTestV264.test_boot_servers_with_affinity_no_valid_host intermittently failing with "OpenStackApiNotFoundException: Item not found" Status in OpenStack Compute (nova): Fix Released Bug description: http://logs.openstack.org/48/613348/1/check/nova-tox- functional/16b5d01/job-output.txt.gz#_2018-10-29_10_14_46_484703 2018-10-29 10:14:46.550516 | ubuntu-xenial | 2018-10-29 10:14:45,457 INFO [nova.api.openstack.requestlog] 127.0.0.1 "POST /v2.1/6f70656e737461636b20342065766572/servers" status: 202 len: 480 microversion: 2.64 time: 0.182153 2018-10-29 10:14:46.550841 | ubuntu-xenial | 2018-10-29 10:14:45,485 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/6f70656e737461636b20342065766572/servers/e5f8520d-9f2f-4895-bcf8-485e50898bd0" status: 200 len: 1806 microversion: 2.64 time: 0.023981 2018-10-29 10:14:46.551172 | ubuntu-xenial | 2018-10-29 10:14:45,639 INFO [nova.api.openstack.placement.requestlog] 127.0.0.1 "GET /placement/allocation_candidates?limit=1000&resources=DISK_GB%3A40%2CMEMORY_MB%3A4096%2CVCPU%3A2" status: 200 len: 467 microversion: 1.29 2018-10-29 10:14:46.551333 | ubuntu-xenial | 2018-10-29 10:14:45,657 INFO [nova.filters] Filter ServerGroupAffinityFilter returned 0 hosts 2018-10-29 10:14:46.552003 | ubuntu-xenial | 2018-10-29 10:14:45,657 INFO [nova.filters] Filtering removed all hosts for the request with instance ID 'e5f8520d-9f2f-4895-bcf8-485e50898bd0'. Filter results: ['RetryFilter: (start: 1, end: 1)', 'AvailabilityZoneFilter: (start: 1, end: 1)', 'ComputeFilter: (start: 1, end: 1)', 'ComputeCapabilitiesFilter: (start: 1, end: 1)', 'ImagePropertiesFilter: (start: 1, end: 1)', 'ServerGroupAntiAffinityFilter: (start: 1, end: 1)', 'ServerGroupAffinityFilter: (start: 1, end: 0)'] 2018-10-29 10:14:46.552140 | ubuntu-xenial | 2018-10-29 10:14:45,658 ERROR [nova.conductor.manager] Failed to schedule instances 2018-10-29 10:14:46.552209 | ubuntu-xenial | Traceback (most recent call last): 2018-10-29 10:14:46.552345 | ubuntu-xenial | File "nova/conductor/manager.py", line 1255, in schedule_and_build_instances 2018-10-29 10:14:46.552423 | ubuntu-xenial | instance_uuids, return_alternates=True) 2018-10-29 10:14:46.552545 | ubuntu-xenial | File "nova/conductor/manager.py", line 750, in _schedule_instances 2018-10-29 10:14:46.552626 | ubuntu-xenial | return_alternates=return_alternates) 2018-10-29 10:14:46.552740 | ubuntu-xenial | File "nova/scheduler/utils.py", line 953, in wrapped 2018-10-29 10:14:46.552810 | ubuntu-xenial | return func(*args, **kwargs) 2018-10-29 10:14:46.552942 | ubuntu-xenial | File "nova/scheduler/client/__init__.py", line 53, in select_destinations 2018-10-29 10:14:46.553051 | ubuntu-xenial | instance_uuids, return_objects, return_alternates) 2018-10-29 10:14:46.553167 | ubuntu-xenial | File "nova/scheduler/client/__init__.py", line 37, in __run_method 2018-10-29 10:14:46.553268 | ubuntu-xenial | return getattr(self.instance, __name)(*args, **kwargs) 2018-10-29 10:14:46.553387 | ubuntu-xenial | File "nova/scheduler/client/query.py", line 42, in select_destinations 2018-10-29 10:14:46.553483 | ubuntu-xenial | instance_uuids, return_objects, return_alternates) 2018-10-29 10:14:46.553596 | ubuntu-xenial | File "nova/scheduler/rpcapi.py", line 158, in select_destinations 2018-10-29 10:14:46.553703 | ubuntu-xenial | return cctxt.call(ctxt, 'select_destinations', **msg_args) 2018-10-29 10:14:46.553934 | ubuntu-xenial | File "/home/zuul/src/git.openstack.org/openstack/nova/.tox/py27/local/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line 179, in call 2018-10-29 10:14:46.553995 | ubuntu-xenial | retry=self.retry) 2018-10-29 10:14:46.554235 | ubuntu-xenial | File "/home/zuul/src/git.openstack.org/openstack/nova/.tox/py27/local/lib/python2.7/site-packages/oslo_messaging/transport.py", line 128, in _send 2018-10-29 10:14:46.554280 | ubuntu-xenial | retry=retry) 2018-10-29 10:14:46.554693 | ubuntu-xenial | File "/home/zuul/src/git.openstack.org/openstack/nova/.tox/py27/local/lib/python2.7/site-packages/oslo_messaging/_drivers/impl_fake.py", line 222, in send 2018-10-29 10:14:46.554906 | ubuntu-xenial | return self._send(target, ctxt, message, wait_for_reply, timeout) 2018-10-29 10:14:46.555228 | ubuntu-xenial | File "/home/zuul/src/git.openstack.org/openstack/nova/.tox/py27/local/lib/python2.7/site-packages/oslo_messaging/_drivers/impl_fake.py", line 209, in _send 2018-10-29 10:14:46.555319 | ubuntu-xenial | raise failure 2018-10-29 10:14:46.555485 | ubuntu-xenial | NoValidHost: No valid host was found. There are not enough hosts available. 2018-10-29 10:14:46.555816 | ubuntu-xenial | 2018-10-29 10:14:45,697 WARNING [nova.scheduler.utils] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. 2018-10-29 10:14:46.556006 | ubuntu-xenial | 2018-10-29 10:14:45,698 WARNING [nova.scheduler.utils] Setting instance to ERROR state. Looks like the test creates two servers in an affinity group and then tries to create a third server in the group and expects it to fail because there are not enough resources on the host with the other two servers. That all seems to work. The problem is when creating the 3rd server, while waiting for its status to change to ERROR the API returns a 404: 2018-10-29 10:14:46.484852 | ubuntu-xenial | Captured traceback: 2018-10-29 10:14:46.484897 | ubuntu-xenial | ~~~~~~~~~~~~~~~~~~~ 2018-10-29 10:14:46.484967 | ubuntu-xenial | Traceback (most recent call last): 2018-10-29 10:14:46.485159 | ubuntu-xenial | File "nova/tests/functional/test_server_group.py", line 303, in test_boot_servers_with_affinity_no_valid_host 2018-10-29 10:14:46.485247 | ubuntu-xenial | expected_status='ERROR') 2018-10-29 10:14:46.485394 | ubuntu-xenial | File "nova/tests/functional/test_server_group.py", line 112, in _boot_a_server_to_group 2018-10-29 10:14:46.485490 | ubuntu-xenial | self.admin_api, created_server, expected_status) 2018-10-29 10:14:46.485633 | ubuntu-xenial | File "nova/tests/functional/integrated_helpers.py", line 259, in _wait_for_state_change 2018-10-29 10:14:46.485743 | ubuntu-xenial | admin_api, server, {'status': expected_status}, max_retries) 2018-10-29 10:14:46.485892 | ubuntu-xenial | File "nova/tests/functional/integrated_helpers.py", line 243, in _wait_for_server_parameter 2018-10-29 10:14:46.485980 | ubuntu-xenial | server = admin_api.get_server(server['id']) 2018-10-29 10:14:46.486104 | ubuntu-xenial | File "nova/tests/functional/api/client.py", line 268, in get_server 2018-10-29 10:14:46.486218 | ubuntu-xenial | return self.api_get('/servers/%s' % server_id).body['server'] 2018-10-29 10:14:46.486330 | ubuntu-xenial | File "nova/tests/functional/api/client.py", line 225, in api_get 2018-10-29 10:14:46.486439 | ubuntu-xenial | return APIResponse(self.api_request(relative_uri, **kwargs)) 2018-10-29 10:14:46.486557 | ubuntu-xenial | File "nova/tests/functional/api/client.py", line 207, in api_request 2018-10-29 10:14:46.486671 | ubuntu-xenial | raise OpenStackApiNotFoundException(response=response) 2018-10-29 10:14:46.486811 | ubuntu-xenial | nova.tests.functional.api.client.OpenStackApiNotFoundException: Item not found My guess would be something in this recent change is causing a problem looking up the instance from the cell0 DB: https://review.openstack.org/#/c/611665/ To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1800472/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
