Public bug reported: Seen here:
https://748441c9eeeb89eb36db- b17eeb704755458252956ad3afd603fe.ssl.cf5.rackcdn.com/697670/3/gate/nova- tox-functional-py36/54fe49d/testr_results.html.gz ft1.1: nova.tests.functional.libvirt.test_vpmem.VPMEMTests.test_create_servers_with_vpmemtesttools.testresult.real._StringException: pythonlogging:'': {{{ 2019-12-18 17:05:18,534 WARNING [placement.db_api] TransactionFactory already started, not reconfiguring. 2019-12-18 17:05:18,885 INFO [nova.service] Starting conductor node (version 20.1.0) 2019-12-18 17:05:18,917 INFO [nova.service] Starting scheduler node (version 20.1.0) 2019-12-18 17:05:18,981 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/flavors/detail" status: 200 len: 2605 microversion: 2.81 time: 0.020144 2019-12-18 17:05:18,996 INFO [nova.api.openstack.requestlog] 127.0.0.1 "POST /v2.1/flavors" status: 200 len: 464 microversion: 2.81 time: 0.012445 2019-12-18 17:05:19,018 INFO [nova.api.openstack.requestlog] 127.0.0.1 "POST /v2.1/flavors/99512851/os-extra_specs" status: 200 len: 37 microversion: 2.81 time: 0.018592 2019-12-18 17:05:19,035 INFO [nova.virt.driver] Loading compute driver 'libvirt.LibvirtDriver' 2019-12-18 17:05:19,038 WARNING [os_brick.initiator.connectors.remotefs] Connection details not present. RemoteFsClient may not initialize properly. 2019-12-18 17:05:19,040 INFO [nova.service] Starting compute node (version 20.1.0) 2019-12-18 17:05:19,058 WARNING [nova.compute.manager] Compute node host1 not found in the database. If this is the first time this service is starting on this host, then you can ignore this warning. 2019-12-18 17:05:19,063 INFO [nova.compute.manager] Looking for unclaimed instances stuck in BUILDING status for nodes managed by this host 2019-12-18 17:05:19,073 WARNING [nova.compute.manager] No compute node record found for host host1. If this is the first time this service is starting on this host, then you can ignore this warning. 2019-12-18 17:05:19,082 WARNING [nova.compute.resource_tracker] No compute node record for host1:host1 2019-12-18 17:05:19,086 INFO [nova.compute.resource_tracker] Compute node record created for host1:host1 with uuid: 56a25de3-2d91-48a3-9e77-a66f1b5d65b5 2019-12-18 17:05:19,123 INFO [placement.requestlog] 127.0.0.1 "GET /placement/resource_providers?in_tree=56a25de3-2d91-48a3-9e77-a66f1b5d65b5" status: 200 len: 26 microversion: 1.14 2019-12-18 17:05:19,131 INFO [placement.requestlog] 127.0.0.1 "POST /placement/resource_providers" status: 200 len: 828 microversion: 1.20 2019-12-18 17:05:19,132 INFO [nova.scheduler.client.report] [req-54d75b29-ade3-4f2a-8911-d1147ef449fc] Created resource provider record via placement API for resource provider with UUID 56a25de3-2d91-48a3-9e77-a66f1b5d65b5 and name host1. 2019-12-18 17:05:19,133 INFO [nova.virt.libvirt.host] kernel doesn't support AMD SEV 2019-12-18 17:05:19,149 INFO [placement.requestlog] 127.0.0.1 "PUT /placement/resource_classes/CUSTOM_PMEM_NAMESPACE_SMALL" status: 201 len: 0 microversion: 1.7 2019-12-18 17:05:19,159 INFO [placement.requestlog] 127.0.0.1 "PUT /placement/resource_classes/CUSTOM_PMEM_NAMESPACE_4GB" status: 201 len: 0 microversion: 1.7 2019-12-18 17:05:19,173 INFO [placement.requestlog] 127.0.0.1 "PUT /placement/resource_providers/56a25de3-2d91-48a3-9e77-a66f1b5d65b5/inventories" status: 200 len: 665 microversion: 1.26 2019-12-18 17:05:19,181 INFO [placement.requestlog] 127.0.0.1 "GET /placement/traits?name=in:COMPUTE_NET_ATTACH_INTERFACE_WITH_TAG,COMPUTE_IMAGE_TYPE_AMI,HW_CPU_X86_VMX,HW_CPU_X86_AESNI,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_IMAGE_TYPE_AKI,COMPUTE_IMAGE_TYPE_ISO,COMPUTE_VOLUME_ATTACH_WITH_TAG,COMPUTE_IMAGE_TYPE_ARI,COMPUTE_IMAGE_TYPE_RAW,COMPUTE_DEVICE_TAGGING,HW_CPU_HYPERTHREADING,COMPUTE_VOLUME_EXTEND,COMPUTE_IMAGE_TYPE_QCOW2,COMPUTE_TRUSTED_CERTS,COMPUTE_NODE" status: 200 len: 432 microversion: 1.6 2019-12-18 17:05:19,200 INFO [placement.requestlog] 127.0.0.1 "PUT /placement/resource_providers/56a25de3-2d91-48a3-9e77-a66f1b5d65b5/traits" status: 200 len: 467 microversion: 1.6 2019-12-18 17:05:19,224 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/os-hypervisors?hypervisor_hostname_pattern=host1" status: 200 len: 133 microversion: 2.81 time: 0.019077 2019-12-18 17:05:19,236 INFO [placement.requestlog] 127.0.0.1 "GET /placement/resource_providers/56a25de3-2d91-48a3-9e77-a66f1b5d65b5/inventories" status: 200 len: 665 microversion: 1.0 2019-12-18 17:05:19,253 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/os-hypervisors?hypervisor_hostname_pattern=host1" status: 200 len: 133 microversion: 2.81 time: 0.014420 2019-12-18 17:05:19,255 INFO [nova.tests.functional.libvirt.test_vpmem] booting on host1 2019-12-18 17:05:19,329 INFO [nova.api.openstack.requestlog] 127.0.0.1 "POST /v2.1/servers" status: 202 len: 414 microversion: 2.81 time: 0.071830 2019-12-18 17:05:19,352 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/3a085e4e-52ba-49cc-be7c-6eca2974765a" status: 200 len: 1778 microversion: 2.81 time: 0.019663 2019-12-18 17:05:19,473 INFO [placement.requestlog] 127.0.0.1 "GET /placement/allocation_candidates?in_tree=56a25de3-2d91-48a3-9e77-a66f1b5d65b5&limit=1000&required=%21COMPUTE_STATUS_DISABLED&resources=CUSTOM_PMEM_NAMESPACE_SMALL%3A1%2CDISK_GB%3A1%2CMEMORY_MB%3A1024%2CVCPU%3A1" status: 200 len: 1087 microversion: 1.34 2019-12-18 17:05:19,495 INFO [nova.scheduler.host_manager] Host filter forcing available hosts to host1 2019-12-18 17:05:19,503 INFO [placement.requestlog] 127.0.0.1 "GET /placement/allocations/3a085e4e-52ba-49cc-be7c-6eca2974765a" status: 200 len: 19 microversion: 1.28 2019-12-18 17:05:19,530 INFO [placement.requestlog] 127.0.0.1 "PUT /placement/allocations/3a085e4e-52ba-49cc-be7c-6eca2974765a" status: 204 len: 0 microversion: 1.34 2019-12-18 17:05:19,724 INFO [placement.requestlog] 127.0.0.1 "GET /placement/allocations/3a085e4e-52ba-49cc-be7c-6eca2974765a" status: 200 len: 265 microversion: 1.28 2019-12-18 17:05:19,731 INFO [nova.compute.claims] Claim successful on node host1 2019-12-18 17:05:19,985 INFO [nova.compute.manager] Took 0.00 seconds to spawn the instance on the hypervisor. 2019-12-18 17:05:20,077 INFO [nova.compute.manager] Took 0.36 seconds to build instance. 2019-12-18 17:05:20,087 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/3a085e4e-52ba-49cc-be7c-6eca2974765a" status: 200 len: 1875 microversion: 2.81 time: 0.227589 2019-12-18 17:05:20,122 INFO [nova.scheduler.host_manager] Received an update from an unknown host 'host1'. Re-created its InstanceList. 2019-12-18 17:05:20,679 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/3a085e4e-52ba-49cc-be7c-6eca2974765a" status: 200 len: 1880 microversion: 2.81 time: 0.087029 2019-12-18 17:05:20,690 INFO [placement.requestlog] 127.0.0.1 "GET /placement/allocations/3a085e4e-52ba-49cc-be7c-6eca2974765a" status: 200 len: 169 microversion: 1.0 2019-12-18 17:05:20,691 INFO [nova.tests.functional.libvirt.test_vpmem] booting on host1 2019-12-18 17:05:20,766 INFO [nova.api.openstack.requestlog] 127.0.0.1 "POST /v2.1/servers" status: 202 len: 414 microversion: 2.81 time: 0.072347 2019-12-18 17:05:20,786 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/68f4cbad-7c69-492a-a173-6869dfa312d7" status: 200 len: 1778 microversion: 2.81 time: 0.017267 2019-12-18 17:05:20,902 INFO [placement.requestlog] 127.0.0.1 "GET /placement/allocation_candidates?in_tree=56a25de3-2d91-48a3-9e77-a66f1b5d65b5&limit=1000&required=%21COMPUTE_STATUS_DISABLED&resources=CUSTOM_PMEM_NAMESPACE_SMALL%3A1%2CDISK_GB%3A1%2CMEMORY_MB%3A1024%2CVCPU%3A1" status: 200 len: 1090 microversion: 1.34 2019-12-18 17:05:20,922 INFO [nova.scheduler.host_manager] Host filter forcing available hosts to host1 2019-12-18 17:05:20,930 INFO [placement.requestlog] 127.0.0.1 "GET /placement/allocations/68f4cbad-7c69-492a-a173-6869dfa312d7" status: 200 len: 19 microversion: 1.28 2019-12-18 17:05:20,955 INFO [placement.requestlog] 127.0.0.1 "PUT /placement/allocations/68f4cbad-7c69-492a-a173-6869dfa312d7" status: 204 len: 0 microversion: 1.34 2019-12-18 17:05:21,131 INFO [placement.requestlog] 127.0.0.1 "GET /placement/allocations/68f4cbad-7c69-492a-a173-6869dfa312d7" status: 200 len: 265 microversion: 1.28 2019-12-18 17:05:21,138 INFO [nova.compute.claims] Claim successful on node host1 2019-12-18 17:05:21,391 INFO [nova.compute.manager] Took 0.00 seconds to spawn the instance on the hypervisor. 2019-12-18 17:05:21,450 INFO [nova.compute.manager] Took 0.33 seconds to build instance. 2019-12-18 17:05:21,490 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/68f4cbad-7c69-492a-a173-6869dfa312d7" status: 200 len: 1863 microversion: 2.81 time: 0.183892 2019-12-18 17:05:22,075 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/68f4cbad-7c69-492a-a173-6869dfa312d7" status: 200 len: 1880 microversion: 2.81 time: 0.080401 2019-12-18 17:05:22,084 INFO [placement.requestlog] 127.0.0.1 "GET /placement/allocations/68f4cbad-7c69-492a-a173-6869dfa312d7" status: 200 len: 169 microversion: 1.0 2019-12-18 17:05:22,085 INFO [nova.tests.functional.libvirt.test_vpmem] booting on host1 2019-12-18 17:05:22,162 INFO [nova.api.openstack.requestlog] 127.0.0.1 "POST /v2.1/servers" status: 202 len: 414 microversion: 2.81 time: 0.074114 2019-12-18 17:05:22,183 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/afeb21d7-a89d-40ea-ac6e-28a7b80cec95" status: 200 len: 1778 microversion: 2.81 time: 0.018034 2019-12-18 17:05:22,283 INFO [placement.requestlog] 127.0.0.1 "GET /placement/allocation_candidates?in_tree=56a25de3-2d91-48a3-9e77-a66f1b5d65b5&limit=1000&required=%21COMPUTE_STATUS_DISABLED&resources=CUSTOM_PMEM_NAMESPACE_SMALL%3A1%2CDISK_GB%3A1%2CMEMORY_MB%3A1024%2CVCPU%3A1" status: 200 len: 53 microversion: 1.34 2019-12-18 17:05:22,284 INFO [nova.scheduler.manager] Got no allocation candidates from the Placement API. This could be due to insufficient resources or a temporary occurrence as compute nodes start up. 2019-12-18 17:05:22,284 ERROR [nova.conductor.manager] Failed to schedule instances Traceback (most recent call last): File "/home/zuul/src/opendev.org/openstack/nova/nova/conductor/manager.py", line 1412, in schedule_and_build_instances instance_uuids, return_alternates=True) File "/home/zuul/src/opendev.org/openstack/nova/nova/conductor/manager.py", line 855, in _schedule_instances return_alternates=return_alternates) File "/home/zuul/src/opendev.org/openstack/nova/nova/scheduler/client/query.py", line 42, in select_destinations instance_uuids, return_objects, return_alternates) File "/home/zuul/src/opendev.org/openstack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations return cctxt.call(ctxt, 'select_destinations', **msg_args) File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py36/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 181, in call transport_options=self.transport_options) File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py36/lib/python3.6/site-packages/oslo_messaging/transport.py", line 129, in _send transport_options=transport_options) File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py36/lib/python3.6/site-packages/oslo_messaging/_drivers/impl_fake.py", line 222, in send transport_options) File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py36/lib/python3.6/site-packages/oslo_messaging/_drivers/impl_fake.py", line 208, in _send raise failure File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py36/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 235, in inner return func(*args, **kwargs) File "/home/zuul/src/opendev.org/openstack/nova/nova/scheduler/manager.py", line 199, in select_destinations raise exception.NoValidHost(reason="") nova.exception.NoValidHost: No valid host was found. 2019-12-18 17:05:22,325 WARNING [nova.scheduler.utils] Failed to compute_task_build_instances: No valid host was found. 2019-12-18 17:05:22,325 WARNING [nova.scheduler.utils] Setting instance to ERROR state. 2019-12-18 17:05:22,773 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/afeb21d7-a89d-40ea-ac6e-28a7b80cec95" status: 200 len: 3749 microversion: 2.81 time: 0.084113 2019-12-18 17:05:22,899 INFO [nova.api.openstack.requestlog] 127.0.0.1 "DELETE /v2.1/servers/68f4cbad-7c69-492a-a173-6869dfa312d7" status: 204 len: 0 microversion: 2.81 time: 0.122724 2019-12-18 17:05:22,901 INFO [nova.tests.functional.libvirt.test_vpmem] booting on host1 2019-12-18 17:05:22,952 INFO [nova.compute.manager] Terminating instance 2019-12-18 17:05:22,968 INFO [nova.virt.libvirt.driver] Instance destroyed successfully. 2019-12-18 17:05:22,976 INFO [nova.virt.libvirt.driver] Deletion of /home/zuul/src/opendev.org/openstack/nova/instances/68f4cbad-7c69-492a-a173-6869dfa312d7_del complete 2019-12-18 17:05:23,042 INFO [nova.compute.manager] Took 0.08 seconds to destroy the instance on the hypervisor. 2019-12-18 17:05:23,051 INFO [nova.api.openstack.requestlog] 127.0.0.1 "POST /v2.1/servers" status: 202 len: 414 microversion: 2.81 time: 0.147582 2019-12-18 17:05:23,077 INFO [nova.compute.manager] Took 0.02 seconds to deallocate network for instance. 2019-12-18 17:05:23,095 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/e67add72-1327-466a-8f96-7e5025fb9847" status: 200 len: 1778 microversion: 2.81 time: 0.040906 2019-12-18 17:05:23,179 INFO [placement.requestlog] 127.0.0.1 "GET /placement/allocations/68f4cbad-7c69-492a-a173-6869dfa312d7" status: 200 len: 265 microversion: 1.28 2019-12-18 17:05:23,204 INFO [placement.requestlog] 127.0.0.1 "GET /placement/allocation_candidates?in_tree=56a25de3-2d91-48a3-9e77-a66f1b5d65b5&limit=1000&required=%21COMPUTE_STATUS_DISABLED&resources=CUSTOM_PMEM_NAMESPACE_SMALL%3A1%2CDISK_GB%3A1%2CMEMORY_MB%3A1024%2CVCPU%3A1" status: 200 len: 53 microversion: 1.34 2019-12-18 17:05:23,205 INFO [nova.scheduler.manager] Got no allocation candidates from the Placement API. This could be due to insufficient resources or a temporary occurrence as compute nodes start up. 2019-12-18 17:05:23,208 ERROR [nova.conductor.manager] Failed to schedule instances Traceback (most recent call last): File "/home/zuul/src/opendev.org/openstack/nova/nova/conductor/manager.py", line 1412, in schedule_and_build_instances instance_uuids, return_alternates=True) File "/home/zuul/src/opendev.org/openstack/nova/nova/conductor/manager.py", line 855, in _schedule_instances return_alternates=return_alternates) File "/home/zuul/src/opendev.org/openstack/nova/nova/scheduler/client/query.py", line 42, in select_destinations instance_uuids, return_objects, return_alternates) File "/home/zuul/src/opendev.org/openstack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations return cctxt.call(ctxt, 'select_destinations', **msg_args) File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py36/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 181, in call transport_options=self.transport_options) File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py36/lib/python3.6/site-packages/oslo_messaging/transport.py", line 129, in _send transport_options=transport_options) File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py36/lib/python3.6/site-packages/oslo_messaging/_drivers/impl_fake.py", line 222, in send transport_options) File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py36/lib/python3.6/site-packages/oslo_messaging/_drivers/impl_fake.py", line 208, in _send raise failure File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py36/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 235, in inner return func(*args, **kwargs) File "/home/zuul/src/opendev.org/openstack/nova/nova/scheduler/manager.py", line 199, in select_destinations raise exception.NoValidHost(reason="") nova.exception.NoValidHost: No valid host was found. 2019-12-18 17:05:23,240 INFO [placement.requestlog] 127.0.0.1 "PUT /placement/allocations/68f4cbad-7c69-492a-a173-6869dfa312d7" status: 204 len: 0 microversion: 1.28 2019-12-18 17:05:23,241 INFO [nova.scheduler.client.report] Deleted allocation for instance 68f4cbad-7c69-492a-a173-6869dfa312d7 2019-12-18 17:05:23,312 WARNING [nova.scheduler.utils] Failed to compute_task_build_instances: No valid host was found. 2019-12-18 17:05:23,313 WARNING [nova.scheduler.utils] Setting instance to ERROR state. 2019-12-18 17:05:23,696 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/e67add72-1327-466a-8f96-7e5025fb9847" status: 200 len: 3749 microversion: 2.81 time: 0.094132 2019-12-18 17:05:24,254 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/e67add72-1327-466a-8f96-7e5025fb9847" status: 200 len: 3749 microversion: 2.81 time: 0.053813 2019-12-18 17:05:24,817 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/e67add72-1327-466a-8f96-7e5025fb9847" status: 200 len: 3749 microversion: 2.81 time: 0.058160 2019-12-18 17:05:25,376 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/e67add72-1327-466a-8f96-7e5025fb9847" status: 200 len: 3749 microversion: 2.81 time: 0.054598 2019-12-18 17:05:25,943 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/e67add72-1327-466a-8f96-7e5025fb9847" status: 200 len: 3749 microversion: 2.81 time: 0.062802 2019-12-18 17:05:26,502 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/e67add72-1327-466a-8f96-7e5025fb9847" status: 200 len: 3749 microversion: 2.81 time: 0.054831 2019-12-18 17:05:27,067 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/e67add72-1327-466a-8f96-7e5025fb9847" status: 200 len: 3749 microversion: 2.81 time: 0.060461 2019-12-18 17:05:27,629 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/e67add72-1327-466a-8f96-7e5025fb9847" status: 200 len: 3749 microversion: 2.81 time: 0.057168 2019-12-18 17:05:28,199 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/e67add72-1327-466a-8f96-7e5025fb9847" status: 200 len: 3749 microversion: 2.81 time: 0.065318 2019-12-18 17:05:28,339 INFO [oslo_db.sqlalchemy.provision] DROP ALL OBJECTS, BACKEND sqlite:// 2019-12-18 17:05:28,372 INFO [oslo_db.sqlalchemy.provision] DISPOSE ENGINE Engine(sqlite://) }}} Traceback (most recent call last): File "/home/zuul/src/opendev.org/openstack/nova/nova/tests/functional/libvirt/test_vpmem.py", line 191, in test_create_servers_with_vpmem self._wait_for_state_change(server3, 'ACTIVE') File "/home/zuul/src/opendev.org/openstack/nova/nova/tests/functional/integrated_helpers.py", line 98, in _wait_for_state_change server, {'status': expected_status}, max_retries) File "/home/zuul/src/opendev.org/openstack/nova/nova/tests/functional/integrated_helpers.py", line 91, in _wait_for_server_parameter expected_params, server)) File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py36/lib/python3.6/site-packages/unittest2/case.py", line 690, in fail raise self.failureException(msg) AssertionError: Wait for state change failed, expected_params={'status': 'ACTIVE'}, server={'id': 'e67add72-1327-466a-8f96-7e5025fb9847', 'name': 'some-server', 'status': 'ERROR', 'tenant_id': '6f70656e737461636b20342065766572', 'user_id': 'admin', 'metadata': {}, 'hostId': '', 'image': {'id': '155d900f-4e14-4e4c-a73d-069cbf4541e6', 'links': [{'rel': 'bookmark', 'href': 'http://2a542091-42cb-4ba0-a2eb-9ed61b4b29da/images/155d900f-4e14-4e4c-a73d-069cbf4541e6'}]}, 'flavor': {'vcpus': 1, 'ram': 1024, 'disk': 1, 'ephemeral': 0, 'swap': 0, 'original_name': 'flavorBGI1I584', 'extra_specs': {'hw:pmem': 'SMALL'}}, 'created': '2019-12-18T17:05:23Z', 'updated': '2019-12-18T17:05:23Z', 'addresses': {}, 'accessIPv4': '', 'accessIPv6': '', 'links': [{'rel': 'self', 'href': 'http://2a542091-42cb-4ba0-a2eb-9ed61b4b29da/v2.1/servers/e67add72-1327-466a-8f96-7e5025fb9847'}, {'rel': 'bookmark', 'href': 'http://2a542091-42cb-4ba0-a2eb-9ed61b4b29da/servers/e67add72-1327-466a-8f96-7e5025fb9847'}], 'OS-DCF:diskConfig': 'MANUAL', 'fault': {'code': 500, 'created': '2019-12-18T17:05:23Z', 'message': 'No valid host was found. ', 'details': 'Traceback (most recent call last):\n File "/home/zuul/src/opendev.org/openstack/nova/nova/conductor/manager.py", line 1412, in schedule_and_build_instances\n instance_uuids, return_alternates=True)\n File "/home/zuul/src/opendev.org/openstack/nova/nova/conductor/manager.py", line 855, in _schedule_instances\n return_alternates=return_alternates)\n File "/home/zuul/src/opendev.org/openstack/nova/nova/scheduler/client/query.py", line 42, in select_destinations\n instance_uuids, return_objects, return_alternates)\n File "/home/zuul/src/opendev.org/openstack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations\n return cctxt.call(ctxt, \'select_destinations\', **msg_args)\n File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py36/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 181, in call\n transport_options=self.transport_options)\n File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py36/lib/python3.6/site-packages/oslo_messaging/transport.py", line 129, in _send\n transport_options=transport_options)\n File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py36/lib/python3.6/site-packages/oslo_messaging/_drivers/impl_fake.py", line 222, in send\n transport_options)\n File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py36/lib/python3.6/site-packages/oslo_messaging/_drivers/impl_fake.py", line 208, in _send\n raise failure\n File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py36/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 235, in inner\n return func(*args, **kwargs)\n File "/home/zuul/src/opendev.org/openstack/nova/nova/scheduler/manager.py", line 199, in select_destinations\n raise exception.NoValidHost(reason="")\nnova.exception.NoValidHost: No valid host was found. \n'}, 'OS-EXT-AZ:availability_zone': 'nova', 'config_drive': '', 'key_name': None, 'OS-SRV-USG:launched_at': None, 'OS-SRV-USG:terminated_at': None, 'OS-EXT-SRV-ATTR:host': None, 'OS-EXT-SRV-ATTR:instance_name': 'instance-00000002', 'OS-EXT-SRV-ATTR:hypervisor_hostname': None, 'OS-EXT-SRV-ATTR:reservation_id': 'r-e4t4h5f5', 'OS-EXT-SRV-ATTR:launch_index': 0, 'OS-EXT-SRV-ATTR:hostname': 'some-server', 'OS-EXT-SRV-ATTR:kernel_id': '', 'OS-EXT-SRV-ATTR:ramdisk_id': '', 'OS-EXT-SRV-ATTR:root_device_name': None, 'OS-EXT-SRV-ATTR:user_data': None, 'OS-EXT-STS:task_state': None, 'OS-EXT-STS:vm_state': 'error', 'OS-EXT-STS:power_state': 0, 'os-extended-volumes:volumes_attached': [], 'host_status': '', 'locked': False, 'locked_reason': None, 'description': None, 'tags': [], 'trusted_image_certificates': None, 'server_groups': []} http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22'extra_specs'%3A%20%7B'hw%3Apmem'%3A%20'SMALL'%7D%7D%5C%22%20AND%20tags%3A%5C%22console%5C%22&from=7d 18 hits in 7 days, check and gate, all failures. ** Affects: nova Importance: Medium Status: Confirmed ** Tags: gate-failure testing ** Changed in: nova Status: New => Confirmed ** Changed in: nova Importance: Undecided => Medium ** Tags added: gate-failure testing -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1856902 Title: nova.tests.functional.libvirt.test_vpmem.VPMEMTests.test_create_servers_with_vpmem fails intermittently Status in OpenStack Compute (nova): Confirmed Bug description: Seen here: https://748441c9eeeb89eb36db- b17eeb704755458252956ad3afd603fe.ssl.cf5.rackcdn.com/697670/3/gate /nova-tox-functional-py36/54fe49d/testr_results.html.gz ft1.1: nova.tests.functional.libvirt.test_vpmem.VPMEMTests.test_create_servers_with_vpmemtesttools.testresult.real._StringException: pythonlogging:'': {{{ 2019-12-18 17:05:18,534 WARNING [placement.db_api] TransactionFactory already started, not reconfiguring. 2019-12-18 17:05:18,885 INFO [nova.service] Starting conductor node (version 20.1.0) 2019-12-18 17:05:18,917 INFO [nova.service] Starting scheduler node (version 20.1.0) 2019-12-18 17:05:18,981 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/flavors/detail" status: 200 len: 2605 microversion: 2.81 time: 0.020144 2019-12-18 17:05:18,996 INFO [nova.api.openstack.requestlog] 127.0.0.1 "POST /v2.1/flavors" status: 200 len: 464 microversion: 2.81 time: 0.012445 2019-12-18 17:05:19,018 INFO [nova.api.openstack.requestlog] 127.0.0.1 "POST /v2.1/flavors/99512851/os-extra_specs" status: 200 len: 37 microversion: 2.81 time: 0.018592 2019-12-18 17:05:19,035 INFO [nova.virt.driver] Loading compute driver 'libvirt.LibvirtDriver' 2019-12-18 17:05:19,038 WARNING [os_brick.initiator.connectors.remotefs] Connection details not present. RemoteFsClient may not initialize properly. 2019-12-18 17:05:19,040 INFO [nova.service] Starting compute node (version 20.1.0) 2019-12-18 17:05:19,058 WARNING [nova.compute.manager] Compute node host1 not found in the database. If this is the first time this service is starting on this host, then you can ignore this warning. 2019-12-18 17:05:19,063 INFO [nova.compute.manager] Looking for unclaimed instances stuck in BUILDING status for nodes managed by this host 2019-12-18 17:05:19,073 WARNING [nova.compute.manager] No compute node record found for host host1. If this is the first time this service is starting on this host, then you can ignore this warning. 2019-12-18 17:05:19,082 WARNING [nova.compute.resource_tracker] No compute node record for host1:host1 2019-12-18 17:05:19,086 INFO [nova.compute.resource_tracker] Compute node record created for host1:host1 with uuid: 56a25de3-2d91-48a3-9e77-a66f1b5d65b5 2019-12-18 17:05:19,123 INFO [placement.requestlog] 127.0.0.1 "GET /placement/resource_providers?in_tree=56a25de3-2d91-48a3-9e77-a66f1b5d65b5" status: 200 len: 26 microversion: 1.14 2019-12-18 17:05:19,131 INFO [placement.requestlog] 127.0.0.1 "POST /placement/resource_providers" status: 200 len: 828 microversion: 1.20 2019-12-18 17:05:19,132 INFO [nova.scheduler.client.report] [req-54d75b29-ade3-4f2a-8911-d1147ef449fc] Created resource provider record via placement API for resource provider with UUID 56a25de3-2d91-48a3-9e77-a66f1b5d65b5 and name host1. 2019-12-18 17:05:19,133 INFO [nova.virt.libvirt.host] kernel doesn't support AMD SEV 2019-12-18 17:05:19,149 INFO [placement.requestlog] 127.0.0.1 "PUT /placement/resource_classes/CUSTOM_PMEM_NAMESPACE_SMALL" status: 201 len: 0 microversion: 1.7 2019-12-18 17:05:19,159 INFO [placement.requestlog] 127.0.0.1 "PUT /placement/resource_classes/CUSTOM_PMEM_NAMESPACE_4GB" status: 201 len: 0 microversion: 1.7 2019-12-18 17:05:19,173 INFO [placement.requestlog] 127.0.0.1 "PUT /placement/resource_providers/56a25de3-2d91-48a3-9e77-a66f1b5d65b5/inventories" status: 200 len: 665 microversion: 1.26 2019-12-18 17:05:19,181 INFO [placement.requestlog] 127.0.0.1 "GET /placement/traits?name=in:COMPUTE_NET_ATTACH_INTERFACE_WITH_TAG,COMPUTE_IMAGE_TYPE_AMI,HW_CPU_X86_VMX,HW_CPU_X86_AESNI,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_IMAGE_TYPE_AKI,COMPUTE_IMAGE_TYPE_ISO,COMPUTE_VOLUME_ATTACH_WITH_TAG,COMPUTE_IMAGE_TYPE_ARI,COMPUTE_IMAGE_TYPE_RAW,COMPUTE_DEVICE_TAGGING,HW_CPU_HYPERTHREADING,COMPUTE_VOLUME_EXTEND,COMPUTE_IMAGE_TYPE_QCOW2,COMPUTE_TRUSTED_CERTS,COMPUTE_NODE" status: 200 len: 432 microversion: 1.6 2019-12-18 17:05:19,200 INFO [placement.requestlog] 127.0.0.1 "PUT /placement/resource_providers/56a25de3-2d91-48a3-9e77-a66f1b5d65b5/traits" status: 200 len: 467 microversion: 1.6 2019-12-18 17:05:19,224 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/os-hypervisors?hypervisor_hostname_pattern=host1" status: 200 len: 133 microversion: 2.81 time: 0.019077 2019-12-18 17:05:19,236 INFO [placement.requestlog] 127.0.0.1 "GET /placement/resource_providers/56a25de3-2d91-48a3-9e77-a66f1b5d65b5/inventories" status: 200 len: 665 microversion: 1.0 2019-12-18 17:05:19,253 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/os-hypervisors?hypervisor_hostname_pattern=host1" status: 200 len: 133 microversion: 2.81 time: 0.014420 2019-12-18 17:05:19,255 INFO [nova.tests.functional.libvirt.test_vpmem] booting on host1 2019-12-18 17:05:19,329 INFO [nova.api.openstack.requestlog] 127.0.0.1 "POST /v2.1/servers" status: 202 len: 414 microversion: 2.81 time: 0.071830 2019-12-18 17:05:19,352 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/3a085e4e-52ba-49cc-be7c-6eca2974765a" status: 200 len: 1778 microversion: 2.81 time: 0.019663 2019-12-18 17:05:19,473 INFO [placement.requestlog] 127.0.0.1 "GET /placement/allocation_candidates?in_tree=56a25de3-2d91-48a3-9e77-a66f1b5d65b5&limit=1000&required=%21COMPUTE_STATUS_DISABLED&resources=CUSTOM_PMEM_NAMESPACE_SMALL%3A1%2CDISK_GB%3A1%2CMEMORY_MB%3A1024%2CVCPU%3A1" status: 200 len: 1087 microversion: 1.34 2019-12-18 17:05:19,495 INFO [nova.scheduler.host_manager] Host filter forcing available hosts to host1 2019-12-18 17:05:19,503 INFO [placement.requestlog] 127.0.0.1 "GET /placement/allocations/3a085e4e-52ba-49cc-be7c-6eca2974765a" status: 200 len: 19 microversion: 1.28 2019-12-18 17:05:19,530 INFO [placement.requestlog] 127.0.0.1 "PUT /placement/allocations/3a085e4e-52ba-49cc-be7c-6eca2974765a" status: 204 len: 0 microversion: 1.34 2019-12-18 17:05:19,724 INFO [placement.requestlog] 127.0.0.1 "GET /placement/allocations/3a085e4e-52ba-49cc-be7c-6eca2974765a" status: 200 len: 265 microversion: 1.28 2019-12-18 17:05:19,731 INFO [nova.compute.claims] Claim successful on node host1 2019-12-18 17:05:19,985 INFO [nova.compute.manager] Took 0.00 seconds to spawn the instance on the hypervisor. 2019-12-18 17:05:20,077 INFO [nova.compute.manager] Took 0.36 seconds to build instance. 2019-12-18 17:05:20,087 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/3a085e4e-52ba-49cc-be7c-6eca2974765a" status: 200 len: 1875 microversion: 2.81 time: 0.227589 2019-12-18 17:05:20,122 INFO [nova.scheduler.host_manager] Received an update from an unknown host 'host1'. Re-created its InstanceList. 2019-12-18 17:05:20,679 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/3a085e4e-52ba-49cc-be7c-6eca2974765a" status: 200 len: 1880 microversion: 2.81 time: 0.087029 2019-12-18 17:05:20,690 INFO [placement.requestlog] 127.0.0.1 "GET /placement/allocations/3a085e4e-52ba-49cc-be7c-6eca2974765a" status: 200 len: 169 microversion: 1.0 2019-12-18 17:05:20,691 INFO [nova.tests.functional.libvirt.test_vpmem] booting on host1 2019-12-18 17:05:20,766 INFO [nova.api.openstack.requestlog] 127.0.0.1 "POST /v2.1/servers" status: 202 len: 414 microversion: 2.81 time: 0.072347 2019-12-18 17:05:20,786 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/68f4cbad-7c69-492a-a173-6869dfa312d7" status: 200 len: 1778 microversion: 2.81 time: 0.017267 2019-12-18 17:05:20,902 INFO [placement.requestlog] 127.0.0.1 "GET /placement/allocation_candidates?in_tree=56a25de3-2d91-48a3-9e77-a66f1b5d65b5&limit=1000&required=%21COMPUTE_STATUS_DISABLED&resources=CUSTOM_PMEM_NAMESPACE_SMALL%3A1%2CDISK_GB%3A1%2CMEMORY_MB%3A1024%2CVCPU%3A1" status: 200 len: 1090 microversion: 1.34 2019-12-18 17:05:20,922 INFO [nova.scheduler.host_manager] Host filter forcing available hosts to host1 2019-12-18 17:05:20,930 INFO [placement.requestlog] 127.0.0.1 "GET /placement/allocations/68f4cbad-7c69-492a-a173-6869dfa312d7" status: 200 len: 19 microversion: 1.28 2019-12-18 17:05:20,955 INFO [placement.requestlog] 127.0.0.1 "PUT /placement/allocations/68f4cbad-7c69-492a-a173-6869dfa312d7" status: 204 len: 0 microversion: 1.34 2019-12-18 17:05:21,131 INFO [placement.requestlog] 127.0.0.1 "GET /placement/allocations/68f4cbad-7c69-492a-a173-6869dfa312d7" status: 200 len: 265 microversion: 1.28 2019-12-18 17:05:21,138 INFO [nova.compute.claims] Claim successful on node host1 2019-12-18 17:05:21,391 INFO [nova.compute.manager] Took 0.00 seconds to spawn the instance on the hypervisor. 2019-12-18 17:05:21,450 INFO [nova.compute.manager] Took 0.33 seconds to build instance. 2019-12-18 17:05:21,490 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/68f4cbad-7c69-492a-a173-6869dfa312d7" status: 200 len: 1863 microversion: 2.81 time: 0.183892 2019-12-18 17:05:22,075 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/68f4cbad-7c69-492a-a173-6869dfa312d7" status: 200 len: 1880 microversion: 2.81 time: 0.080401 2019-12-18 17:05:22,084 INFO [placement.requestlog] 127.0.0.1 "GET /placement/allocations/68f4cbad-7c69-492a-a173-6869dfa312d7" status: 200 len: 169 microversion: 1.0 2019-12-18 17:05:22,085 INFO [nova.tests.functional.libvirt.test_vpmem] booting on host1 2019-12-18 17:05:22,162 INFO [nova.api.openstack.requestlog] 127.0.0.1 "POST /v2.1/servers" status: 202 len: 414 microversion: 2.81 time: 0.074114 2019-12-18 17:05:22,183 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/afeb21d7-a89d-40ea-ac6e-28a7b80cec95" status: 200 len: 1778 microversion: 2.81 time: 0.018034 2019-12-18 17:05:22,283 INFO [placement.requestlog] 127.0.0.1 "GET /placement/allocation_candidates?in_tree=56a25de3-2d91-48a3-9e77-a66f1b5d65b5&limit=1000&required=%21COMPUTE_STATUS_DISABLED&resources=CUSTOM_PMEM_NAMESPACE_SMALL%3A1%2CDISK_GB%3A1%2CMEMORY_MB%3A1024%2CVCPU%3A1" status: 200 len: 53 microversion: 1.34 2019-12-18 17:05:22,284 INFO [nova.scheduler.manager] Got no allocation candidates from the Placement API. This could be due to insufficient resources or a temporary occurrence as compute nodes start up. 2019-12-18 17:05:22,284 ERROR [nova.conductor.manager] Failed to schedule instances Traceback (most recent call last): File "/home/zuul/src/opendev.org/openstack/nova/nova/conductor/manager.py", line 1412, in schedule_and_build_instances instance_uuids, return_alternates=True) File "/home/zuul/src/opendev.org/openstack/nova/nova/conductor/manager.py", line 855, in _schedule_instances return_alternates=return_alternates) File "/home/zuul/src/opendev.org/openstack/nova/nova/scheduler/client/query.py", line 42, in select_destinations instance_uuids, return_objects, return_alternates) File "/home/zuul/src/opendev.org/openstack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations return cctxt.call(ctxt, 'select_destinations', **msg_args) File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py36/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 181, in call transport_options=self.transport_options) File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py36/lib/python3.6/site-packages/oslo_messaging/transport.py", line 129, in _send transport_options=transport_options) File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py36/lib/python3.6/site-packages/oslo_messaging/_drivers/impl_fake.py", line 222, in send transport_options) File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py36/lib/python3.6/site-packages/oslo_messaging/_drivers/impl_fake.py", line 208, in _send raise failure File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py36/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 235, in inner return func(*args, **kwargs) File "/home/zuul/src/opendev.org/openstack/nova/nova/scheduler/manager.py", line 199, in select_destinations raise exception.NoValidHost(reason="") nova.exception.NoValidHost: No valid host was found. 2019-12-18 17:05:22,325 WARNING [nova.scheduler.utils] Failed to compute_task_build_instances: No valid host was found. 2019-12-18 17:05:22,325 WARNING [nova.scheduler.utils] Setting instance to ERROR state. 2019-12-18 17:05:22,773 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/afeb21d7-a89d-40ea-ac6e-28a7b80cec95" status: 200 len: 3749 microversion: 2.81 time: 0.084113 2019-12-18 17:05:22,899 INFO [nova.api.openstack.requestlog] 127.0.0.1 "DELETE /v2.1/servers/68f4cbad-7c69-492a-a173-6869dfa312d7" status: 204 len: 0 microversion: 2.81 time: 0.122724 2019-12-18 17:05:22,901 INFO [nova.tests.functional.libvirt.test_vpmem] booting on host1 2019-12-18 17:05:22,952 INFO [nova.compute.manager] Terminating instance 2019-12-18 17:05:22,968 INFO [nova.virt.libvirt.driver] Instance destroyed successfully. 2019-12-18 17:05:22,976 INFO [nova.virt.libvirt.driver] Deletion of /home/zuul/src/opendev.org/openstack/nova/instances/68f4cbad-7c69-492a-a173-6869dfa312d7_del complete 2019-12-18 17:05:23,042 INFO [nova.compute.manager] Took 0.08 seconds to destroy the instance on the hypervisor. 2019-12-18 17:05:23,051 INFO [nova.api.openstack.requestlog] 127.0.0.1 "POST /v2.1/servers" status: 202 len: 414 microversion: 2.81 time: 0.147582 2019-12-18 17:05:23,077 INFO [nova.compute.manager] Took 0.02 seconds to deallocate network for instance. 2019-12-18 17:05:23,095 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/e67add72-1327-466a-8f96-7e5025fb9847" status: 200 len: 1778 microversion: 2.81 time: 0.040906 2019-12-18 17:05:23,179 INFO [placement.requestlog] 127.0.0.1 "GET /placement/allocations/68f4cbad-7c69-492a-a173-6869dfa312d7" status: 200 len: 265 microversion: 1.28 2019-12-18 17:05:23,204 INFO [placement.requestlog] 127.0.0.1 "GET /placement/allocation_candidates?in_tree=56a25de3-2d91-48a3-9e77-a66f1b5d65b5&limit=1000&required=%21COMPUTE_STATUS_DISABLED&resources=CUSTOM_PMEM_NAMESPACE_SMALL%3A1%2CDISK_GB%3A1%2CMEMORY_MB%3A1024%2CVCPU%3A1" status: 200 len: 53 microversion: 1.34 2019-12-18 17:05:23,205 INFO [nova.scheduler.manager] Got no allocation candidates from the Placement API. This could be due to insufficient resources or a temporary occurrence as compute nodes start up. 2019-12-18 17:05:23,208 ERROR [nova.conductor.manager] Failed to schedule instances Traceback (most recent call last): File "/home/zuul/src/opendev.org/openstack/nova/nova/conductor/manager.py", line 1412, in schedule_and_build_instances instance_uuids, return_alternates=True) File "/home/zuul/src/opendev.org/openstack/nova/nova/conductor/manager.py", line 855, in _schedule_instances return_alternates=return_alternates) File "/home/zuul/src/opendev.org/openstack/nova/nova/scheduler/client/query.py", line 42, in select_destinations instance_uuids, return_objects, return_alternates) File "/home/zuul/src/opendev.org/openstack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations return cctxt.call(ctxt, 'select_destinations', **msg_args) File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py36/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 181, in call transport_options=self.transport_options) File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py36/lib/python3.6/site-packages/oslo_messaging/transport.py", line 129, in _send transport_options=transport_options) File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py36/lib/python3.6/site-packages/oslo_messaging/_drivers/impl_fake.py", line 222, in send transport_options) File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py36/lib/python3.6/site-packages/oslo_messaging/_drivers/impl_fake.py", line 208, in _send raise failure File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py36/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 235, in inner return func(*args, **kwargs) File "/home/zuul/src/opendev.org/openstack/nova/nova/scheduler/manager.py", line 199, in select_destinations raise exception.NoValidHost(reason="") nova.exception.NoValidHost: No valid host was found. 2019-12-18 17:05:23,240 INFO [placement.requestlog] 127.0.0.1 "PUT /placement/allocations/68f4cbad-7c69-492a-a173-6869dfa312d7" status: 204 len: 0 microversion: 1.28 2019-12-18 17:05:23,241 INFO [nova.scheduler.client.report] Deleted allocation for instance 68f4cbad-7c69-492a-a173-6869dfa312d7 2019-12-18 17:05:23,312 WARNING [nova.scheduler.utils] Failed to compute_task_build_instances: No valid host was found. 2019-12-18 17:05:23,313 WARNING [nova.scheduler.utils] Setting instance to ERROR state. 2019-12-18 17:05:23,696 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/e67add72-1327-466a-8f96-7e5025fb9847" status: 200 len: 3749 microversion: 2.81 time: 0.094132 2019-12-18 17:05:24,254 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/e67add72-1327-466a-8f96-7e5025fb9847" status: 200 len: 3749 microversion: 2.81 time: 0.053813 2019-12-18 17:05:24,817 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/e67add72-1327-466a-8f96-7e5025fb9847" status: 200 len: 3749 microversion: 2.81 time: 0.058160 2019-12-18 17:05:25,376 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/e67add72-1327-466a-8f96-7e5025fb9847" status: 200 len: 3749 microversion: 2.81 time: 0.054598 2019-12-18 17:05:25,943 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/e67add72-1327-466a-8f96-7e5025fb9847" status: 200 len: 3749 microversion: 2.81 time: 0.062802 2019-12-18 17:05:26,502 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/e67add72-1327-466a-8f96-7e5025fb9847" status: 200 len: 3749 microversion: 2.81 time: 0.054831 2019-12-18 17:05:27,067 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/e67add72-1327-466a-8f96-7e5025fb9847" status: 200 len: 3749 microversion: 2.81 time: 0.060461 2019-12-18 17:05:27,629 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/e67add72-1327-466a-8f96-7e5025fb9847" status: 200 len: 3749 microversion: 2.81 time: 0.057168 2019-12-18 17:05:28,199 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/e67add72-1327-466a-8f96-7e5025fb9847" status: 200 len: 3749 microversion: 2.81 time: 0.065318 2019-12-18 17:05:28,339 INFO [oslo_db.sqlalchemy.provision] DROP ALL OBJECTS, BACKEND sqlite:// 2019-12-18 17:05:28,372 INFO [oslo_db.sqlalchemy.provision] DISPOSE ENGINE Engine(sqlite://) }}} Traceback (most recent call last): File "/home/zuul/src/opendev.org/openstack/nova/nova/tests/functional/libvirt/test_vpmem.py", line 191, in test_create_servers_with_vpmem self._wait_for_state_change(server3, 'ACTIVE') File "/home/zuul/src/opendev.org/openstack/nova/nova/tests/functional/integrated_helpers.py", line 98, in _wait_for_state_change server, {'status': expected_status}, max_retries) File "/home/zuul/src/opendev.org/openstack/nova/nova/tests/functional/integrated_helpers.py", line 91, in _wait_for_server_parameter expected_params, server)) File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py36/lib/python3.6/site-packages/unittest2/case.py", line 690, in fail raise self.failureException(msg) AssertionError: Wait for state change failed, expected_params={'status': 'ACTIVE'}, server={'id': 'e67add72-1327-466a-8f96-7e5025fb9847', 'name': 'some-server', 'status': 'ERROR', 'tenant_id': '6f70656e737461636b20342065766572', 'user_id': 'admin', 'metadata': {}, 'hostId': '', 'image': {'id': '155d900f-4e14-4e4c-a73d-069cbf4541e6', 'links': [{'rel': 'bookmark', 'href': 'http://2a542091-42cb-4ba0-a2eb-9ed61b4b29da/images/155d900f-4e14-4e4c-a73d-069cbf4541e6'}]}, 'flavor': {'vcpus': 1, 'ram': 1024, 'disk': 1, 'ephemeral': 0, 'swap': 0, 'original_name': 'flavorBGI1I584', 'extra_specs': {'hw:pmem': 'SMALL'}}, 'created': '2019-12-18T17:05:23Z', 'updated': '2019-12-18T17:05:23Z', 'addresses': {}, 'accessIPv4': '', 'accessIPv6': '', 'links': [{'rel': 'self', 'href': 'http://2a542091-42cb-4ba0-a2eb-9ed61b4b29da/v2.1/servers/e67add72-1327-466a-8f96-7e5025fb9847'}, {'rel': 'bookmark', 'href': 'http://2a542091-42cb-4ba0-a2eb-9ed61b4b29da/servers/e67add72-1327-466a-8f96-7e5025fb9847'}], 'OS-DCF:diskConfig': 'MANUAL', 'fault': {'code': 500, 'created': '2019-12-18T17:05:23Z', 'message': 'No valid host was found. ', 'details': 'Traceback (most recent call last):\n File "/home/zuul/src/opendev.org/openstack/nova/nova/conductor/manager.py", line 1412, in schedule_and_build_instances\n instance_uuids, return_alternates=True)\n File "/home/zuul/src/opendev.org/openstack/nova/nova/conductor/manager.py", line 855, in _schedule_instances\n return_alternates=return_alternates)\n File "/home/zuul/src/opendev.org/openstack/nova/nova/scheduler/client/query.py", line 42, in select_destinations\n instance_uuids, return_objects, return_alternates)\n File "/home/zuul/src/opendev.org/openstack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations\n return cctxt.call(ctxt, \'select_destinations\', **msg_args)\n File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py36/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 181, in call\n transport_options=self.transport_options)\n File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py36/lib/python3.6/site-packages/oslo_messaging/transport.py", line 129, in _send\n transport_options=transport_options)\n File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py36/lib/python3.6/site-packages/oslo_messaging/_drivers/impl_fake.py", line 222, in send\n transport_options)\n File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py36/lib/python3.6/site-packages/oslo_messaging/_drivers/impl_fake.py", line 208, in _send\n raise failure\n File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py36/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 235, in inner\n return func(*args, **kwargs)\n File "/home/zuul/src/opendev.org/openstack/nova/nova/scheduler/manager.py", line 199, in select_destinations\n raise exception.NoValidHost(reason="")\nnova.exception.NoValidHost: No valid host was found. \n'}, 'OS-EXT-AZ:availability_zone': 'nova', 'config_drive': '', 'key_name': None, 'OS-SRV-USG:launched_at': None, 'OS-SRV-USG:terminated_at': None, 'OS-EXT-SRV-ATTR:host': None, 'OS-EXT-SRV-ATTR:instance_name': 'instance-00000002', 'OS-EXT-SRV-ATTR:hypervisor_hostname': None, 'OS-EXT-SRV-ATTR:reservation_id': 'r-e4t4h5f5', 'OS-EXT-SRV-ATTR:launch_index': 0, 'OS-EXT-SRV-ATTR:hostname': 'some-server', 'OS-EXT-SRV-ATTR:kernel_id': '', 'OS-EXT-SRV-ATTR:ramdisk_id': '', 'OS-EXT-SRV-ATTR:root_device_name': None, 'OS-EXT-SRV-ATTR:user_data': None, 'OS-EXT-STS:task_state': None, 'OS-EXT-STS:vm_state': 'error', 'OS-EXT-STS:power_state': 0, 'os-extended-volumes:volumes_attached': [], 'host_status': '', 'locked': False, 'locked_reason': None, 'description': None, 'tags': [], 'trusted_image_certificates': None, 'server_groups': []} http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22'extra_specs'%3A%20%7B'hw%3Apmem'%3A%20'SMALL'%7D%7D%5C%22%20AND%20tags%3A%5C%22console%5C%22&from=7d 18 hits in 7 days, check and gate, all failures. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1856902/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
