Public bug reported: Steps to reproduce:
Create a glance image with hw_vif_multiqueue_enabled='true'. Boot a instance with this image, and use a flavor with 46 cpus. Bug introduced in: https://review.openstack.org/#/c/128829/ See nova/virt/libvirt/vif.py line 163 https://github.com/openstack/nova/blob/master/nova/virt/libvirt/vif.py#L163 If we have a flavor with more than 32 Cpus, the spawning of the instance will fail because it is not possible to have more than 32 hardware queues in the virtual network nic. In our case we have a Flavor with 46 Cpus, and we get the following: Invalid number of queues (= 46), must be a postive integer less than 31. Whatever is the number of Cpus of the flavor, the max number of hw queues for the network nic should be 32. Here the complete stacktrace: 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [req-ccfc9497-1d91-43f1-b7d1-0bfcf5d90bfb 5c3f2df216fa42a987f8f4600e5e0da2 bdf747f88fee4b5a9faca3da7c26754c - - -] [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] Instance failed to spawn 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] Traceback (most recent call last): 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2156, in _build_resources 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] yield resources 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2009, in _build_and_run_instance 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] block_device_info=block_device_info) 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 2534, in spawn 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] block_device_info=block_device_info) 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 4620, in _create_domain_and_network 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] xml, pause=pause, power_on=power_on) 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 4550, in _create_domain 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] guest.launch(pause=pause) 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/guest.py", line 142, in launch 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] self._encoded_xml, errors='ignore') 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 195, in __exit__ 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] six.reraise(self.type_, self.value, self.tb) 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/guest.py", line 137, in launch 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] return self._domain.createWithFlags(flags) 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 183, in doit 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] result = proxy_call(self._autowrap, f, *args, **kwargs) 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 141, in proxy_call 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] rv = execute(f, *args, **kwargs) 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 122, in execute 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] six.reraise(c, e, tb) 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 80, in tworker 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] rv = meth(*args, **kwargs) 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] File "/usr/lib/python2.7/dist-packages/libvirt.py", line 1059, in createWithFlags 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self) 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] libvirtError: internal error: process exited while connecting to monitor: 2016-11-25T09:44:50.496310Z qemu-system-x86_64: -device virtio-net-pci,mq=on,vectors=94,netdev=hostnet0,id=net0,mac=fa:16:3e:19:32:9f,bus=pci.0,addr=0x3: Invalid number of queues (= 46), must be a postive integer less than 31. 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] 2016-11-25T09:44:50.496389Z qemu-system-x86_64: -device virtio-net-pci,mq=on,vectors=94,netdev=hostnet0,id=net0,mac=fa:16:3e:19:32:9f,bus=pci.0,addr=0x3: Device 'virtio-net-pci' could not be initialized 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] ** Affects: nova Importance: Undecided Status: New ** Summary changed: - If flavor has more than 32 cpus cannot spawn instancee if glance image has hw_vif_multiqueue_enabled='true' + If flavor has more than 32 cpus cannot spawn instance if glance image has hw_vif_multiqueue_enabled='true' -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1644839 Title: If flavor has more than 32 cpus cannot spawn instance if glance image has hw_vif_multiqueue_enabled='true' Status in OpenStack Compute (nova): New Bug description: Steps to reproduce: Create a glance image with hw_vif_multiqueue_enabled='true'. Boot a instance with this image, and use a flavor with 46 cpus. Bug introduced in: https://review.openstack.org/#/c/128829/ See nova/virt/libvirt/vif.py line 163 https://github.com/openstack/nova/blob/master/nova/virt/libvirt/vif.py#L163 If we have a flavor with more than 32 Cpus, the spawning of the instance will fail because it is not possible to have more than 32 hardware queues in the virtual network nic. In our case we have a Flavor with 46 Cpus, and we get the following: Invalid number of queues (= 46), must be a postive integer less than 31. Whatever is the number of Cpus of the flavor, the max number of hw queues for the network nic should be 32. Here the complete stacktrace: 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [req-ccfc9497-1d91-43f1-b7d1-0bfcf5d90bfb 5c3f2df216fa42a987f8f4600e5e0da2 bdf747f88fee4b5a9faca3da7c26754c - - -] [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] Instance failed to spawn 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] Traceback (most recent call last): 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2156, in _build_resources 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] yield resources 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2009, in _build_and_run_instance 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] block_device_info=block_device_info) 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 2534, in spawn 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] block_device_info=block_device_info) 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 4620, in _create_domain_and_network 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] xml, pause=pause, power_on=power_on) 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 4550, in _create_domain 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] guest.launch(pause=pause) 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/guest.py", line 142, in launch 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] self._encoded_xml, errors='ignore') 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 195, in __exit__ 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] six.reraise(self.type_, self.value, self.tb) 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/guest.py", line 137, in launch 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] return self._domain.createWithFlags(flags) 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 183, in doit 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] result = proxy_call(self._autowrap, f, *args, **kwargs) 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 141, in proxy_call 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] rv = execute(f, *args, **kwargs) 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 122, in execute 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] six.reraise(c, e, tb) 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 80, in tworker 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] rv = meth(*args, **kwargs) 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] File "/usr/lib/python2.7/dist-packages/libvirt.py", line 1059, in createWithFlags 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self) 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] libvirtError: internal error: process exited while connecting to monitor: 2016-11-25T09:44:50.496310Z qemu-system-x86_64: -device virtio-net-pci,mq=on,vectors=94,netdev=hostnet0,id=net0,mac=fa:16:3e:19:32:9f,bus=pci.0,addr=0x3: Invalid number of queues (= 46), must be a postive integer less than 31. 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] 2016-11-25T09:44:50.496389Z qemu-system-x86_64: -device virtio-net-pci,mq=on,vectors=94,netdev=hostnet0,id=net0,mac=fa:16:3e:19:32:9f,bus=pci.0,addr=0x3: Device 'virtio-net-pci' could not be initialized 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] 2016-11-25 10:44:52.265 4528 ERROR nova.compute.manager [instance: 8e8af25a-74ad-4a68-9f39-7a987c2a049c] To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1644839/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
