On Fri, Dec 23, 2016 at 1:37 PM Steve Gordon <sgor...@redhat.com> wrote:
> ----- Original Message -----
> > From: "Neil Jerram" <n...@tigera.io>
> > To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev@lists.openstack.org>
> >
> > I appreciate that even libvirt 2.0.0 will be ancient history by now, to
> its
> > developers, but I am seeing further issues that look associated with the
> > recent CentOS 7 transition from libvirt 1.2.7 to libvirt 2.0.0, and would
> > appreciate any comments on them that people may have. I believe these
> > issues are independent of those that have already been discussed on other
> > threads.
> >
> > First, this traceback in nova-compute.log:
> >
> > Traceback (most recent call last):
> > File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line
> > 2156, in _build_resources
> > yield resources
> > File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line
> > 2009, in _build_and_run_instance
> > block_device_info=block_device_info)
> > File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py",
> line
> > 2534, in spawn
> > block_device_info=block_device_info)
> > File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py",
> line
> > 4620, in _create_domain_and_network
> > xml, pause=pause, power_on=power_on)
> > File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py",
> line
> > 4550, in _create_domain
> > guest.launch(pause=pause)
> > File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/guest.py",
> line
> > 142, in launch
> > self._encoded_xml, errors='ignore')
> > File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line
> 195,
> > in __exit__
> > six.reraise(self.type_, self.value, self.tb)
> > File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/guest.py",
> line
> > 137, in launch
> > return self._domain.createWithFlags(flags)
> > File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 186, in
> > doit
> > result = proxy_call(self._autowrap, f, *args, **kwargs)
> > File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 144, in
> > proxy_call
> > rv = execute(f, *args, **kwargs)
> > File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 125, in
> > execute
> > six.reraise(c, e, tb)
> > File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 83, in
> > tworker
> > rv = meth(*args, **kwargs)
> > File "/usr/lib64/python2.7/site-packages/libvirt.py", line 1065, in
> > createWithFlags
> > if ret == -1: raise libvirtError ('virDomainCreateWithFlags()
> failed',
> > dom=self)
> > libvirtError: Cannot find '' in path: No such file or directory
> >
> > which I believe is caused by the empty path attribute in this part of the
> > XML:
> >
> > <interface type='ethernet'>
> > <mac address='fa:16:3e:3c:96:33'/>
> > <script path=''/>
> > <target dev='tap06992dfb-5d'/>
> > <model type='virtio'/>
> > <driver name='qemu'/>
> > <address type='pci' domain='0x0000' bus='0x00' slot='0x03'
> > function='0x0'/>
> > </interface>
> >
> > which is in turn caused, I think, by
> >
> https://github.com/openstack/nova/blob/master/nova/virt/libvirt/designer.py#L61
> >
> > Is it plausible that libvirt 1.2.7 would have avoided trying to invoke a
> > script with an empty path, whereas libvirt 2.0.0 does not?
>
> It seems someone filed this as
> https://bugs.launchpad.net/nova/+bug/1649527 from a Nova POV. The Libvirt
> folks helped me narrow this down to this commit being the first one that
> would have exhibited this behavior:
>
>
> https://libvirt.org/git/?p=libvirt.git;a=commit;h=9c17d665fdc5f0ab74500a14c30627014c11b2c0
>
> Michal Privoznik provided some additional context:
>
> "Previously, libvirt merely just appended 'script=' onto qemu cmd line
> according to what <script path=''/> contained letting qemu execute the
> script. That was flawed from security POV (we don't want qemu to be
> allowed to execute anything) thus libvirt is executing the script now.
> However, we obviously forgot about this corner case."
>
> This actually apparently first appeared in v1.3.3-rc1~71, so it's not new
> in Libvirt 2.0.0 *but*:
>
> * The Ubuntu-based gate is apparently running v1.3.1 at the moment.
> * The RHEL 7.2 and aligned CentOS included v1.2.17.
>
> This is at least partially why this was not seen until recently,
Thanks for pinning all of that down!
> but it also seems like it might be specific to certain guest networking
> setup.
Yes, there are only a few networking backends that use <interface
type=ethernet>, I believe; mine is networking-calico, which uses
VIF_TYPE_TAP; I think (from code reading) that the others are 'ivs',
'iovisor', 'midonet' and 'vrouter'. (Although another thing to keep in
mind here is that os-vif should soon be taking over this area of code - so
any fixes in nova/virt/libvirt may also be needed in os-vif.)
You mentioned you have multiple interfaces attached to the guest, which VIF
> types and associated Neutron plugins are you using?
>
Note that the multiple interfaces issue is an additional one. The empty
path issue causes all VIF_TYPE_TAP tests to fail, whether with just a
single VM interface, or multiple VM interfaces. I fixed most of those by
patching my Nova locally (as in https://review.openstack.org/#/c/411936),
but am still seeing failures in tests with multiple interfaces, which I
haven't yet investigated further.
Regards,
Neil
> Thanks,
>
> Steve
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
