Looks more like a libvirt error, or nova side problem. Neutron does not
take responsibilities to create the tap-XXX device. It is plugged by
nova-compute. Need to find out why the tap device is not created before
TC rules creating.
** Also affects: nova
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/2051863
Title:
Race condition during VM creation - could not open network device
tapXXX (No such device)
Status in neutron:
New
Status in OpenStack Compute (nova):
New
Bug description:
[High-Level Description]
While creating an amphora from the Octavia service, we are encountering a
race condition. Nova-compute is unable to add a QC ingress rule because the tap
interface does not exist yet. This situation results in the service becoming
temporarily unavailable, failing in approximately 50% of cases.
[Pre-conditions]
Create a simple octavia loadbalancer with environment deployed by
kolla-environment. For amphora server, use flavor with qos properties as from
example: https://paste.openstack.org/show/bkx3RgEFqUwxCU0Qr8z2/
[Step-by-Step Reproduction]
Create a load balancer as part of a Heat stack, following the example
provided at:
https://github.com/syseleven/heat-examples/tree/master/lbaas-octavia-http
Use a qos flavor as shown in example:
https://paste.openstack.org/show/bkx3RgEFqUwxCU0Qr8z2/
[Expected Output]
Expect 100% successful creation of the amphora and subsequent Nova instances
serving as members for the load balancer.
[Actual Output]
During the creation process, it is observed that nova-compute fails due to:
/var/log/kolla/libvirt/libvirtd.log:2024-01-31 09:24:55.852+0000: 4065628:
error : virCommandWait:2748 : internal error: Child process (tc filter add dev
tap5f365c23-89 parent ffff: protocol all u32 match u32 0 0 police rate
64000kbps burst 64000kb mtu 64kb drop flowid :1) unexpected exit status 2:
Error: Parent Qdisc doesn't exists.
/var/log/kolla/nova/nova-compute.log:2024-01-31 10:24:56.086 7 ERROR
nova.virt.libvirt.guest libvirt.libvirtError: internal error: Child process (tc
filter add dev tap5f365c23-89 parent ffff: protocol all u32 match u32 0 0
police rate 64000kbps burst 64000kb mtu 64kb drop flowid :1) unexpected exit
status 2: Error: Parent Qdisc doesn't exists.
Unfortunately ovs-vswitchd.log file says that tap interface was not created
at this point:
/var/log/kolla/openvswitch/ovs-vswitchd.log:2024-01-31T09:24:55.469Z|00094|bridge|WARN|could
not open network device tap5f365c23-89 (No such device)
/var/log/kolla/openvswitch/ovs-vswitchd.log:2024-01-31T09:24:55.809Z|00095|bridge|INFO|bridge
br-int: added interface tap5f365c23-89 on port 27
/var/log/kolla/openvswitch/ovs-vswitchd.log:2024-01-31T09:24:55.885Z|00096|bridge|INFO|bridge
br-int: deleted interface tap5f365c23-89 on port 27
/var/log/kolla/openvswitch/ovs-vswitchd.log:2024-01-31T09:24:55.888Z|00097|bridge|WARN|could
not open network device tap5f365c23-89 (No such device)
/var/log/kolla/openvswitch/ovs-vswitchd.log:2024-01-31T09:24:55.927Z|00098|bridge|WARN|could
not open network device tap5f365c23-89 (No such device)
/var/log/kolla/openvswitch/ovs-vswitchd.log:2024-01-31T09:24:55.934Z|00099|bridge|WARN|could
not open network device tap5f365c23-89 (No such device)
/var/log/kolla/openvswitch/ovs-vswitchd.log:2024-01-31T09:24:55.943Z|00100|bridge|WARN|could
not open network device tap5f365c23-89 (No such device)
/var/log/kolla/openvswitch/ovs-vswitchd.log:2024-01-31T09:24:56.024Z|00101|bridge|WARN|could
not open network device tap5f365c23-89 (No such device)
[Version]
OpenStack Zed, deployed by kolla-ansible with all defaults
Ubuntu 22.04 LTS
libvirt: 8.0.0-1ubuntu7.7
nova: zed
python3-openvswitch: 3.0.3-0ubuntu0.22.10.3~cloud3
neutron: zed
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/2051863/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
