Adding upstream Neutron project to this LP.
The lock contention arises from the update of the metadata_port:
https://github.com/openstack/neutron/blob/24590a334fff0ed1cb513b0f496be965bc9309d4/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py#L2111
https://github.com/openstack/neutron/blob/24590a334fff0ed1cb513b0f496be965bc9309d4/neutron/db/ipam_backend_mixin.py#L653-L680
Updating the fixed_ips field of the metadata_port will make Neutron
attempt to lock all the subnets involved regardless of
update_metadata_port only changing one subnet or all of them.
I wonder how the OVS driver dealt with this, as it would have the exact
same issue.
Perhaps the only option is for Neutron to gracefully ignore a
update_metadata_port failure at subnet creation and update the metadata
port at a later time in one of its maintenance jobs.
** Also affects: neutron
Importance: Undecided
Status: New
** Changed in: charm-neutron-api
Status: Triaged => Invalid
** Changed in: charm-neutron-api
Importance: High => Undecided
** Changed in: charm-neutron-api
Assignee: Frode Nordahl (fnordahl) => (unassigned)
** Changed in: neutron
Status: New => In Progress
** Changed in: neutron
Assignee: (unassigned) => Frode Nordahl (fnordahl)
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1890432
Title:
Create subnet is failing under high load with OVN
Status in OpenStack neutron-api charm:
Invalid
Status in neutron:
In Progress
Bug description:
Under a high concurrency level create subnet is starting to fail.
(12-14% failure rate) The bundle is OVN / Ussuri.
neutronclient.common.exceptions.Conflict: Unable to complete operation
on subnet This subnet is being modified by another concurrent
operation.
Stacktrace: https://pastebin.ubuntu.com/p/sQ5CqD6NyS/
Rally task:
{% set flavor_name = flavor_name or "m1.medium" %}
{% set image_name = image_name or "bionic-kvm" %}
---
NeutronNetworks.create_and_delete_subnets:
-
args:
network_create_args: {}
subnet_create_args: {}
subnet_cidr_start: "1.1.0.0/30"
subnets_per_network: 2
runner:
type: "constant"
times: 100
concurrency: 10
context:
network: {}
users:
tenants: 30
users_per_tenant: 1
quotas:
neutron:
network: -1
subnet: -1
Concurrency level set to 1 instead of 10 is not triggering the issue.
To manage notifications about this bug go to:
https://bugs.launchpad.net/charm-neutron-api/+bug/1890432/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
