Reviewed: https://review.opendev.org/c/openstack/neutron/+/929941
Committed:
https://opendev.org/openstack/neutron/commit/c1b05e29adf9d0d68c1ac636013a8a363a92eb85
Submitter: "Zuul (22348)"
Branch: master
commit c1b05e29adf9d0d68c1ac636013a8a363a92eb85
Author: Rodolfo Alonso Hernandez <ralon...@redhat.com>
Date: Thu Sep 19 14:00:57 2024 +0000
Change the load method of SG rule "default_security_group"
Since [1], the SG rule SQL view also retrieves the table
"default_security_group", using a complex relationship [2].
When the number of SG rules of a SG is high (above 50 it
is clearly noticeable the performance degradation), the
API call can take several seconds. For example, for 100
SG rules it can take up to one minute.
This patch changes the load method of the SG rule
"default_security_group" relationship to "selectin".
Benchmarks with a single default SG and 100 rules,
doing "openstack security group show $sg":
* 2023.2 (without this feature): around 0.05 seconds
* master: between 45-50 seconds (1000x time increase)
* loading method "selectin" or "dynamic": around 0.5 seconds.
NOTE: this feature [1] was implemented in 2024.1. At this
time, SQLAlchemy version was <2.0 and "selectin" method was
not available. For this version, "dynamic" can be used instead.
[1]https://review.opendev.org/q/topic:%22bug/2019960%22
[2]https://github.com/openstack/neutron/blob/08fff4087dc342be40db179fca0cd9bbded91053/neutron/db/models/securitygroup.py#L120-L121
Closes-Bug: #2081087
Change-Id: I46af1179f6905307c0d60b5c0fdee264a40a4eac
** Changed in: neutron
Status: In Progress => Fix Released
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/2081087
Title:
Performance regression in neutron-server from 2023.1 to 2024.1 when
fetching a Security Group
Status in neutron:
Fix Released
Bug description:
With upgrade from 2023.1 to 2024.1 with driver ML2/OVS we've spotted a
significant (10 times) performance regression on some operations.
As best example - we can take security groups operations.
Neutron is running in eventlet, since uWSGI is not yet fully
functional for 2024.1 (see
https://review.opendev.org/c/openstack/neutron/+/926922).
So neutron-server is just being launched with exactly same database
and config, just from different venvs.
```
# cat /etc/systemd/system/neutron-server.service
[Unit]
Description = neutron-server service
After = network-online.target
After = syslog.target
[Service]
Type = simple
User = neutron
Group = neutron
ExecStart = /openstack/venvs/neutron-29.0.2/bin/neutron-server --config-file
/etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini
ExecReload = /bin/kill -HUP $MAINPID
# Give a reasonable amount of time for the server to start up/shut down
TimeoutSec = 120
Restart = on-failure
RestartSec = 2
# This creates a specific slice which all services will operate from
# The accounting options give us the ability to see resource usage through
# the `systemd-cgtop` command.
Slice = neutron.slice
# Set Accounting
CPUAccounting = True
BlockIOAccounting = True
MemoryAccounting = True
TasksAccounting = True
# Set Sandboxing
PrivateTmp = False
PrivateDevices = False
PrivateNetwork = False
PrivateUsers = False
[Install]
WantedBy = multi-user.target
# time curl -X GET
http://127.0.0.1:9696/v2.0/security-groups?project_id=${OS_PROJECT_ID} -H
"X-Auth-Token: ${TOKEN}"
...
real 0m24.450s
user 0m0.008s
sys 0m0.010s
# time curl -X GET
http://127.0.0.1:9696/v2.0/security-groups/${security_group_uuid} -H
"X-Auth-Token: ${TOKEN}"
...
real 0m54.841s
user 0m0.010s
sys 0m0.012s
# sed -i 's/29.0.2/27.4.0/g' /etc/systemd/system/neutron-server.service
# systemctl daemon-reload
# systemctl restart neutron-server
# time curl -X GET
http://127.0.0.1:9696/v2.0/security-groups?project_id=${OS_PROJECT_ID} -H
"X-Auth-Token: ${TOKEN}"
...
real 0m1.040s
user 0m0.011s
sys 0m0.007s
# time curl -X GET
http://127.0.0.1:9696/v2.0/security-groups/${security_group_uuid} -H
"X-Auth-Token: ${TOKEN}"
...
real 0m0.589s
user 0m0.012s
sys 0m0.007s
```
So as you might see, difference in response time is very significant,
while the only change I've made is to use previous codebase for the
service.
I am also providing pip freeze for both venvs for comparison, though both of
them were using upper-constraints:
# /openstack/venvs/neutron-27.4.0/bin/pip freeze
alembic==1.8.1
amqp==5.1.1
appdirs==1.4.4
attrs==22.1.0
autopage==0.5.1
bcrypt==4.0.0
cachetools==5.2.0
certifi==2023.11.17
cffi==1.15.1
charset-normalizer==2.1.1
cliff==4.2.0
cmd2==2.4.2
cryptography==38.0.2
debtcollector==2.5.0
decorator==5.1.1
dnspython==2.2.1
dogpile.cache==1.1.8
eventlet==0.33.1
extras==1.0.0
fasteners==0.18
fixtures==4.0.1
futurist==2.4.1
greenlet==1.1.3
httplib2==0.20.4
idna==3.4
importlib-metadata==5.0.0
iso8601==1.1.0
Jinja2==3.1.2
jmespath==1.0.1
jsonpatch==1.32
jsonpointer==2.3
jsonschema==4.16.0
keystoneauth1==5.1.2
keystonemiddleware==10.2.0
kombu==5.2.4
logutils==0.3.5
lxml==4.9.1
Mako==1.2.3
MarkupSafe==2.1.1
msgpack==1.0.4
ncclient==0.6.13
netaddr==0.8.0
netifaces==0.11.0
neutron==22.1.1.dev52
neutron-dynamic-routing==22.0.0
neutron-lib==3.4.1
neutron-vpnaas==22.0.1.dev2
openstacksdk==1.0.1
os-client-config==2.1.0
os-ken==2.6.0
os-resource-classes==1.1.0
os-service-types==1.7.0
os-traits==2.10.0
os-vif==3.1.1
osc-lib==2.7.0
oslo.cache==3.3.1
oslo.concurrency==5.1.1
oslo.config==9.1.1
oslo.context==5.6.0
oslo.db==12.3.1
oslo.i18n==6.0.0
oslo.log==5.2.0
oslo.messaging==14.7.1
oslo.metrics==0.6.0
oslo.middleware==5.1.1
oslo.policy==4.1.1
oslo.privsep==3.1.0
oslo.reports==3.0.0
oslo.rootwrap==7.0.1
oslo.serialization==5.1.1
oslo.service==3.1.1
oslo.upgradecheck==2.1.1
oslo.utils==6.1.0
oslo.versionedobjects==3.1.0
osprofiler==3.4.3
ovs==2.17.1.post1
ovsdbapp==2.2.1
packaging==21.3
paramiko==2.11.0
Paste==3.5.2
PasteDeploy==2.1.1
pbr==5.11.1
pecan==1.4.2
prettytable==3.4.1
prometheus-client==0.14.1
psutil==5.9.2
pycadf==3.1.1
pycparser==2.21
pyinotify==0.9.6
pymemcache==3.5.2
PyMySQL==1.0.2
PyNaCl==1.5.0
pyOpenSSL==22.1.0
pyparsing==3.0.9
pyperclip==1.8.2
pyroute2==0.7.3
pyrsistent==0.18.1
python-binary-memcached==0.31.1
python-dateutil==2.8.2
python-designateclient==5.2.0
python-keystoneclient==5.1.0
python-memcached==1.59
python-neutronclient==9.0.0
python-novaclient==18.3.0
pytz==2022.4
PyYAML==6.0
repoze.lru==0.7
requests==2.28.1
requestsexceptions==1.4.0
rfc3986==1.5.0
Routes==2.5.1
setproctitle==1.3.2
simplejson==3.17.6
six==1.16.0
sortedcontainers==2.4.0
SQLAlchemy==1.4.41
sqlalchemy-migrate==0.13.0
sqlparse==0.4.3
statsd==3.3.0
stevedore==5.0.0
systemd-python==234
Tempita==0.5.2
tenacity==6.3.1
testresources==2.0.1
testscenarios==0.5.0
testtools==2.5.0
tooz==3.2.0
uhashring==2.1
urllib3==1.26.12
vine==5.0.0
voluptuous==0.13.1
wcwidth==0.2.5
WebOb==1.8.7
wrapt==1.14.1
yappi==1.4.0
zipp==3.8.1
# /openstack/venvs/neutron-29.0.2/bin/pip freeze
alembic==1.9.4
amqp==5.2.0
async-timeout==4.0.3
attrs==23.2.0
autopage==0.5.2
bcrypt==4.1.2
cachetools==5.3.2
certifi==2024.8.30
cffi==1.16.0
charset-normalizer==3.3.2
cliff==4.6.0
cmd2==2.4.3
cryptography==42.0.4
debtcollector==3.0.0
decorator==5.1.1
dnspython==2.6.1
dogpile.cache==1.3.1
eventlet==0.36.1
fasteners==0.19
fixtures==4.1.0
futurist==3.0.0
greenlet==3.0.3
httplib2==0.22.0
idna==3.6
iso8601==2.1.0
Jinja2==3.1.3
jmespath==1.0.1
jsonpatch==1.33
jsonpointer==2.4
jsonschema==4.19.2
jsonschema-specifications==2023.12.1
keystoneauth1==5.6.0
keystonemiddleware==10.6.0
kombu==5.3.5
logutils==0.3.5
lxml==5.1.0
Mako==1.3.2
MarkupSafe==2.1.5
msgpack==1.0.7
ncclient==0.6.15
netaddr==0.10.1
netifaces==0.11.0
neutron==24.0.2.dev15
neutron-dynamic-routing==24.0.1.dev2
neutron-lib==3.11.0
neutron-vpnaas==24.0.2.dev1
openstacksdk==3.0.0
os-client-config==2.1.0
os-ken==2.8.1
os-resource-classes==1.1.0
os-service-types==1.7.0
os-traits==3.0.0
os-vif==3.5.0
osc-lib==3.0.1
oslo.cache==3.7.0
oslo.concurrency==6.0.0
oslo.config==9.4.0
oslo.context==5.5.0
oslo.db==15.0.0
oslo.i18n==6.3.0
oslo.log==5.5.1
oslo.messaging==14.7.1
oslo.metrics==0.8.0
oslo.middleware==6.1.0
oslo.policy==4.3.0
oslo.privsep==3.3.0
oslo.reports==3.3.0
oslo.rootwrap==7.2.0
oslo.serialization==5.4.0
oslo.service==3.4.1
oslo.upgradecheck==2.3.0
oslo.utils==7.1.0
oslo.versionedobjects==3.3.0
osprofiler==4.1.0
ovs==3.1.2
ovsdbapp==2.6.1
packaging==23.2
paramiko==3.4.0
Paste==3.7.1
PasteDeploy==3.1.0
pbr==6.0.0
pecan==1.5.1
platformdirs==4.2.0
prettytable==3.9.0
prometheus_client==0.20.0
psutil==5.9.8
pycadf==3.1.1
pycparser==2.21
pyinotify==0.9.6
PyJWT==2.8.0
pymemcache==4.0.0
PyMySQL==1.1.0
PyNaCl==1.5.0
pyOpenSSL==24.0.0
pyparsing==3.1.1
pyperclip==1.8.2
pyroute2==0.7.12
python-binary-memcached==0.31.2
python-dateutil==2.8.2
python-designateclient==6.0.1
python-keystoneclient==5.4.0
python-memcached==1.62
python-neutronclient==11.2.0
python-novaclient==18.6.0
pytz==2024.1
PyYAML==6.0.1
redis==5.0.1
referencing==0.33.0
repoze.lru==0.7
requests==2.31.0
requestsexceptions==1.4.0
rfc3986==2.0.0
Routes==2.5.1
rpds-py==0.18.0
setproctitle==1.3.3
simplejson==3.19.2
six==1.16.0
sortedcontainers==2.4.0
SQLAlchemy==1.4.51
statsd==4.0.1
stevedore==5.2.0
systemd-python==235
tenacity==8.2.3
testresources==2.0.1
testscenarios==0.5.0
testtools==2.7.1
tooz==6.2.0
typing_extensions==4.9.0
tzdata==2024.1
uhashring==2.3
urllib3==1.26.18
vine==5.1.0
voluptuous==0.14.2
wcwidth==0.2.13
WebOb==1.8.7
wrapt==1.16.0
yappi==1.6.0
#
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/2081087/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
