Reviewed: https://review.openstack.org/628691 Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=2eb31f84c9a6c9fc6340819f756a7a82cbf395f3 Submitter: Zuul Branch: master
commit 2eb31f84c9a6c9fc6340819f756a7a82cbf395f3 Author: Doug Wiegley <[email protected]> Date: Fri Jan 4 14:55:29 2019 -0700 Fix performance regression adding rules to security groups Sometime between liberty and pike, adding rules to SG's got slow, and slower with every rule. Streamline the rule create path, and get close to the old performance back. Two performance fixes: 1. Get rid of an n^2 duplicate check, using a hash table instead, on bulk creates. This is more memory intensive than the previous loop, but usable far past where the other becomes too slow to be useful. 2. Use an object existence check in a few places where we do not want to load all of the child rules. Co-Authored-By: William Hager <[email protected]> Change-Id: I34e41a128f28211f2e7ab814a2611ce22620fcf3 Closes-bug: 1810563 ** Changed in: neutron Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1810563 Title: adding rules to security groups is slow Status in neutron: Fix Released Bug description: Sometime between liberty and pike, adding rules to SG's got slow, and slower with every rule added. Gerrit review with fixes is incoming. You can repro with a vanilla devstack install on master, and this script: #!/bin/bash OPENSTACK_TOKEN=$(openstack token issue | grep '| id' | awk '{print $4}') export OPENSTACK_TOKEN CCN1=10.210.162.2 CCN3=10.210.162.10 export ENDPOINT=localhost make_rules() { iter=$1 prefix=$2 file="$3" echo "generating rules" cat >$file <<EOF {"security_group_rules":[ EOF comma="," i=0 while [ $i -lt $iter ]; do j=0 while [ $j -lt 10 ]; do if [ $i -eq $(($iter-1)) -a $j -eq 9 ]; then comma="" fi cat >>$file <<EOF {"direction":"ingress","ethertype":"IPv4","port_range_max":10000,"port_range_min":8000,"protocol":"tcp" ,"remote_ip_prefix":"$prefix.$i.$j.0/24","security_group_id":"$SG_UUID"}${comma} EOF j=$((j+1)) done i=$((i+1)) done cat >>$file <<EOF ]} EOF } hit_api() { json="$1" echo "hitting api" start=$(perl -e "print time();") time curl --silent -g -i -X POST http://$ENDPOINT:9696/v2.0/security-group-rules.json -H "User-Agen t: python-neutronclient" -H "Content-Type: application/json" -H "Accept: application/json" -H "X-Auth-T oken: $OPENSTACK_TOKEN" -d @${json} >/dev/null end=$(perl -e "print time();") echo $((end-start)) } tmp=/tmp/sg-test.$$.tmp echo "Doing test with 1000 rules in bulk" openstack security group delete dw-test-1 uuid=$(openstack security group create dw-test-1 | grep '| id' | awk '{print $4}') export SG_UUID="$uuid" make_rules 100 4 $tmp hit_api $tmp echo "Doing loop test" openstack security group delete dw-test-2 uuid=$(openstack security group create dw-test-2 | grep '| id' | awk '{print $4}') export SG_UUID="$uuid" elapsed=0 mm=0 while [ $mm -lt 20 ]; do make_rules 5 $(($mm+1)) $tmp n=$(hit_api $tmp | tail -1) elapsed=$((elapsed+n)) mm=$((mm+1)) done echo "Loop test took $elapsed seconds" To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1810563/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
