Public bug reported: https://review.openstack.org/255334 Dear bug triager. This bug was created since a commit was marked with DOCIMPACT. Your project "openstack/neutron" is set up so that we directly report the documentation bugs against it. If this needs changing, the docimpact-group option needs to be added for the project. You can ask the OpenStack infra team (#openstack-infra on freenode) for help if you need to.
commit fb55693713991782a56006af73e0ee67cffe9a56 Author: Kevin Benton <[email protected]> Date: Sat Oct 3 07:25:19 2015 -0700 Use diffs for iptables restore instead of all rules This patch changes our iptables logic to generate a delta of iptables commands (inserts + deletes) to get from the current iptables state to the new state. This will significantly reduce the amount of data that we have to shell out to iptables-restore on every call (and reduce the amount of data iptables-restore has to parse). We no longer have to worry about preserving counters since we are adding and deleting specific rules, so the rule modification code got a nice cleanup to get rid of the old rule matching. This also gives us a new method of functionally testing that we are generating rules in the correct manner. After applying new rules once, a subsequent call should always have no work to do. The new functional tests added leverage that property heavily and should protect us from regressions in how rules are formed. Performance metrics relative to HEAD~1: +====================================+============+=======+ | Scenario | This patch | HEAD~1| |------------------------------------|------------|-------| | 200 VMs*22 rules existing - startup| | | | _modify_rules| 0.67s | 1.05s | | _apply_synchronized| 1.87s | 2.89s | |------------------------------------|------------|-------| | 200 VMs*22 rules existing - add VM | | | | _modify_rules| 0.68s | 1.05s | | _apply_synchronized| 2.07s | 2.92s | |------------------------------------+------------+-------+ |200 VMs*422 rules existing - startup| | | | _modify_rules| 5.43s | 8.17s | | _apply_synchronized| 12.77s |28.00s | |------------------------------------|------------|-------| |200 VMs*422 rules existing - add VM | | | | _modify_rules| 6.41s | 8.33s | | _apply_synchronized| 33.09s |33.80s | +------------------------------------+------------+-------+ The _apply_synchronized times seem to converge when dealing with ~85k rules. In the profile I can see that both approaches seem to wait on iptables-restore for approximately the same amount of time so it could be hitting the performance limits of iptables-restore. DocImpact Partial-Bug: #1502297 Change-Id: Ia6470c85b6b71979006ffe5da9095fdcce3122c1 (cherry picked from commit f066e46bb77362ccca838b541eb42c7ae8ddc125) ** Affects: neutron Importance: Undecided Status: New ** Tags: neutron -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1526967 Title: Use diffs for iptables restore instead of all rules Status in neutron: New Bug description: https://review.openstack.org/255334 Dear bug triager. This bug was created since a commit was marked with DOCIMPACT. Your project "openstack/neutron" is set up so that we directly report the documentation bugs against it. If this needs changing, the docimpact-group option needs to be added for the project. You can ask the OpenStack infra team (#openstack-infra on freenode) for help if you need to. commit fb55693713991782a56006af73e0ee67cffe9a56 Author: Kevin Benton <[email protected]> Date: Sat Oct 3 07:25:19 2015 -0700 Use diffs for iptables restore instead of all rules This patch changes our iptables logic to generate a delta of iptables commands (inserts + deletes) to get from the current iptables state to the new state. This will significantly reduce the amount of data that we have to shell out to iptables-restore on every call (and reduce the amount of data iptables-restore has to parse). We no longer have to worry about preserving counters since we are adding and deleting specific rules, so the rule modification code got a nice cleanup to get rid of the old rule matching. This also gives us a new method of functionally testing that we are generating rules in the correct manner. After applying new rules once, a subsequent call should always have no work to do. The new functional tests added leverage that property heavily and should protect us from regressions in how rules are formed. Performance metrics relative to HEAD~1: +====================================+============+=======+ | Scenario | This patch | HEAD~1| |------------------------------------|------------|-------| | 200 VMs*22 rules existing - startup| | | | _modify_rules| 0.67s | 1.05s | | _apply_synchronized| 1.87s | 2.89s | |------------------------------------|------------|-------| | 200 VMs*22 rules existing - add VM | | | | _modify_rules| 0.68s | 1.05s | | _apply_synchronized| 2.07s | 2.92s | |------------------------------------+------------+-------+ |200 VMs*422 rules existing - startup| | | | _modify_rules| 5.43s | 8.17s | | _apply_synchronized| 12.77s |28.00s | |------------------------------------|------------|-------| |200 VMs*422 rules existing - add VM | | | | _modify_rules| 6.41s | 8.33s | | _apply_synchronized| 33.09s |33.80s | +------------------------------------+------------+-------+ The _apply_synchronized times seem to converge when dealing with ~85k rules. In the profile I can see that both approaches seem to wait on iptables-restore for approximately the same amount of time so it could be hitting the performance limits of iptables-restore. DocImpact Partial-Bug: #1502297 Change-Id: Ia6470c85b6b71979006ffe5da9095fdcce3122c1 (cherry picked from commit f066e46bb77362ccca838b541eb42c7ae8ddc125) To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1526967/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
