At Mon, 11 Apr 2016 14:42:59 +0200, Miguel Angel Ajo Pelayo wrote: > > On Mon, Apr 11, 2016 at 11:40 AM, IWAMOTO Toshihiro > <iwam...@valinux.co.jp> wrote: > > At Fri, 8 Apr 2016 12:21:21 +0200, > > Miguel Angel Ajo Pelayo wrote: > >> > >> Hi, good that you're looking at this, > >> > >> > >> You could create a lot of ports with this method [1] and a bit of extra > >> bash, without the extra expense of instance RAM. > >> > >> > >> [1] > >> http://www.ajo.es/post/89207996034/creating-a-network-interface-to-tenant-network-in > >> > >> > >> This effort is going to be still more relevant in the context of > >> openvswitch firewall. We still need to make sure it's tested with the > >> native interface, and eventually we will need flow bundling (like in > >> ovs-ofctl --bundle add-flows) where the whole addition/removal/modification > >> is sent to be executed atomically by the switch. > > > > Bad news is that ovs-firewall isn't currently using the native > > of_interface much. I can add install_xxx methods to > > OpenFlowSwitchMixin classes so that ovs-firewall can use the native > > interface. > > Do you have a plan for implementing flow bundling or using conjunction? > > > > Adding Jakub to the thread, > > IMO, if the native interface is going to provide us with greater speed > for rule manipulation, we should look into it. > > We don't use bundling or conjunctions yet, but it's part of the plan. > Bundling will allow atomicity of operations with rules (switching > firewall rules, etc, as we have with iptables-save / > iptables-restore), and conjunctions will reduce the number of entries. > (No expansion of IP addresses for remote groups, no expansion of > security group rules per port, when several ports are on the same > security group on the same compute host). > > Do we have any metric of bare rule manipulation time (ms/rule, for example)?
No bare numbers but from a graph in the other mail I sent last week, bind_devices for 160 ports (iirc, that amounts to 800 flows) takes 4.5sec with of_interface=native and 8sec with of_interface=ovs-ofctl, which means an native add-flow is 4ms faster than the other. As the ovs firewall uses DeferredOVSBridge and has less exec overheads, I have no idea how much gain the native of_interface brings. > As a note, we're around 80 rules/port with IPv6 + IPv4 on the default > sec group plus a couple of rules. I booted 120VMs on one network and the default security group generated 62k flows. It seems using conjunction is the #1 item for performance. > > >> On Thu, Apr 7, 2016 at 10:00 AM, IWAMOTO Toshihiro <iwam...@valinux.co.jp> > >> wrote: > >> > >> > At Thu, 07 Apr 2016 16:33:02 +0900, > >> > IWAMOTO Toshihiro wrote: > >> > > > >> > > At Mon, 18 Jan 2016 12:12:28 +0900, > >> > > IWAMOTO Toshihiro wrote: > >> > > > > >> > > > I'm sending out this mail to share the finding and discuss how to > >> > > > improve with those interested in neutron ovs performance. > >> > > > > >> > > > TL;DR: The native of_interface code, which has been merged recently > >> > > > and isn't default, seems to consume less CPU time but gives a mixed > >> > > > result. I'm looking into this for improvement. > >> > > > >> > > I went on to look at implementation details of eventlet etc, but it > >> > > turned out to be fairly simple. The OVS agent in the > >> > > of_interface=native mode waits for a openflow connection from > >> > > ovs-vswitchd, which can take up to 5 seconds. > >> > > > >> > > Please look at the attached graph. > >> > > The x-axis is time from agent restarts, the y-axis is numbers of ports > >> > > processed (in treat_devices and bind_devices). Each port is counted > >> > > twice; the first slope is treat_devices and the second is > >> > > bind_devices. The native of_interface needs some more time on > >> > > start-up, but bind_devices is about 2x faster. > >> > > > >> > > The data was collected with 160 VMs with the devstack default settings. > >> > > >> > And if you wonder how other services are doing meanwhile, here is a > >> > bonus chart. > >> > > >> > The ovs agent was restarted 3 times with of_interface=native, then 3 > >> > times with of_interface=ovs-ofctl. > >> > > >> > As the test machine has 16 CPUs, 6.25% CPU usage can mean a single > >> > threaded process is CPU bound. > >> > > >> > Frankly, the OVS agent would have little rooms for improvement than > >> > other services. Also, it might be fun to draw similar charts for > >> > other types of workloads. > >> > > >> > > >> > __________________________________________________________________________ > >> > OpenStack Development Mailing List (not for usage questions) > >> > Unsubscribe: > >> > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > >> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >> > > >> > > > > > __________________________________________________________________________ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
