On Wed, May 13, 2015 at 6:05 AM, Guillermo Ontañón <guille...@midokura.com> wrote: > On Wed, May 13, 2015 at 1:00 AM, Ben Pfaff <b...@nicira.com> wrote: >> On Tue, May 12, 2015 at 10:21:19AM +0200, Duarte Nunes wrote: >> Second, is it possible to use a better data structure? Perhaps one >> could, for example, use a mutex per hash chain, instead of a single >> mutex, or per-CPU data structures. Ideally, if the data structures were >> improved, then one would not need to change the datapath interface at >> all. > > We can look at other options, of course. > > IIUC, per-CPU data structures sound like they'd require table lookups > to spill over to other CPUs after a miss, which sounds bad, and makes > me think I didn't understand correctly, can you elaborate? > > A mutex per hash-chain would work, the number of buckets is high > enough to make contention rare. It means: > > * A mutex per hash table bucket in the flow table. > * A mutex for the flow mask list, grab it when a flow operation needs > to modify it. > * Grab the bucket mutex instead of ovs_mutex in flow CRUD ops. > * Two operations need to grab all per-bucket locks in a flow table and > the mask list mutex: > * Flow table flush. > * Datapath deletion. > > While doable, to us it sounds like it would be both more complicated > and less scalable than just sharding of flow tables in a way that's > known to userspace. The interface changes would be minimal: > > * Datapaths get an optional number-of-shards attribute. > * Packets that ovs punts to userspace get a new attribute: the hash. > If part of the flow id, this becomes unnecessary. > > ..and we can forget about passing hash functions around or making them > known to userspace.
This is basically an internal kernel optimization, so I agree with Ben that I would prefer not making userspace aware of this. I don't really think that changing the locking along the lines of what is described above is actually more complicated anyways - presumably the sharding that you're talking about will also require similar lock changes. I suspect that there is a decent amount of headroom to be gained without making interfaces changes, so that seems like a better place to experiment as it doesn't commit us to anything long term before we know what the tradeoffs are. Plus, it's not clear to me that things like connection tracking will ever perform that great going up to userspace for every flow - which is why in kernel connection tracking support is being worked on. _______________________________________________ dev mailing list dev@openvswitch.org http://openvswitch.org/mailman/listinfo/dev
