On Mon, Jul 07, 2014 at 08:48:52PM +0400, Andrey Korolyov wrote: > On Mon, Jul 7, 2014 at 8:36 PM, Ben Pfaff <b...@nicira.com> wrote: > > On Mon, Jul 07, 2014 at 08:33:50PM +0400, Andrey Korolyov wrote: > >> On Mon, Jul 7, 2014 at 7:15 PM, Ben Pfaff <b...@nicira.com> wrote: > >> > On Fri, Jul 04, 2014 at 09:02:40PM +0400, Andrey Korolyov wrote: > >> >> On Fri, Jul 4, 2014 at 8:39 PM, Ben Pfaff <b...@nicira.com> wrote: > >> >> > On Fri, Jul 04, 2014 at 02:39:18PM +0400, Andrey Korolyov wrote: > >> >> >> is there any plans around to introduce built-in performance > >> >> >> statistics > >> >> >> for userspace switch process? Right now one is able to check memory > >> >> >> consumption statistics and it will be quite awesome to introduce kind > >> >> >> of interface for measurement of thread performance too. > >> >> > > >> >> > We have performance counters. What do you want to measure? > >> >> > >> >> For example, total cpu utilization by workers not involving external > >> >> monitoring tools. There are two cases - reactive flows, which > >> >> currently have almost linear characteristic for utilization vs flow > >> >> rate when adding unique flows on moderate speed (30k/s with six cores > >> >> of E5 series as a ceiling), and proactive mask/asterisk flows which > >> >> has sublinear characteristic even being far away from the actual > >> >> ceiling. In the second case vswitchd heavily relies on decisions made > >> >> inside module` heuristic, so same flow hit rate can produce very > >> >> different cpu utilization for userspace vswitchd. > >> > > >> > I'm not sure how OVS would do a better job of that than external > >> > monitoring tools. Doing a good job would probably mean producing a time > >> > series for each thread's CPU usage. A regular log file isn't ideal for > >> > that. > >> > >> Ok, I see. Jiffie counting is a pidstat job, but I thought that there > >> is a easier way around. > > > > Are you talking about performance monitoring for work done in the kernel > > module (e.g. from bottom halves etc.)? I'm only talking about > > userspace. I don't know the relevant kernel performance monitoring > > infrastructure and can't speak for the right way to do it. > > No, just userspace. Bh profiling looks unnecessary since 2.1+ module > doing very great job as a packet processor. I was afraid that the > unmonitored vswitchd process may eat significant part of the compute > node and may get unnoticed with couple of conditions, e.g. relatively > small packet rate with high flow rate or high hit rate on the reactive > flows (if controller is stupid enough to just report the rate, not to > analyze/group asking behavior). Of course, most straightforward way to > prevent this is to check process consumption via pidstat or another > top-like jiffie counter, I just thought of some ready-to-use mechanism > inside vswitchd itself which I was not simply aware of.
OVS has a mechanism (for years) that's designed to point out code that causes 100% consumption: if a thread starts using more than 50% of a CPU, then it starts logging its own CPU consumption and the line of code that caused the wakeup. I don't know whether that is good enough for your needs. _______________________________________________ discuss mailing list discuss@openvswitch.org http://openvswitch.org/mailman/listinfo/discuss
