Adrian: There's nothing in my comments below that we have not discussed before, but it seems like a good time to air some of my thoughts on a public thread ...
adrian cockcroft wrote: > On 9/14/07, Alexander Kolbasov <[EMAIL PROTECTED]> wrote: > > >> During any particular clock cycle do you consider a core to be busy if >> there is an instruction active? >> > Yes > > >> Is it busier if there are four >> instructions active instead of one? >> > No, the IPC of a workload is a characteristic of that workload, all > the OS can do is give clock cycles to a workload > Well, here we have to sharpen our pens on "what is a workload?" Most benchmark engineers tend to think it's "whatever is happening of the system", but I think it's more useful to consider a workload to be a particular business or technical function. In this era of consolidation and complex solutions like ERP, aggregate system utliization represents something like a "market equilibrium" from a highly heterogeneous set of workloads competing for what economists call "scarce" resources. Mixed workloads give another reason why utilization is virtually useless. The IPC of each workload can vary quite a bit due to cache and data path contention by other workloads, and even with competing activity from other virtual environments competing for some shared resources (as with LDOMs). Of course, any process can improve its IPC by spinning a lot, but that's not a good thing, eh? In other words, both CPU supply and demand are highly "elastic". CPU service times are becoming increasingly less constant and predictable. An OS *could* do far more than handing out cycles. For example, it could consider the *quality* of the cycles it's dispensing based on the elasticity factors in the CPU "supply" (such as competition from interrupts and other software threads, or memory localities) and the elasticity in the demand profile of threads (such as their priority, urgency, and performance feedback). I believe a lot could be done in adapting OR-type thinking into a new scheduler class. Current scheduling options do not resemble the problems we are trying to solve, nor have they attained the adoption rate that they warrant! I'm also concerned at the time that goes into CPU management versus other classes of service. We now live in a world where we manage I/O hogs by starving them of CPU in cases where it would clearly be better to throttle their I/O to prevent them from competing unfairly with OLTP. What's generally best is for the I/O hogs to suck up as much priority-zero CPU as they can without causing collateral damage. I believe we'll need a quantum leap in scheduling, modeling, and workload management to cope with all of this in a valuable way. I've seen commercial systems with clearly-demonstrated 400% transactional headroom at 100% CPU utilization where the customer upgraded anyway. I've seen intense angst over the capacity of "pure OLTP systems" where batch jobs, bugs, and one or two poorly-coded transactions were clearly the lead causes of the "unacceptably high utilization". I'm now seeing more cases where customers are upset at the higher utilization they see due to Solaris product enhancements, and they are generally unaware of the corresponding improvements in end-user QoS ... because they have no instrumentation there whatsoever! (It hails back to your ARM adoption rant from when we first met!) How can we get our industry onto a more intelligent track? I'm all for more accurate cycle accounting, but I also think it's a small part of the larger problem everyone is trying to solve with managing performance and capacity. Our industry's obsession with utilization reminds me of trying to fly a helicopter with only one instrument - its tachometer. I'd like to see more focus on altitude, speed, heading, attitude, visibility, and rate-of-descent! Cheers, -- Bob > >> Is it equally busy whether the >> instruction is in an execution unit, waiting on per-core cache, waiting >> on per-chip cache, or waiting on memory? >> > Yes, but it would be nice to know if its waiting on some level of the > memory hierarchy > > >> Is it any busier if it has both >> an integer and a floating point instruction active? >> > For shared FPU Niagara this is an issue, not an issue for most CPUs > By the time it could be addressed in software, will anyone still care > about Niagara? > > >> What if it's >> handling instructions for a scout thread, is that active or inactive? >> > Inactive, if its just prefetching loads, active if its executing instructions > > >> If >> there's one FP unit per 8 cores and it's active, which core gets to >> count that as active? >> > They all do, just like blocking on any shared resource. > > >> If some functional units may be dynamically >> reconfigured then is a core which owns such a unit and isn't using it >> any less busy than one which does not own such a unit? >> > No, but Intel hyperthreading and Power6 style reconfigurable pipelines > vary the work done per clock cycle independently of the application > workload, so not all clock ticks have the same value. > > >> Just being Devil's advocate. I don't doubt that you should be able to >> improve the measurement significantly. I just think that the notion of >> CPU utilization is a vague and processor dependent concept, and you >> shouldn't expect to get any perfect answers. >> > > I agree. I think that things that are largely deterministic for a > given workload (pipeline utilization) can be ignored in the OS level > metrics, there are other tools for getting at these effects. Things > that are non-deterministic (mostly sharing/coupling effects between > workloads) need to be instrumented, because they are the main cause of > surprises and problems. > > Adrian > > > >> "Everyone just needs to give 110 percent." >> University of Connecticut basketball coach Jim Calhoun >> >> >> ------- End of Forwarded Message >> >> >> >> _______________________________________________ >> perf-discuss mailing list >> perf-discuss@opensolaris.org >> >> > _______________________________________________ > perf-discuss mailing list > perf-discuss@opensolaris.org > _______________________________________________ perf-discuss mailing list perf-discuss@opensolaris.org
