On 9/14/07, adrian cockcroft <[EMAIL PROTECTED]> wrote: > Hi Bob, > > I think what is being discussed here is necessary (to cope with > virtualization) but not sufficient (to cope with actual customer > problems and workloads). > > What I meant by a workload was conceptually an address space. The > scheduler is basically in the business of handing out CPU time slots > to threads in address spaces, the first necessary fix is to relax the > current assumptions that a slot of CPU time has a value equal to its > duration, and that there is a constant supply of time slots.
Agreed > Variable clock rates and the like vary the value of a slot of CPU > time, and virtualization and other issues vary the supply. Thats why I > think it would be a useful step in the right direction to know how > many clock cycles were issued to each thread, and what the global > supply of clock cycles was over a measurement interval. I think I understand where you are going with this. Please correct me if I am wrong. Theoretically if the power scaling is performing ideally, a system will always be running at 100% of traditional "utilization". In other words, ever available clock cycle will be consumed, because only the cycles needed will be clocked. Taking it further,utilization of the system would now be measured as the total cycles/second divided the max clock sum of all the cores. (I would just have to divide the sum of current clock speeds by the sum of max clock speeds.) Is this correct? > There is a separate discussion to figure out how to measure things > like transactions and relate them to resource consumption and > utilization/headroom. > > Cheers > Adrian > > On 9/14/07, Bob Sneed, SMI PAE <[EMAIL PROTECTED]> wrote: > > > > Adrian: > > > > There's nothing in my comments below that we have not discussed before, > > but it seems like a good time to air some of my thoughts on a public > > thread ... > > > > adrian cockcroft wrote: > > > On 9/14/07, Alexander Kolbasov <[EMAIL PROTECTED]> wrote: > > > > > > > > >> During any particular clock cycle do you consider a core to be busy if > > >> there is an instruction active? > > >> > > > Yes > > > > > > > > >> Is it busier if there are four > > >> instructions active instead of one? > > >> > > > No, the IPC of a workload is a characteristic of that workload, all > > > the OS can do is give clock cycles to a workload > > > > > > > Well, here we have to sharpen our pens on "what is a workload?" Most > > benchmark engineers tend to think it's "whatever is happening of the > > system", > > but I think it's more useful to consider a workload to be a particular > > business > > or technical function. In this era of consolidation and complex > > solutions like > > ERP, aggregate system utliization represents something like a "market > > equilibrium" from a highly heterogeneous set of workloads competing for > > what economists call "scarce" resources. > > > > Mixed workloads give another reason why utilization is virtually useless. > > The IPC of each workload can vary quite a bit due to cache and data path > > contention by other workloads, and even with competing activity from other > > virtual environments competing for some shared resources (as with LDOMs). > > Of course, any process can improve its IPC by spinning a lot, but that's > > not > > a good thing, eh? In other words, both CPU supply and demand are highly > > "elastic". CPU service times are becoming increasingly less constant and > > predictable. > > > > An OS *could* do far more than handing out cycles. For example, it could > > consider the *quality* of the cycles it's dispensing based on the > > elasticity factors > > in the CPU "supply" (such as competition from interrupts and other software > > threads, or memory localities) and the elasticity in the demand profile > > of threads > > (such as their priority, urgency, and performance feedback). I believe > > a lot > > could be done in adapting OR-type thinking into a new scheduler class. > > Current > > scheduling options do not resemble the problems we are trying to solve, nor > > have they attained the adoption rate that they warrant! > > > > I'm also concerned at the time that goes into CPU management versus other > > classes of service. We now live in a world where we manage I/O hogs by > > starving them of CPU in cases where it would clearly be better to > > throttle their > > I/O to prevent them from competing unfairly with OLTP. What's generally > > best is for the I/O hogs to suck up as much priority-zero CPU as they can > > without causing collateral damage. > > > > I believe we'll need a quantum leap in scheduling, modeling, and workload > > management to cope with all of this in a valuable way. I've seen commercial > > systems with clearly-demonstrated 400% transactional headroom at 100% > > CPU utilization where the customer upgraded anyway. I've seen intense > > angst over the capacity of "pure OLTP systems" where batch jobs, bugs, > > and one or two poorly-coded transactions were clearly the lead causes of the > > "unacceptably high utilization". I'm now seeing more cases where customers > > are upset at the higher utilization they see due to Solaris product > > enhancements, > > and they are generally unaware of the corresponding improvements in end-user > > QoS ... because they have no instrumentation there whatsoever! (It > > hails back > > to your ARM adoption rant from when we first met!) How can we get our > > industry onto a more intelligent track? > > > > I'm all for more accurate cycle accounting, but I also think it's a > > small part of > > the larger problem everyone is trying to solve with managing performance and > > capacity. Our industry's obsession with utilization reminds me of trying to > > fly a helicopter with only one instrument - its tachometer. I'd like to > > see more > > focus on altitude, speed, heading, attitude, visibility, and > > rate-of-descent! > > > > Cheers, > > -- Bob > > > > > >> Is it equally busy whether the > > >> instruction is in an execution unit, waiting on per-core cache, waiting > > >> on per-chip cache, or waiting on memory? > > >> > > > Yes, but it would be nice to know if its waiting on some level of the > > > memory hierarchy > > > > > > > > >> Is it any busier if it has both > > >> an integer and a floating point instruction active? > > >> > > > For shared FPU Niagara this is an issue, not an issue for most CPUs > > > By the time it could be addressed in software, will anyone still care > > > about Niagara? > > > > > > > > >> What if it's > > >> handling instructions for a scout thread, is that active or inactive? > > >> > > > Inactive, if its just prefetching loads, active if its executing > > > instructions > > > > > > > > >> If > > >> there's one FP unit per 8 cores and it's active, which core gets to > > >> count that as active? > > >> > > > They all do, just like blocking on any shared resource. > > > > > > > > >> If some functional units may be dynamically > > >> reconfigured then is a core which owns such a unit and isn't using it > > >> any less busy than one which does not own such a unit? > > >> > > > No, but Intel hyperthreading and Power6 style reconfigurable pipelines > > > vary the work done per clock cycle independently of the application > > > workload, so not all clock ticks have the same value. > > > > > > > > >> Just being Devil's advocate. I don't doubt that you should be able to > > >> improve the measurement significantly. I just think that the notion of > > >> CPU utilization is a vague and processor dependent concept, and you > > >> shouldn't expect to get any perfect answers. > > >> > > > > > > I agree. I think that things that are largely deterministic for a > > > given workload (pipeline utilization) can be ignored in the OS level > > > metrics, there are other tools for getting at these effects. Things > > > that are non-deterministic (mostly sharing/coupling effects between > > > workloads) need to be instrumented, because they are the main cause of > > > surprises and problems. > > > > > > Adrian > > > > > > > > > > > >> "Everyone just needs to give 110 percent." > > >> University of Connecticut basketball coach Jim Calhoun > > >> > > >> > > >> ------- End of Forwarded Message > > >> > > >> > > >> > > >> _______________________________________________ > > >> perf-discuss mailing list > > >> perf-discuss@opensolaris.org > > >> > > >> > > > _______________________________________________ > > > perf-discuss mailing list > > > perf-discuss@opensolaris.org > > > > > > > > _______________________________________________ > perf-discuss mailing list > perf-discuss@opensolaris.org > -- - Brian Gupta http://opensolaris.org/os/project/nycosug/ _______________________________________________ perf-discuss mailing list perf-discuss@opensolaris.org
