On Mon, 28 Nov 2005, Mike Stump wrote: > On Nov 28, 2005, at 6:21 PM, Hans-Peter Nilsson wrote: > > I've attached the work-in-progress so I don't have to get into > > detail about what it does :-) except noting that you'll see in > > gcc.sum something like: > > > > PASS: csibe -O1 runtime zlib-1.1.4:minigzip not slower than best > > PASS: csibe -O1 runtime zlib-1.1.4:minigzip not more than .1% > > slower than best > > PASS: csibe -O1 runtime zlib-1.1.4:minigzip not more than 1% slower > > than best > > PASS: csibe -O1 runtime zlib-1.1.4:minigzip not more than 10% > > slower than best > > Hum, I'd prefer that the output format be: > > PERF: %f name of test
You seem to be interpreting the gcc.sum format, thinking it's the raw "baseline" format. Which for the record is like: ... runtime,-O1,zlib-1.1.4:minigzip,previous 0.32 runtime,-O1,bzip2-1.0.2:bzip2.d,previous 0.32 runtime,-O1,bzip2-1.0.2:bzip2recover,previous 0.19 ... That was the "native" x86_64 output. Here's for cris-linux+cris-sim: .. runtime,-O1,zlib-1.1.4:minigzip,previous 1262089345.0 runtime,-O1,bzip2-1.0.2:bzip2.d,previous 945199067.0 runtime,-O1,bzip2-1.0.2:bzip2recover,previous 1555998754.0 Can't be compared with each other, if that's what you mean (how would that make sense?) but quite comparable to other baselines methinks. I refer to the implementation for further details. > then have an analysis package crunch that into the above format. ...and emitting PASS/FAIL after _comparing_ two or more arbitrary runs according to some criteria? Like above? ;-) > This way, one can _compare_ two arbitrary runs, which is a useful > property. The number can be thought of as time, such as the number > of clock cycles, but we only reeally care that it starts at zero, and > gets bigger as things go bad. You have to elaborate here. How does "biasing" the number of cycles to make it 0 help? Do you mean a deviation normalized between 0 and 1? How would you "reset" it? I think it'd just be confusing, and you'd have to expect and handle negative numbers. Better keep an explicit built-in baseline. Already implemented by looking in gcc.performance/csibe/baselines/$target_triplet and reading whatever file is there according to the format above. ...oops, a bug in the posted code; missing "$" on $subdir. And untested anyway. brgds, H-P
