On Thursday 04 December 2003 22:51, Michael G Schwern wrote: > Calibration would be pretty straight forward. Just have a handful of > loops to run known snippets of Perl and calibrate the pmip based on how long > they take to run. This would change from Perl version to Perl version > and platform to platform, but you can find something that's not off by more > than 25%.
I'm not sure about that 25%. Say the pmip calibrator doesn't fit in the CPU cache on any machine, then if the tested algorithm fits in the CPU cache on one machine but not on another then there will be a huge difference in the number of "perl seconds" they require. Worse still, if the pmip calibrator fits in the cache on my fancy new machine I'll probably get lots of false negatives because everything seems taking "perl ages" to run. Then you have the multi user machine where the test passes when the machine is quiet but fails when's there lots of cache contention. You also have a (somewhat rarer) problem with people changing CPUs and not recalibrating their pmips. And the increasingly common laptops with varying clock speeds and voltage stepping etc. If Test::Harness had a protocol for warnings rather than just pass and fail then this would be more useful, F
