Re: 5-STABLE Roadmap

[Brad Knowles]

At 9:47 PM -0800 2003/02/13, Sam Leffler wrote:

     SpecFS (NFS ops/sec benchmark)

 List price on SPEC SFS97 R1 is $900.  And my recollection is that it was
 involved to setup and run.

$450 for educational organizations. Wouldn't the FreeBSD Foundation qualify?

 Benchmarks must be unencumbered; be easy to setup+run by one person; and not
 require lots of equipment.  For the most part we are looking for benchmarks
 that will help tune system performance; not generate press releases.

Some of the more interesting benchmarks take more hardware to properly run. They're not necessarily particularly hard to setup, but they do want client machines to be used to generate test load.

Rick Jones had to use 20 client machines with netperf in order to find the limits of performance for Nominum Authoritative Name Server (ANS), and he works for HP.


I think you need to decide just how thorough you want your testing to be. If it's all going to be just single people running benchmarks on single machines, I think that there are going to be a lot of things you may miss.

I have this problem myself with the benchmarks I've been running in conjunction with the invited talks I did at LISA 2002 and BSDCon Europe 2002, and people have repeatedly called me to task on this issue.

--
Brad Knowles, <[EMAIL PROTECTED]>

"They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety."
-Benjamin Franklin, Historical Review of Pennsylvania.

GCS/IT d+(-) s:+(++)>: a C++(+++)$ UMBSHI++++$ P+>++ L+ !E-(---) W+++(--) N+
!w--- O- M++ V PS++(+++) PE- Y+(++) PGP>+++ t+(+++) 5++(+++) X++(+++) R+(+++)
tv+(+++) b+(++++) DI+(++++) D+(++) G+(++++) e++>++++ h--- r---(+++)* z(+++)

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message