Hi, I'd tend to suggest first sticking close to the benchmarks that directly measure OS performance and various services like: IO tests, Network Tests, and Micro-benchmarks ( ex: libmicro). Getting this lower level tests well done would help everyone and haven't been done well before. And it has the benefit of directly telling you what to improve.
The problem with many of the application benchmarks that have been suggested is that often the are very susceptible to application tuning and compiler tuning differences. It also would take a LOT of work to make sure the apps tests are testing what you think they are. SPEC and TPC and ISVs (that have benchmarks) spend huge amounts of time trying to address these issues. Someone suggested Bonnie, but it still has some issues... A nearly single threaded test optimized for single disk performance, it does not work well with modern striped or RAID disks. bonnie++. Has some improvements over bonnie: can disable putc/getc with "-f" flag. Some other thoughts on benchmarks... Postmark A file system benchmark developed by Network Appliance and reputed to emulate the operation patterns of email users and netnews servers. Primary liability is the question of disk vs CPU benchmarking: since postmark tests very small transactions, it can be turned into a fairly heavy CPU benchmark though it claims to be testing IO. As a CPU benchmark, it suffers from being single threaded. As a file system benchmark, it offers limited testing of file system operations (only create/delete and read/append), and operation mix and size is not standardized and thus can be tuned to the system under test. No central publication authority or peer review exists to ensure fair and accurate usage of the benchmark. As a corollary to mpogue's "Most simple benchmarks suck", I'd add "Most complicated benchmarks don't measure what you assume they do" -- Brad (I'm a sun performance/benchmark guy) This message posted from opensolaris.org _______________________________________________ perf-discuss mailing list perf-discuss@opensolaris.org
