See below.

Original Message dated 11/8/00, 12:55:34 PM
Author: "Craig R. McClanahan" <Craig.McClanahan@eng.sun.com>
Re: Re: Msg formatting problem: trying again:

Craig: That's right ... once a servlet is loaded it stays in memory, so there would be little or no I/O in this kind of test scenario. 
Roy: In the old days we called it cacheing, yes? :-)
Craig: NOTE:  Of course, that is one of the issues that leads to caution in interpreting benchmark results as *absolute* indicators of performance -- this is clearly not going to be representative of the performance of a servlet that heavily accesses a database, for example.  However, it is very useful in *relative* performance measurement.  The standard practices of scientific investigation apply ... tweak one variable at a time and see what happens. 
Roy: There is a school of thought that  varying multiple factors and then analyzing their combined effects on performance indicators is a good way to go unless it is known in advance that interactions are not important It was used at IBM for Operating System 360 (ugh) parameter tuning.
Craig: ... you might want to try at least the 1.2.2 and 1.3.0 releases from Sun (on the same hardware) to compare.  That would also give some interesting information on how the new HotSpot in 1.3 (and in particular the "server version") affects things.
Roy: Will do ASAIC 
Craig: One thing that will help in this regard is that Sun is making a machine with some serious hardware (six processors, 3GB of memory, scads of disk) available for performance tuning.  That should help us shake out thread-related bugs and performance problems pretty quickly :-). 
Roy: But, isn't this optimizing for the SERIOUS machine architecture? Is there an implicit target architecture for TC? In the past, it would not have been the SERIOUS machine, would it? (Is the SERIOUS machine part of the LinuxDevelopmentLab?) How would lab performance be translated to outside-the-lab performance? (BTW, I think it can, via modeling along the lines suggested in my presentation that I posted a couple of weeks back).
Roy: And, as all commercial benchmarkers know, it is possible to tweak almost any system to get performance that satisfies some benchmark criteria on a limited set of requests. So, one more time: What workload is Tomcat expected to run, and what will TC thruput and response be running that workload on a particular architecture? Or, since TC is free, this is up to the user to simply try it and see?
Roy
-- 
Roy Wilson
E-mail: designrw@bellatlantic.net