Good proposal goals provide a way to test if the goal
has been achieved and a way to argue if the goal is
worthwhile. "Improve performance" as a goal is both
untestable and impossible to argue with, so it's a
badly stated goal.
Remy Maucherat wrote:
>
> To evaluate code, I strongly recommend using a profiling
> tool instead of benchmarks, as it also helps finding places
> where your code is inefficient.
>
Profiling tools and internal benchmarks answer different
questions. You don't use a profiling tool instead of a
benchmark, you use it with a benchmark.
Talking about performance improvements without also
talking about workload is pointless. Setting up some
standard internal workloads has several advantages:
#1) If forces you to consider what's important and
what's not. In that sense, it's a very precise
communication tool as much as anything else.
It provides a way to nail down your goals. Saying
that "our goal is to improve performance" is like
saying "our goal is to make the software better".
Sure, ok, who could vote against that?
Saying something like "we need a 50% improvement
in request processing time for servlets that produce
large (>50k) pages" provides a testable goal to
shoot for (and an opportunity for other developers
to say "wait, that's not a good goal to shoot for")
#2) It provides a common framework. If you claim a
25% performance improvement, you've told me
nothing. If you claim a 25% improvement on the
helloworld servlet performance test, then I
know what you mean and can try it out for
myself. It's very easy to fool yourself, and a
standard workload acts as a BS detector.
#3) It saves time in the long run, since everyone
doesn't come up with their own mini-internal
performance test-suite. In fact, you generally
start something like this by posting your personal
performance testing setup so others can use it,
and it grows from there. It doesn't have to be
completely formal, it just needs to be acknowledged
as an issue.
--
Christopher St. John [EMAIL PROTECTED]
DistribuTopia http://www.distributopia.com
--
To unsubscribe, e-mail: <mailto:[EMAIL PROTECTED]>
For additional commands, e-mail: <mailto:[EMAIL PROTECTED]>
